We delivered a multifaceted, tailored, intervention aimed at increasing referrals of women with mild to moderate PND for psychological treatments as a first stage treatment. Our interrupted time series analysis on monthly referral for psychological treatments data suggests the impact of our intervention on referrals had a short-term 11% effect that was not sustained over the ten-month post-intervention period. An additional interrupted time series analysis run on antidepressant prescribing rates also indicated no impact, with rates remaining high post-intervention.
The finding that the increase in referrals for psychological treatments was not sustained was disappointing. The intervention was multifaceted to try and ensure that all three factors identified from our diagnostic analysis were targeted, and to increase the likelihood of the health professionals being exposed to at least one of the components, with an appreciation that not all health professionals would engage with all components. In example, while the educational meeting, targeting all three key factors, was run twice to facilitate attendance, only 44 GPs attended. The educational materials, targeting factors one and two, were distributed as both electronic and hard copies, but still required recipients to engage with the content. The reminder system, targeting factor one, whereas, was embedded within the electronic recording system used by all GPs and nurse practitioners and was anticipated to produce more lasting effects than the other components due to this. There is currently, however, a lack of sound evidence available to guide the selection of intervention components to target specific barriers , let alone to predict duration of effect. For example, it is possible that the impact of the intervention could have been more sustained had an entirely different type of component been included (for example, opinion leaders), with some evidence to suggest that educational meetings-as a single component- may not be effective for changing more complex behaviors .
The time series analysis run on the percentage of first drug prescriptions found no significant effect on anti-depressant prescribing, with rates remaining high post-intervention. Implementation research typically seeks to increase adoption of innovations. However, in order to increase the adoption of certain innovations, in some instances other competing clinical actions need to be dis-adopted . We aimed to increase adoption of the recommendation that women with mild to moderate PND should receive referral for psychological treatments. Our three intervention components promoted adoption through targeting influential factors identified from our diagnostic analysis. Our implementation strategy might have been more effective, however, if it had also directly challenged the health professionals’ attitudes towards anti-depressant prescribing. Future implementation studies may benefit from a more holistic approach, promoting adoption of new, targeted, innovations and dis-adoption of competing clinical actions where appropriate.
A limitation of the study is the small number of PND cases diagnosed each month; this averaged 33 but increased over time, suggesting a potential increase in awareness of PND amongst either patients or the health professionals. There is, however, no formal guidance regarding the appropriate number of cases making up each data point in a time series analysis. The number of cases each month, however, may have made the analysis more vulnerable to exaggerated peaks and troughs in the data that were difficult to model robustly: the mean percentage of model error reported in the model fit statistics was high. We considered combining the data into two monthly intervals, but this would have resulted in loss of data points, particularly in the post-intervention period. We also recognize that the number of health professionals interviewed for the qualitative element is small, but it was not intended to provide a second outcome measure to corroborate the ITS analysis. Rather, we sought to provide some explanation for the outcomes identified. Further rich exploration and analysis would be needed to fully understand the way in which messages have been received, understood and applied.
A mediational analysis would have enabled a more precise understanding of intervention impact, comparing pre-and post-intervention survey measures across each of the targeted factors. This had been planned; however, a particularly low response rate for the baseline survey  would have made any analyses underpowered, and so the pre-intervention questionnaire was not re-administered post-intervention to avoid wasting health professionals’ time completing a survey that we could not analyze robustly. In not re-administering the survey, we, however, also lost the opportunity to collect self-report fidelity measures that had been designed to be added into the survey, and asked about extent of engagement with the intervention to enable subgroup analyses at the health professional level by degree of engagement. An attempt was, instead, made to obtain fidelity measures for health professionals’ receipt of the educational materials and their accessing of the template changes. This was through the system used in the host site to distribute electronic documents, and through the electronic recording system that the template/reminder system was embedded in, with an interest in identifying those health professionals who had engaged with these two components. However, limitations of the two systems meant that this was not possible. Similarly, to substantiate the finding from the qualitative interviews that the intervention may have resulted in more appropriate prescribing of anti-depressants post-compared with pre-intervention, data were downloaded on the types of drugs prescribed to enable a statistical test of difference in prescribing patterns. However, system limitations again prevented this, with the searches run on the local electronic recording system requiring overly complex combinations of ‘if, then’ rules that resulted in the returned data lacking robustness. Overall, the lack of formal fidelity measures, beyond attendance at the educational event, and the aggregated nature of our two time series outcome measures meant we are unable to reliably assess the extent to which the intervention was delivered as planned, and whether this impacted on intervention effectiveness. Future research, therefore, should build in fidelity measures at the outset, to enable these relationships to be explored.
Our findings do, however, illustrate the importance of using multiple outcome measures. If we had only analyzed referrals data, the lack of impact on prescribing rates, and professionals’ simultaneous referring and prescribing behavior would not have emerged. Although limited in number, the interviews provide some insight into intervention effectiveness, suggesting that the lack of change in prescribing behavior may be due to mixed messages received from the intervention. For example, it is possible that our attempts to ensure that our messages were balanced with recognition of the need for stepped care led to some participants focussing on mixed psychological and anti-depressants treatments, and some taking home a message reinforcing use of anti-depressants. The interviews also provide some suggestion that the intervention messages may not have been adequately targeted by our intervention; with some of the barriers cited mirroring those identified through our diagnostic analysis.