Evaluation of a web-based intervention to reduce antibiotic prescribing for LRTI in six European countries: quantitative process analysis of the GRACE/INTRO randomised controlled trial

Setting and participants

The trial was carried out in 229 practices in England, Wales, Belgium, the Netherlands, Spain and Poland. There were 424 GP usernames recorded on the web-based intervention (most GPs logged in individually, but some logged in as a group), and 346 GPs completed the self-report measures at either baseline, follow-up or both time points (the sample size therefore varies somewhat and is specified for each analysis). Of the 4,264 patients recruited to the study, 2,886 (67.7%) completed the self-report measures analysed here.

Interventions

The web-based training was developed using the LifeGuide software [27], which allows development and easy modification (including translation) of web-based interventions without the need for programming. The web-based training consisted of a single session that drew on existing successful theory-based interventions to reduce antibiotic prescribing in primary care [17–19]. The session comprised three sections; an introduction, a module providing training in using a C-reactive protein point of care (CRP) test, and a module providing training in communication skills and use of a patient booklet. The communication skills training followed the STAR (Stemming the Tide of Antibiotic Resistance) model [17], aiming first to persuade GPs why a reduction in prescribing was necessary and then how it could be accomplished. The STAR model proposes that there are three elements of an effective consultation: to gather information, exchange information, and check information. Building on a recent successful website and booklet intervention to reduce prescribing for childhood respiratory infection [18], the GRACE/INTRO training also illustrated (using videos) how a patient booklet could be used in the consultation to address specific patient concerns; GPs were encouraged to use tick boxes in the booklet to highlight specific sections relevant to individual patients. The content of the patient booklet drew on previously validated content [28] that addressed perceptions of symptoms and antibiotics based on the extended Common Sense Model [29, 30], which describes the dimensions of symptoms salient to patients (i.e., identity, cause, duration, severity of consequences, and potential for control or cure) as well as the salient dimensions of medication (i.e., perceived need and potential for harm). The materials were piloted in every country and modified according to feedback from interviews with health professionals and patients in each country, allowing small between-country differences in the website where this seemed advisable [31].

To encourage reflection on intervention implementation, during the training phase GPs were asked to document the presentation, management and issues raised by consultations for up to 10 cases in which the GP had attempted to implement the intervention. Where feasible, GPs were encouraged to participate in a one-hour seminar to share these experiences, and upload summaries of these discussions to an online forum.

The website was tailored to intervention group; all GPs in the intervention groups could access the introduction, but only GPs in practices randomised to the relevant groups could view the training module relevant to their intervention arm (i.e., CRP and/or communication skills). Further details of the intervention are given elsewhere [6]; the website can be viewed at https://www.lifeguideonline.org/player/play/intro_demo, and the patient booklet is provided as Additional file 1.

Procedure

GP practices were cluster randomised to intervention group (for details of trial procedures, see trial paper [6]). GPs in the three intervention groups were sent a website address to login to the intervention, and were required to complete the baseline self-report measures when they logged on, before they could view the intervention. The post-intervention survey was administered to GPs in all groups by sending them an email invitation with a website address for completing the survey online (plus emailed reminders to non-respondents). Self-report measures were completed by patients at home as part of an illness diary, pro forma (returned by post), or a brief telephone-administered questionnaire.

GP measures

Attitudes were assessed before and after the intervention by four items rated agree/disagree on a 7-point scale. Based on previous research into common reasons for prescribing [7, 10–13], these items measured whether GPs thought reducing prescribing was a) important, b) risky, and c) could damage their relationship with patients, and how confident GPs were that they could reduce their prescribing. The post-intervention survey also asked whether GPs felt that taking part in the intervention had helped them to reduce their prescribing. The Website Satisfaction Questionnaire [28] comprised three items rated agree/disagree on a 10-point scale, that assessed whether GPs found the website helpful and trustworthy and the advice it provided sufficient. Self-report items post-intervention assessed whether GPs completed the online training individually or as a group and how they engaged in the reflective exercise. GP receipt of the intervention was measured objectively by the total duration spent viewing the intervention webpages.

Patient measures

For simplicity, all patient self-report attitude measures were rated on a 10-point scale (strongly agree/strongly disagree). Patient perceptions of antibiotics were assessed by two items based on the two key dimensions of treatment perceptions in the extended Common Sense Model [29]: perceived necessity (‘It is usually necessary to take antibiotics to clear up a chesty cough’) and perceived harm (‘Taking antibiotics for a chesty cough can do more harm than good’). Patients were asked whether they had received a booklet and whether they found it useful. Patients then completed the Patient Enablement Instrument [32] and a three-item Consultation Satisfaction Questionnaire that assessed whether they felt they had received all the information and advice they needed and were generally satisfied with the consultation.

Analysis

Since many self-report items were assessed by a single item, we did not replace missing data, but instead give the specific sample size for each analysis. All the scales employed had good internal reliability (Website Satisfaction Questionnaire, alpha = 0.93, n = 230; Patient Enablement Instrument, alpha = 0.92, n = 2,847; Consultation Satisfaction Questionnaire, alpha = 0.93, n = 2,888).

Effects of the intervention were analysed using General Linear Modelling (PASW statistics version 18) with univariate, multivariate or repeated measures as appropriate. Country differences were examined as post hoc exploratory analyses also using General Linear Modelling; the results could not be reported for every analysis but are given for key analyses and where significant variations between countries were found.

GP intervention receipt and attitudes

The sample who completed the post-intervention survey comprised 147 men and 199 women with a mean age of 42.25 (s.d. 8.87), who had practised for a mean of 19.22 (s.d. 9.63) years. This sample represented 93.0% of the 372 GPs who supplied at least one case in the trial.

Most respondents reported completing the website training alone (189/230, 82.2%), but the remainder completed it as a group. The mean time members of the intervention groups spent on the website was 35.52 mins., with considerable variation (s.d. 28.12 mins.). Time spent on the website differed between groups [F (2,310) = 6.05, p =0.003], which was predictable as the communication skills training (26 core pages) was longer than the CRP training (15 core pages). The CRP group had the shortest duration (26.54 mins., s.d. 20.50), differing significantly on post hoc group comparisons from the communication group (37.44 mins., s.d. 28.94) and combined group (mean = 39.76 mins., s.d. 30.50). Of the 159 GPs who responded to the post-intervention question about how their seminar was organised, most took part in a practice-based seminar with multiple GPs (70/159; 44.0%), some engaged in self-reflection alone (28/159; 17.6%), a minority of practices met together (44/159; 27. 7%), and the remaining practices held a multi-practice teleconference.

In the post-intervention survey there were clear group differences in overall perceptions of the extent to which taking part in the study had helped GPs reduce their antibiotic prescribing [F (3,287) = 11.06, p <0.001]; those in the communication and combined groups had the highest scores, and those in the control group the lowest (see Figure 1), with those in the CRP group intermediate. There were significant between-country differences in perceptions of study helpfulness [F (5,293) = 3.37, p = 0.006] and the helpfulness of the CRP test [F (5,150) = 6.89, p <0.001], though not in the perceived helpfulness of the booklet. Perceptions of the helpfulness of the study and especially the CRP test were highest in Spain and Poland (see Table 1). However, it is important to note that these findings are based on small numbers of respondents, particularly in Poland.

Patient intervention receipt and attitudes

The patient sample comprised 1,024 (35.5%) men and 1,862 (64.5%) women with a mean age of 57.01 (s.d. 22.12). Of those in the communication and combined groups, 1,514/1,804 (83.9%) reported having been given a GRACE/INTRO booklet, most of whom also reported having used it (1,335/1,718; 77.7%). A third of those in the control and CRP groups (503/1,520; 33.1%) also reported having been given a booklet of some kind (presumably as a normal part of routine care), and a similar proportion (445/1,334; 33.4%) reported using it. Ratings of how useful the booklet was (where 7 indicates maximum usefulness) were relatively high in members of the communication groups who received the GRACE/INTRO booklet (mean = 5.69, s.d. 1.05), and were slightly higher [t (1,2034) = 6.99, p <0.001] than ratings for the booklets provided in the other groups (mean = 5.30, s.d. 1.18). According to the GP records, just over a third of patients in the CRP and combined groups received the CRP test (884/2,357; 37.5%).

Strengths and limitations

A strength of this study is that we used a theory-based approach and were able to document changes in beliefs and attitudes that have been shown to be relevant to GP and patient behaviour. A limitation of our design was that the control group did not access the website at baseline, and so their attitudes were not assessed pre-intervention. In addition, it was essential to minimise the burden on participating GPs and patients, and so only a few of the many potentially relevant constructs could be assessed, and we were obliged to use single items to measure some constructs (resulting in unknown measurement reliability for these measures). However, we also carried out qualitative process studies with both GPs and patients, and we note that the conclusions from our inductive qualitative research converge with the conclusions from the deductive quantitative research reported here (unpublished observations; papers in preparation).

It seems likely that there are complex multidirectional relationships between prescribing rates and the GP and patient attitudes we measured, and causal relationships could not be inferred from this study since attitudes were measured post-intervention. Although many of our findings were statistically significant, the effect sizes we observed were small. We were unable to analyse whether specific changes in GP attitudes mediated the effect of the intervention on prescribing rates because prescribing rates for individual GPs were not assessed prior to the intervention.

An innovative feature of the study is that it was carried out in six different countries, providing potentially useful information about the extent to which implementation of these interventions is likely to be influenced by local context. No negative views of the interventions were detected, but lower GP response rates in some countries (especially Poland) make it difficult to be sure that responses were representative of all countries. The practices that participated in this trial may not be typical and may have over-represented GPs who were particularly enthusiastic about reducing prescribing – although to minimise this risk, our recruitment criteria specified that only practices that had not previously taken part in a study of reducing antibiotic prescribing could participate, and prescribing in the baseline audit and the control group did not appear atypical.

Implications for research and practice

Overall, these findings suggest that the interventions should be broadly acceptable to both GPs and patients across a range of European healthcare contexts and may be feasible to roll out more widely across Europe. The effectiveness of our interventions across quite different European countries might appear surprising given that different factors can influence prescribing in the different healthcare contexts; for example, a problem in some southern European countries is that patients can obtain antibiotics independently, while a problem in some Eastern European countries is the lack of clear prescribing guidelines [40]. However, there are also many shared influences on prescribing [14, 15, 40], and our process of co-design of the interventions by clinicians from every country, with minor country-specific modifications where necessary, may have been important. It is also possible that different elements of our complex interventions may have been effective in different countries or for different GPs; for example, the GP education element may have been more important for some and the patient booklet element more helpful for others.

There were some indications that the communication intervention might help to engender changes in GP and patient attitudes that will be helpful in the longer-term, such as increased awareness of the potential disadvantages of antibiotics and increased confidence to manage LRTI without them. There is a potential for experience of positive outcomes without antibiotics to reassure both doctors and patients that antibiotics are often unnecessary, which could reduce prescribing by doctors and also future consultation by patients [16, 18, 41]. However, there is also a possibility that patients could consult more often based on the safety-netting advice in the patient booklet [ref trial] or if they believe it is important to be given a CRP test to exclude serious infection [35, 36]. A recent longitudinal follow-up of an intervention comparing the effects of communication skills training and use of the CRP test found that only communication skills training had longer-term effects on prescribing, while neither intervention significantly affected consultation rates [42]; however, further research is needed to confirm the longer-term effects of both interventions on prescribing and consultation rates.

Given that the CRP test was very effective in reducing antibiotic prescribing but had less positive effects than the booklet on patient beliefs and attitudes, it seems logical to extend the use of the patient booklet to patients receiving the CRP test, as well as those for whom the CRP test is not appropriate. The CRP test alone does not appear sufficient to maximise patient satisfaction and enablement, whereas receiving a booklet is likely to address patient concerns and provide effective reassurance [10, 41]. In a previous study, patient satisfaction levels were higher in patients receiving the CRP test when those with intermediate test results were given a delayed prescription [36]. Future research could examine whether adding use of a delayed prescription to the CRP test and booklet (when appropriate) might optimise antibiotic use, patient satisfaction and re-consultation rates.