Skip to main content

Effectiveness of educational outreach visits compared with usual guideline dissemination to improve family physician prescribing—an 18-month open cluster-randomized trial



Educational outreach visits are meant to improve the practice of health professionals by promoting face-to-face visits to deliver educational contents. They have been shown to change prescription behavior, but long-term effects are still uncertain. This trial aimed to determine if they improve family physician prescribing compared with passive guideline dissemination.


Parallel, open, superiority, and cluster-randomized trial. National Health Service primary care practices (clusters) were recruited in the Lisbon region—Portugal between March 2013 and January 2014. They could enter if they had at least four family physicians willing to participate and not planning to retire in the follow-up period. Three national guidelines were chosen for dissemination: acid secretion modifiers, non-steroidal anti-inflammatory drugs, and antiplatelets. Physicians in the intervention group received one 15 to 20 min educational outreach visit at their workplace for each guideline. Physicians in the control group had access to guidelines through the Directorate-General for Health’s website (passive dissemination). Primary outcomes were the proportion of COX-2 inhibitors prescribed within the NSAID class and the proportion of omeprazole within the PPI class at 18 months after the intervention. A cost-benefit analysis was performed. Practices were randomized by minimization. Data analyses were done at individual physician level using generalized mixed-effects regression models. Participants could not be blinded.


Thirty-eight practices with 239 physicians were randomized (120 to intervention and 119 to control). Of 360 planned visits, 322 were delivered. No differences were found between physicians in the intervention and control groups regarding the proportion of omeprazole prescribed among PPIs 18 months after the visit (46.28 vs 47.15%, p = 0.971) or the proportion of COX-2 inhibitors among NSAIDs (12.07 vs 13.08%, p = 0.085). All secondary outcome comparisons showed no effect. There was no difference in cumulative drug costs at 18 months (3223.50€/1000 patients in the intervention group and 3143.92€/1000 patients in the control group, p = 0.848).


Educational outreach visits were unsuccessful in improving compliance with guideline recommendations among Portuguese family physicians. No effects were observed at 1, 6, and 18 months after the intervention, and there were no associated cost savings.

Trial registration NCT01984034. Registered 7 November 2013.

Peer Review reports


Clinical practice guidelines have the potential to improve the quality of care by summarizing current medical knowledge and promoting interventions with proven efficacy, safety, and cost-effectiveness [1]. Yet, issuing guidelines does not guarantee changes in clinical practice, as clinicians may not follow them for a variety of reasons. Among them are not being aware of or familiar with guidelines, considering they are ambiguous or disagreeing with recommendations, perceiving lack of self-efficacy, organizational constraints, and patient barriers [2, 3]. This may contribute to the problem of translating new medical knowledge to improvements in public health and of affordability of care [4, 5]. To address this problem, several guideline implementation strategies have been tried, with a systematic review finding small to moderate effects in a majority of trials [6].

Educational outreach visits are a type of strategy aimed at improving clinical practice [7, 8]. They consist of face-to-face visits done by an individual (henceforth named a detailer), usually a healthcare professional trained in communication skills, to physicians or other health workers in their own setting. During the visit, the detailer enquires about the physician’s baseline knowledge and motivations, presents content prepared by an independent organization with clear educational goals and using concise and graphic appealing materials, all the while stimulating physician interaction and providing positive reinforcement [9]. Visits are to one individual or a small group, in contrast with large educational meetings. A systematic review concluded that educational outreach visits have a small, but consistent and potentially important, effect on prescription improvement [8]. It also highlighted a gap in knowledge about the performance of this type of intervention in the long term (beyond 1 year). Trials using multifaceted interventions that included educational outreach to reduce inappropriate prescribing showed sustained effectiveness 1 year after the intervention [10,11,12], but it is unclear if educational outreach alone can achieve the same effect.

Portugal has a publicly funded National Health Service (NHS) with universal coverage providing the majority of primary care [13]. Primary care services are organized in small local practices, with 4 to 12 family physicians, plus a roughly equal number of nurses and a smaller number of secretaries [14]. In some practices, physicians are paid a fixed salary (personalized care units), while in others, they can have pay for performance incentives (family health units). Patients are registered with a single family physician, but may visit other physicians within the same practice if theirs is unavailable. Physicians in a practice meet frequently to discuss organization of care and performance indicators (which include items on prescribing). The Directorate-General for Health (an agency of the Ministry of Health) is responsible for issuing prescribing guidelines [15, 16]. Guidelines are made available on the agency’s website, and all health professionals are supposed to abide by them. This makes all physicians in Portugal exposed to the guidelines simultaneously. Guidelines are also available contextually on the Ministry of Health’s prescribing software and can be used as part of local clinical audits. However, their effectiveness in changing actual practice has not been studied.

In Portugal, family physicians work in group-based practices. Although educational outreach visits are directed at individual physicians, contamination among physicians working in the same practice is a plausible concern. In addition, a public health program of educational outreach visits would probably be delivered to all doctors in the same practice to minimize costs and loss of detailer time traveling. Thus, it may be more reasonable to assess educational outreach visits in the context of Portuguese primary care using a cluster-randomized design.

The primary objectives of the Trial to Assess the Effectiveness of Educational Outreach in Prescription Guidelines (TEP trial) [17] were to determine if educational outreach visits, compared with passive guideline dissemination, resulted in a reduction of the proportion of COX-2 inhibitors prescribed among the NSAID class and an increase in omeprazole prescriptions among the PPI class 18 months after the visit (long term). Secondary objectives were the effects on the same drugs at 1 (short term) and 6 months (medium term), and the short-, medium-, and long-term effects in the prescription of clopidogrel, thus the duration or persistence of effect post intervention. The trial also aimed to determine the cost-benefit of educational outreach visits.


The protocol for this trial has been published previously, along with a PaT plot, and a cascade diagram [17].

Trial design

The TEP trial aimed to determine the long-term effectiveness of educational outreach visits and their cost-benefit relation. It was a parallel, open, superiority, cluster-randomized controlled trial conducted in Portuguese primary care physicians. Clusters were Portuguese NHS primary care practices.


The trial recruited family physicians working in NHS practices of the Lisbon region, Portugal, between March 2013 and January 2014. A practice would be eligible to participate if it had at least four physicians. All family physicians were eligible, except those planning to retire or without a stable patient list. Participants were recruited through practice coordinators. Family practices were the units of randomization. There was no financial incentive to participation. Participating physicians completed a baseline characteristic questionnaire and consented to schedule educational outreach visits as well as to the collection of their aggregate prescription data.


A detailed description of the intervention is provided in Additional file 1 as a TIDieR checklist [18].

Three guidelines were chosen for dissemination: non-steroidal anti-inflammatory drugs (NSAID), acid secretion modifiers, and proton pump inhibitors (PPI) and antiplatelets [19,20,21]. These guidelines had been issued by the Directorate-General for Health to address inappropriate prescribing and high spending in these drug classes. Physicians randomized to intervention clusters received three educational outreach visits during a 6-month period. Key messages were identified in each guideline. The NSAID guideline advocated for less use of COX-2 inhibitors, recommending they would only be prescribed to patients with increased gastrointestinal risk who did not tolerate a classical NSAID with a gastroprotective agent. For acid secretion modifiers, the guideline recommended that omeprazole should be preferred as it was as effective as other proton pump inhibitors and less expensive. The antiplatelets guideline recommended less use of clopidogrel, which should not be maintained long term after myocardial infarction, acute coronary syndrome, or percutaneous coronary intervention. Thus, we had a diverse mix of objectives aimed at improving rational prescribing: in one case to increase the usage of a specific drug and in two other cases to decrease drug usage.

Each visit was planned to focus on one guideline, last from 15 to 20 min and had one family physician present (up to three were allowed if physicians preferred, but one-to-one visits were encouraged). The visit would begin with an introduction about the detailer and the purpose of the visits, confirming the physician’s availability. Then educational needs would be assessed by asking about the physician’s usual practice with open questions. These would shape how the detailer would deliver key messages about the guideline, addressing scientific evidence, and benefits of following the guideline, barriers, and facilitators of change. The physician would be given the opportunity to present objections, which were addressed by the detailer. The visit ended with a summary and encouragement for the physician to commit to change. A point of care summary was distributed with each visit, and a brochure was used by the detailer as a visual aid. Copies of the brochures and point of care summaries are made available in Additional file 2.

Whenever possible, a single detailer performed all three visits to the same physician. Visits could take place in between patient visits or at other times indicated by the physician. Visits could be rescheduled up to the day before they were planned, but if the physician was unable to attend and could not warn the detailer beforehand, that visit would be skipped. Detailers filled a short questionnaire at the end of each visit, included those that were not made.

The detailers were three members of the research team (two family physicians and one academic pharmacologist) and nine physicians that were trained for the trial (six family physicians and three family medicine residents). All detailers were trained on the principles of educational outreach and the contents of each visit, to ensure consistency.

For the control group, usual guideline implementation consisted of passive dissemination by their publication on the Directorate-General for Health’s website.


The trial had two primary outcomes, measured at the physician level: the proportion of COX-2 inhibitors prescribed within the NSAID class and the proportion of omeprazole within the PPI class, both measured in defined daily doses (DDD) at 18 months after the intervention. There were seven secondary outcomes: the same proportions of COX-2 inhibitors and omeprazole measured at 1 and 6 months after the intervention and the number of clopidogrel DDD per 1000 registered patients at 1, 6, and 18 months after the intervention.

We also conducted a cost-benefit analysis using the sum of all prescriptions dispensed for NSAIDs, acid suppressive therapy (PPIs, H2-receptor antagonists, misoprostol, and reimbursed anti-acids), and clopidogrel from month 1 following the intervention until month 18. Costs were considered from the perspective of a government-supported program, with government as the payer; therefore, only the reimbursed portion of drugs was considered. Differences in costs between the intervention and control group would be compared with the cost of training and paying detailers, preparing and printing educational materials, program coordination, and indirect costs of physician time (spent with a detailer rather than seeing a patient).

Only prescriptions that were dispensed were counted. Drug dispensing and cost data was provided by the Lisbon Regional Health Administration. In the Portuguese health system, prescriptions can be made for acute or chronic conditions. The former are valid for dispensing within 30 days and the later for 6 months. Of the studied drugs, only NSAIDs cannot be prescribed for chronic use. Physicians who transferred to other practices within the health region were followed, and their prescriptions monitored. When prescription data was not available, the last known month’s prescription was used.

Sample size

Pilot data was obtained from three primary care practices and was used to estimate within unit variability and the intra-cluster correlation coefficient. Aggregate data from the Regional Health Administration was used to estimate the mean prescription and standard deviation for primary outcomes. Our sample size was calculated assuming the intervention would lead to a 5% absolute difference in compliance with guidelines between intervention and control units for primary outcomes, a mean cluster size of six physicians per practice, a 1:1 allocation ratio of controls per intervention unit, an alpha type error of 0.025, and a dropout rate of about 15%. To achieve 80% power, a sample of 110 physicians in each group was needed. To recruit the necessary 220 physicians, 38 primary care units would be required. STATA 12.0 (STATA Corp, TX, USA) and its sampsi and sampclus commands were used to calculate a sample size.


Clusters were allocated to the intervention or control groups using minimization, a method to achieve good balance regarding baseline characteristics that could influence the outcomes when the number of clusters is small [22]. We stratified for number of physicians in a practice, median baseline prescription of COX-2 inhibitors and omeprazole (above or below the regional median), proportion of physicians with fewer than 10 years of practice after completing vocational training, and type of primary care practice. The sequence of intervention visits for each practice was determined by simple randomization using sequence generator [23].

Allocation was concealed by having the project manager assign a sequential number to each practice as it completed enrollment. The trial statistician received only anonymized data (sequential number and minimization variables), blindly allocated practices to each trial arm, and returned allocation information to the trial manager.

Neither participating physicians nor detailers could be blinded. Outcomes were collected independently from the researchers by the Regional Health Administration and were only provided after the intervention had ended. Unlike what was planned in the protocol, the lead author could not be blinded to group and visit sequence allocation because the Regional Health Administration needed to consult with study author for data extraction. However, we were able to keep the trial statistician blinded.

Statistical methods

Analysis was performed using the intention to treat principle. Physicians who transferred to another unit in the region were followed. For those we were unable to retrieve prescription data (transferred to another region or stopped working), we used the last working month’s prescription. Outcomes in both groups were compared using generalized mixed-effects multi-level regression models using the primary care practice to account for clustering. For model fitting, proportions of omeprazole and COX-2 inhibitors were logarithmically transformed [ln(x/(1 − x)] because of non-normal distributions. Intra-cluster correlation coefficients were calculated for primary outcomes. Statistical significance was assumed for a p value less than 0.025. STATA 12.0 (STATA Corp, TX, USA) was used to conduct the analysis. No interim analyses were done.

Ethical approval

The trial was approved by the ethics committees of the Lisbon Regional Health Administration and NOVA Medical School. Family physicians invited to participate received written information about the main aspects of the trial, and participants gave consent for researchers to access their prescription data. The trial only collected aggregated and non-identifiable patient data.

Given the intervention-posed minimal risks to patients, no data monitoring committee was established and no stopping guidelines were defined.



Recruitment began in March 2013 and ended in January 2014. Participant flow chart is shown in Fig. 1. The research team met with practice coordinators from 13 of 15 health center groups in the Lisbon Region, representing 233 practices. Of these, 193 did not reply to subsequent contacts, were unwilling to participate, or self-excluded for having less than four physicians willing to participate. We randomized 38 clusters with 239 participating physicians.

Fig. 1
figure 1

Participant flow diagram

Baseline characteristics for participants in the intervention and control groups are shown in Table 1. As expected, groups were balanced regarding the characteristics used for minimization. However, the number of patients per physician was higher for the intervention group.

Table 1 Baseline characteristics of participating physicians

Delivery of the intervention

The intervention began in January and ended in June 2014. Of the 360 planned visits, 322 (89.4%) were done. Thirty visits failed because the physician was absent (21), unavailable (5), and on leave (2) or because the detailer was unavailable (2). Two physicians had none of the planned visits, as they were on extended leave. One physician was unavailable for the first visit and chose not to receive further visits. No physician withdrew consent to participate in the prescription analysis. Only one target physician was present in 89.1% of completed visits. The three visits were all made by the same detailer in 88.3% of physicians. Detailers reported delivering the full educational content in 97.8% of visits.


Participating physicians were followed from January 2014 to December 2015. Prescription data was available for both primary outcomes in 96.7% of physicians in the intervention group; in the control group, it was available in 94.1% physicians for PPIs and 90.8% for NSAIDs. Secondary short-term outcomes (1 month after the intervention) were available for 98.3 and 99.2% of the control and intervention groups, respectively, and medium term (6 months after the intervention) for 96.9 and 97.6%. Overall, 29 physicians had one or more months without prescription data for one of the studied drugs (16 in the control group and 13 in the intervention group). Three physicians in the control group and two in the intervention group had no prescriptions of any of the studied drugs in the final 6 months of the study. For the remaining physicians, there were prescriptions after one or more months without data, suggesting temporary absences and not losses to follow-up.

Intervention effects

The results of the intervention are shown in Table 2 and Fig. 2. There were no differences between the intervention and control groups regarding primary outcomes: proportion of omeprazole among PPIs and proportion of COX-2 inhibitors among NSAIDs at 18 months after the intervention. The intra-cluster correlation coefficient was 0.305 (95% confidence interval 0.177–0.473) for omeprazole prescriptions at 18 months. No significant intra-cluster correlation existed for COX-2 inhibitors at 18 months. There were also no significant differences in secondary outcomes, in total costs or class-specific costs for the period between 1 and 18 months after the intervention.

Table 2 Prescription and cost of the studied drugs at 1, 6, and 18 months after the intervention
Fig. 2
figure 2

Prescription timeline of drugs in primary and secondary outcomes (vertical bars) from 1 month before the intervention (− 1) to 18 months after the intervention (18) in the intervention (black line) and control (gray line) groups

As there were no differences in costs between groups, we did not perform a formal cost-benefit analysis.


Summary of main results

Our results showed no effect of educational outreach visits to align prescribing of family physicians with national guideline recommendations. Neither short-, medium-, nor long-term effects were seen, and no differences in drug costs were found between the intervention and control groups.

Strengths and limitations

The planned intervention was adequately implemented: educational visits were delivered with a very high completion rate and were accepted by most participants, and the planned educational content was provided. The intervention was similar to what would have been a government-funded educational outreach program. Outcome data was collected from an administrative database, hence independently from the researchers and with few missing data points. We hypothesize that missing data points correspond to the absence from work due to illness, maternity or paternity leave, or other unforeseen causes and that it is likely that data is missing completely at random and not related with being assigned to the intervention or control groups. We could gather long-term follow-up data for most physicians, which allowed the study of the intervention’s effects over time. Only five among 239 physicians had no data points for outcomes in the last 6 months and may have been lost to follow-up (although we cannot be certain these were not temporary interruptions). Costs were considered for the whole pharmacological class and not just one drug (as costs could have been transferred between drugs within a class). Having the order of visits randomized between clusters allowed us to exclude effects of a possible build-up of the detailer-physician relation. Finally, results are consistent across the different drug classes studied.

The major limitation for the trial is that there was likely selection bias for participation. When compared with regional data, recruitment was higher among family health units than among personalized health care units, and the observed baseline proportions of COX-2 inhibitors and omeprazole were lower than in our pilot data for the region [17]. This suggests that physicians participating in trial had less margin for improvement. Our strategy may not have been effective because all physicians that volunteered to participate were already familiar with the guidelines (passive dissemination could have been enough), and we did not enable the intervention group to adequately address other existing barriers to implement the recommendations. Tools like patient handouts could have been provided to complement point of care summaries for physicians and help them change prescriptions. Other components could have been added to the intervention, such as audit and feedback on prescriptions. However, multifaceted interventions have not been shown to be superior to single-component interventions in changing health professionals’ behaviors [24, 25]. Our choice of drugs to be targeted by the intervention may have introduced influences from other specialists on family physicians’ prescriptions, limiting their ability to change a prescribed treatment [26]. Other drug classes, such as antibiotics, may be less susceptible to these influences, as prescriptions are one-off.

Although groups were balanced in most characteristics selected for minimization, there was some imbalance regarding patient list size. This certainly influenced absolute costs and is why we present costs per 1000 registered patients. It is uncertain if it could have influenced prescriptions, as baseline levels for the drugs of interest are similar. Another limitation was known from the outset and relates to prescription data. We used prescriptions that had been dispensed at a community pharmacy and not all prescriptions issued by the participating physicians. This was because information on issued prescriptions is less reliable in the database, and our pilot data showed a large percentage of prescribed drugs were never actually dispensed (due to factors like lack of patient adherence, mismatch between the patient’s needs and the number of prescribed packages, loss of prescriptions, and errors when issuing the prescription) [17]. Dispensing can occur up to 30 days after a prescription is issued for NSAIDs and up to 6 months for PPIs and clopidogrel. Hence, part of the prescriptions dispensed for PPIs and clopidogrel had not been issued in the same or the previous month.

Another issue to consider is the relatively large random monthly variation in prescription outcomes, which suggests our sample size may have been small to handle physicians with low prescription volume. Considering our choice of when to measure short-, medium-, and long-term outcomes is somewhat arbitrary, this may have been an issue when measuring short- and medium-term outcomes, where this fluctuation is more apparent for COX-2 inhibitors and clopidogrel.

Interpretation of the results

A meta-analysis of educational outreach visits showed a median-adjusted risk difference in compliance with a desired prescribing practice of 4.8%, concluding educational outreach visits had an effect on prescribing, although heterogeneity was noted [8]. Since then, educational outreach visits have continued to yield heterogeneous results regarding effectiveness [27,28,29,30,31]. Educational outreach visits have also been integrated as part of multifaceted interventions, likewise with mixed results [10,11,12, 32,33,34,35,36].

Our study aimed to contribute to one of the unanswered questions in previous literature: if visit performance deteriorates in the long term. However, we were unable to find any short-, medium-, or long-term benefits of educational outreach visits. One possible reason for lack of effect is that physicians more interested in improving their prescribing behavior self-selected for the trial, and baseline prescription was already more compliant with guidelines than in physicians who did not volunteer to participate. It is possible that we did not target what Soumerai describes as “high-potential” physicians—those with prescribing patterns more distant from guideline recommendations; hence, more potential to change [9, 37]. Given the control group was also exposed to the same guidelines, it is also possible the intervention was not so effective when used as an add-on to existing dissemination strategies. However, passive dissemination of guidelines had been shown to be inferior to educational outreach visits and multifaceted interventions that included educational outreach [38, 39]. The Portuguese context may have been important regarding the effects of the intervention, as primary care physicians are government employed, their performance is monitored through quality and spending indicators, and the National Health Directorate’s position on guidelines is normative in nature [13].

Our intervention asked physicians to change a current behavior, promoting the increase in omeprazole use (instead of other proton pump inhibitors) and the decrease in use of COX-2 inhibitors (to be replaced by other NSAIDs) and clopidogrel (to be replaced by aspirin). Physicians would have to stop something they were already doing, in one case increasing and the other two decreasing prescriptions. De-adoption of low-value clinical practices may be different than adopting new activities, [40] and physicians may respond differently when asked to increase or decrease prescription of a given drug. Our trial did not show an effect on either type of prescription behavior.

Not having shown an effect in short- and medium-term outcomes limits our ability to conclude on the long-term effects of educational outreach visits. Even though there was no statistical significance, we could observe some separation between groups in NSAIDs between 2 and 5 months after the intervention and in clopidogrel between 5 and 8 months, both favoring the intervention group. A larger delay for clopidogrel is consistent with the fact that they are classified as chronic prescriptions which are valid for 6 months. However, even these small differences seem to disappear by the end of the follow-up period. This suggests that, even if we had found positive results in the short and medium term for the main outcomes, effects would deteriorate over time.

Costs throughout the follow-up period were not significantly reduced in the intervention group. Since the intervention was ineffective, the investment in producing educational materials, training detailers, and conducting the visits was inefficient.

Implications for implementation science

Our results do not support the widespread adoption of educational outreach visits to change prescribing behavior. Contextual factors relating to the local setting, behavior being targeted, and type of program to be implemented should be considered and effectiveness evaluations should precede large scale programs.


Educational outreach visits were unsuccessful in improving family physicians’ compliance with guideline recommendations to decrease the relative use of COX-2 inhibitors in NSAID prescriptions, increase the relative use of omeprazole in PPI prescriptions, and decrease clopidogrel use in antiplatelet prescriptions. No effects were observed at 1, 6, and 18 months after the intervention, and there were no associated cost savings. Contextual factors may have been important to this result and should be considered when introducing educational outreach programs. Data for a process evaluation of the trial was collected, and its analysis could help us understand why the intervention had no effect.



Confidence interval


Defined daily dose


National Health Service


Non-steroidal anti-inflammatory drug


Proton pump inhibitors


  1. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318:527–30.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  2. Cabana MDRC. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458–65.

    Article  PubMed  CAS  Google Scholar 

  3. Lugtenberg M, Zegers-van Schaick JM, Westert GP, Burgers JS. Why don’t physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implement Sci. 2009;4:54.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Lenfant C. Clinical research to clinical practice—lost in translation? N Engl J Med. 2003;349(9):868–74.

    Article  PubMed  Google Scholar 

  5. Avorn J, Fischer M. “Bench to behavior”: translating comparative effectiveness research into improved clinical practice. Health Aff. 2010;29(10):1891–900.

    Article  Google Scholar 

  6. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8:iii–v. 1–72

    Article  PubMed  CAS  Google Scholar 

  7. Avorn J, Soumerai SB. Improving drug-therapy decisions through educational outreach: a randomized controlled trial of academically based detailing. N Engl J Med. 1983;308(24):1457–63.

    Article  PubMed  CAS  Google Scholar 

  8. O’Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, Haynes RB, Harvey EL. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;4:CD000409.

    Google Scholar 

  9. Soumerai SB. Principles of educational outreach (‘Academic detailing’) to improve clinical decision making. JAMA. 1990;263(4):549.

    Article  PubMed  CAS  Google Scholar 

  10. Rognstad S, Brekke M, Fetveit A, Dalen I, Straand J. Prescription peer academic detailing to reduce inappropriate prescribing for older patients: a cluster randomised controlled trial. Br J Gen Pract. 2013;63(613):e554–62.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Lopatto J, Keith SW, Del Canale S, Templin M, Maio V. Evaluating sustained quality improvements: long-term effectiveness of a physician-focused intervention to reduce potentially inappropriate medication prescribing in an older population. J Clin Pharm Ther. 2014;39(3):266–71.

    Article  PubMed  CAS  Google Scholar 

  12. Clyne B, Smith SM, Hughes CM, Boland F, Cooper JA, Fahey T, et al. Sustained effectiveness of a multifaceted intervention to reduce potentially inappropriate prescribing in older patients in primary care (OPTI-SCRIPT study). Implement Sci. 2016;11(1):79.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Barros PP, Machado SR, Simões J d A. Portugal. Health System Review. Health Syst Transit. 2011;13:1–156.

    PubMed  Google Scholar 

  14. Pisco L. Primary healthcare reform in Portugal on two fronts: autonomous family healthcare units and management of groupings of health centers. Cien Saude Colet. 2011;16:2841–52.

    Article  PubMed  Google Scholar 

  15. Legido-Quigley H, Panteli D, Car J, McKee M, Busse R. Clinical guidelines for chronic conditions in the European Union [Internet]. Copenhagen: WHO Regional Office for Europe; 2013. Available from: Cited 26 Sept 2017

    Google Scholar 

  16. Orientação 027/2011: Processo de emissão de Normas [Internet]. Lisbon: Direcção-Geral da Saúde; 2011. Available from: Cited 26 Sept 2017

  17. Pinto D, Heleno B, Rodrigues DS, Papoila AL, Santos I, Caetano PA. An open cluster-randomized, 18-month trial to compare the effectiveness of educational outreach visits with usual guideline dissemination to improve family physician prescribing. Implement Sci. 2014;9:10.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348(mar07 3):g1687.

    Article  PubMed  Google Scholar 

  19. Heleno B, Caetano PA, Pinto D, Monteiro E, Santos I. Norma 013/2011: anti-inflamatórios não esteróides sistémicos em adultos: orientações para a utilização de inibidores da COX-2 [Internet]. Lisbon: Direcção-Geral da Saúde; 2013. Available from: Cited 26 Sept 2017

    Google Scholar 

  20. Caetano PA, Heleno B, Pinto D, Monteiro E, Santos I. Norma 036/2011: Supressão Ácida: Utilização dos Inibidores da Bomba de Protões e das suas Alternativas Terapêuticas [Internet]. Lisbon: Direcção-Geral da Saúde; 2011. Available from: Cited 26 Sept 2017

    Google Scholar 

  21. Pinto D, Caetano PA, Heleno B, Monteiro E, Santos I. Norma 014/2011: Utilização e seleção de Antiagregantes Plaquetários em Doenças Cardiovasculares [Internet]. Lisbon: Direcção-Geral da Saúde; 2013. Available from: Cited 26 Sept 2017

    Google Scholar 

  22. Ivers NM, Halperin IJ, Barnsley J, Grimshaw JM, Shah BR, Tu K, Upshur R, Zwarenstein M. Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. Trials. 2012;13:120.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Random Sequence Generator [Internet]. Ireland:; 2017. Available from: Cited 26 Sept 2017

  24. Squires JE, Sullivan K, Eccles MP, Worswick J, Grimshaw JM. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:152.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Lynch EA, Cadilhac DA, Luker JA, Hillier SL. Education-only versus a multifaceted intervention for improving assessment of rehabilitation needs after stroke; a cluster randomised trial. Implement Sci. 2016;11(1):120.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Mason A. New medicines in primary care: a review of influences on general practitioner prescribing. J Clin Pharm Ther. 2008;33(1):1–10.

    Article  PubMed  CAS  Google Scholar 

  27. Zillich AJ, Ackermann RT, Stump TE, Ambuehl RJ, Downs SM, Holmes AM, et al. An evaluation of educational outreach to improve evidence-based prescribing in Medicaid: a cautionary tale. J Eval Clin Pract. 2008;14(5):854–60.

    Article  PubMed  Google Scholar 

  28. Naughton C, Feely J, Bennett K. A RCT evaluating the effectiveness and cost-effectiveness of academic detailing versus postal prescribing feedback in changing GP antibiotic prescribing. J Eval Clin Pract. 2009;15(5):807–12.

    Article  PubMed  Google Scholar 

  29. Enriquez-Puga A, Baker R, Paul S, Villoro-Valdes R. Effect of educational outreach on general practice prescribing of antibiotics and antidepressants: a two-year randomised controlled trial. Scand J Prim Health Care. 2009;27(4):195–201.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Senarathna L, Buckley NA, Dibley MJ, Kelly PJ, Jayamanna SF, Gawarammana IB, et al. Effect of a brief outreach educational intervention on the translation of acute poisoning treatment guidelines to practice in rural Sri Lankan hospitals: a cluster randomized controlled trial. PLoS One. 2013;8(8):e71787.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  31. Fairall LR, Folb N, Timmerman V, Lombard C, Steyn K, Bachmann MO, et al. Educational outreach with an integrated clinical tool for nurse-led non-communicable chronic disease management in primary care in South Africa: a pragmatic cluster randomised controlled trial. PLoS Med. 2016;13(11):e1002178.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Gjelstad S, Høye S, Straand J, Brekke M, Dalen I, Lindbæk M. Improving antibiotic prescribing in acute respiratory tract infections: cluster randomised trial from Norwegian general practice (prescription peer academic detailing (Rx-PAD) study). BMJ. 2013;347:f4403.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Brunette MF, Dzebisashvili N, Xie H, Akerman S, Ferron JC, Bartels S. Expanding cessation pharmacotherapy via videoconference educational outreach to prescribers. Nicotine Tob Res. 2015;17(8):960–7.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer prescribing--a trial of education, informatics, and financial incentives. N Engl J Med. 2016;374(11):1053–64.

    Article  PubMed  CAS  Google Scholar 

  35. Gude WT, van Engen-Verheul MM, van der Veer SN, Kemps HMC, Jaspers MWM, de Keizer NF, et al. Effect of a web-based audit and feedback intervention with outreach visits on the clinical performance of multidisciplinary teams: a cluster-randomized trial in cardiac rehabilitation. Implement Sci. 2016;11(1):160.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Salbach NM, Wood-Dauphinee S, Desrosiers J, Eng JJ, Graham ID, Jaglal SB, et al. Facilitated interprofessional implementation of a physical rehabilitation guideline for stroke in inpatient settings: process evaluation of a cluster randomized trial. Implement Sci. 2017;12(1):100.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Eccles MP, Steen IN, Whitty PM, Hall L. Is untargeted educational outreach visiting delivered by pharmaceutical advisers effective in primary care? A pragmatic randomized controlled trial. Implement Sci. 2007;2:23.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Silva JM, Stein AT, Schünemann HJ, Bordin R, Kuchenbecker R, de Lourdes Drachler M. Academic detailing and adherence to guidelines for group B streptococci prenatal screening: a randomized controlled trial. BMC Pregnancy Childbirth. 2013;13:68.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Riis A, Jensen CE, Bro F, Maindal HT, Petersen KD, Bendtsen MD, et al. A multifaceted implementation strategy versus passive implementation of low back pain guidelines in general practice: a cluster randomised controlled trial. Implement Sci. 2016;11(1):143.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Niven DJ, Mrklas KJ, Holodinsky JK, Straus SE, Hemmelgarn BR, Jeffs LP, et al. Towards understanding the de-adoption of low-value clinical practices: a scoping review. BMC Med. 2015;13:255.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


We would like to thank Inês Monteiro Gomes for being the trial manager, Paula Fernandes for providing logistical support, António Faria Vaz for facilitating access to pilot data and general advice regarding the study goals, José Carlos Ramos for collecting the raw pilot data and contributing to the collection of final results from the Regional Health Administration database, Isa Nunes and Inês Oliveira for contributing to the collection of final results data from the Regional Health Administration database, Emilia Monteiro for high-level advice and encouragement including co-authorship of the guidelines, and Luís Pisco on behalf of the Lisbon Regional Health Administration for allowing us to conduct the trial and access the prescription data.


This project was financed by an unrestricted research grant from the National Health Institute Dr. Ricardo Jorge (Governmental Agency)-2011EXT577. Funding for this trial covered steering committee meetings, central organization costs, training, and payment for educational outreach visitors. Material support was provided by the NOVA Medical School (trial coordination center) and by the Regional Health Administration of Lisbon and Tagus Valley (access to the regional prescription reimbursement database, allowing the visited family physicians receive outreach visits in regular working hours). Funders had no role in the design, execution, analysis, interpretation, or publication of the data. Although there is a broad aim within the government to reduce costs amidst prescribing physicians, the authors never experienced any constraints in all areas of the study.

Availability of data and materials

The data that support the findings of this study is a property of the Lisbon Regional Health Administration (Portuguese Ministry of Health), and restrictions apply to its availability. Data was used with authorization for the current study. Data is however available from the authors upon reasonable request and with permission of the Lisbon Regional Health Administration.

Author information

Authors and Affiliations



DP and PAC conceived the study. ALP, BH, DSR, and IS contributed to the study design. ALP, DP, DSR, and PAC calculated the sample size and planed the statistical analysis. PAC was the grant holder. DP, DSR, IG, IS, and PAC contributed to the recruitment of participants, training of detailers, and preparing educational content. DP, DSR, and PAC acted as detailers. IG was the trial manager. DP worked with the Regional Health Administration to collect the data. DP and ALP conducted the statistical analysis of the data. All authors contributed to drafting the manuscript, reviewed its content, and approved the final manuscript.

Corresponding author

Correspondence to Daniel Pinto.

Ethics declarations

Authors’ information

DP: MD, PhD-candidate; Family Medicine Unit, Grupo de Investigação Académica Independente - Chronic Diseases Research Center, NOVA Medical School | Faculdade de Ciências Médicas, Universidade Nova de Lisboa, Portugal.

BH: MD, PhD; Family Medicine Unit, Grupo de Investigação Académica Independente - Chronic Diseases Research Center, NOVA Medical School | Faculdade de Ciências Médicas, Universidade Nova de Lisboa, Portugal.

DSR: MD, PhD-candidate; Family Medicine Unit, Grupo de Investigação Académica Independente - Chronic Diseases Research Center, NOVA Medical School | Faculdade de Ciências Médicas, Universidade Nova de Lisboa, Portugal.

ALP: BMath, MStat, PhD; Department of Biostatistics, Chronic Diseases Research Center; NOVA Medical School | Faculdade de Ciências Médicas, Universidade Nova de Lisboa, Portugal, Center of Statistics and Applications, Universidade de Lisboa.

IS: MD, MEd, PhD; Family Medicine Unit, Grupo de Investigação Académica Independente - Chronic Diseases Research Center, NOVA Medical School | Faculdade de Ciências Médicas, Universidade Nova de Lisboa, Portugal.

PAC: PharmD, PhD, MPH; former member of Grupo de Investigação Académica Independente - Chronic Diseases Research Center, NOVA Medical School | Faculdade de Ciências Médicas, Universidade Nova de Lisboa, Portugal.

Ethics approval and consent to participate

The trial was approved by the ethics committees of the Lisbon Regional Health Administration and NOVA Medical School. Family physicians gave consent to participate. The trial only collected aggregated and non-identifiable patient data.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

TIDieR checklist. (DOCX 22 kb)

Additional file 2:

Copies of the brochures and point of care summaries. (PDF 3619 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pinto, D., Heleno, B., Rodrigues, D.S. et al. Effectiveness of educational outreach visits compared with usual guideline dissemination to improve family physician prescribing—an 18-month open cluster-randomized trial. Implementation Sci 13, 120 (2018).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: