Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Trends in publications regarding evidence-practice gaps: A literature review

  • Ann E Evensen1Email author,
  • Rob Sanson-Fisher2,
  • Catherine D'Este2 and
  • Michael Fitzgerald3
Implementation Science20105:11

DOI: 10.1186/1748-5908-5-11

Received: 6 July 2009

Accepted: 3 February 2010

Published: 3 February 2010

Abstract

Background

Well-designed trials of strategies to improve adherence to clinical practice guidelines are needed to close persistent evidence-practice gaps. We studied how the number of these trials is changing with time, and to what extent physicians are participating in such trials.

Methods

This is a literature-based study of trends in evidence-practice gap publications over 10 years and participation of clinicians in intervention trials to narrow evidence-practice gaps. We chose nine evidence-based guidelines and identified relevant publications in the PubMed database from January 1998 to December 2007. We coded these publications by study type (intervention versus non-intervention studies). We further subdivided intervention studies into those for clinicians and those for patients. Data were analyzed to determine if observed trends were statistically significant.

Results

We identified 1,151 publications that discussed evidence-practice gaps in nine topic areas. There were 169 intervention studies that were designed to improve adherence to well-established clinical guidelines, averaging 1.9 studies per year per topic area. Twenty-eight publications (34%; 95% CI: 24% - 45%) reported interventions intended for clinicians or health systems that met Effective Practice and Organization of Care (EPOC) criteria for adequate design. The median consent rate of physicians asked to participate in these well-designed studies was 60% (95% CI, 25% to 69%).

Conclusions

We evaluated research publications for nine evidence-practice gaps, and identified small numbers of well-designed intervention trials and low rates of physician participation in these trials.

Background

Many clinical guidelines have not been fully implemented in clinical practice, despite widespread acceptance of evidence-based recommendations by the medical community [121]. Closing these 'evidence-practice gaps' would result in significant improvements in public health. This outcome is desirable, but requires removal of barriers at the level of the patient, physician, medical organization, and socioeconomic or political community [22, 23].

Researchers and clinicians who identify specific barriers to guideline adoption and then design interventions to purposefully overcome them are most likely to affect change [24]. This process requires well-designed trials to identify the most successful strategies for change. This type of research is called 'knowledge translation' or T2, 'the translation of results from clinical studies into everyday clinical practice and health decision making' [25]. Funding for T2 research lags significantly behind that for technological innovations, despite estimates that health outcomes are more likely to improve with universal adoption of already proven guidelines [26, 27].

Despite the absence of proven strategies for guideline implementation, physicians are expected to successfully adopt guidelines into their practices. Physicians are held accountable for evidence-practice gaps when their practices are measured by internal quality reviews, insurance companies and government entities, (e.g., 'pay for performance') [28].

We hypothesized that the demand on physicians and health systems for improved patient outcomes would create demand for evidence-based methods for incorporating guidelines into clinical practice. We expected that the number of methodologically rigorous trials examining the differential effectiveness of strategies to change the behavior of clinicians or function of health care systems would increase over time. We also expected that there would be high levels (>75%) of clinician participation in such trials.

Methods

Literature search

We conducted a literature search to identify relevant publications. We examined English language studies regarding nine guidelines (Table 1) from January 1998 to December 2007 by performing a computer-based literature search of the PubMed data base with the following search terms: (clinical performance OR attitude OR knowledge OR evidence practice gap OR practice guidelines as topic* [mh] OR guideline adherence [mh] OR clinical practice guideline* OR guideline* [title] OR recommendation* OR adherence OR best practice* OR implementation OR know to do gap OR knowledge translation) AND (('1998/01/01' [PDat]: '2007/12/31' [PDat]) AND (English [lang])) AND (topic area term). The first and second authors selected the nine practice guidelines for analysis. Guidelines met all of the following criteria: each guideline was broadly applicable to the practice of family medicine; each guideline was supported by well-designed clinical trials; and we could identify a persistent evidence-practice gap for each guideline [110, 1621]. A persistent gap was determined to be present if demographic studies quantified a gap prior to January 1998 and after December 2007 and are referenced in Table 1. Analysis was limited to the practice recommendations listed in Table 1. Other practice recommendations included in the referenced guidelines were not included in this analysis.
Table 1

Medical guidelines selected for analysis

Topic

Guideline

ACE inhibitors

ACE inhibitors are the agent of choice in treatment of hypertension in diabetes mellitus.

Beta-blockers

Beta blockers should be prescribed to patients who have experienced a myocardial infarction.

Asthma

Inhaled anti-inflammatory agents should be used in patients with persistent asthma.

Atrial fibrillation

Patients with atrial fibrillation should be anticoagulated with coumadin.

Pain in cancer patients

Pain should be treated aggressively in terminal cancer patients.

Antibiotics for URTI

Antibiotics should not be used to treat viral upper respiratory tract infections.

Smoking in pregnancy

Pregnant women should be counselled to quit smoking.

Cervical cancer screening

Adult women should have regular cervical cancer screening.

Breast cancer screening

Adult women should have regular mammograms.

References [121]

Article classification

We initially divided articles in each of nine topics into two categories: intervention studies and non-intervention studies or publications. Intervention studies were defined as those that evaluated strategies to close the evidence-practice gap by changing patient or clinician attitudes, clinical behavior, and/or knowledge. If a publication incorporated intervention and non-intervention elements, it was included as an intervention study.

We further subdivided intervention studies based on the target of the intervention (patient or clinician). 'Patient' studies were defined as those that had the intervention applied to patients or family caregivers of patients. For example, a trial that compared rates of mammography in women randomized into two groups (advising by lay health advisors versus no intervention) would be a 'patient' study. 'Clinician' studies were defined as those that had interventions applied to clinicians or the health system. For example, a trial that compared antibiotic prescribing practices of physicians randomized into two groups (guideline dissemination by mail versus discussion of guidelines in a small group of physicians) would be a 'clinician' study.

We classified a study that evaluated interventions targeting both patients and clinicians as a 'clinician' study. We then classified 'clinician' intervention studies using standard Effective Practice And Organization Of Care (EPOC) criteria for research design into 'well-designed studies' (EPOC criteria 1.1-1.2, inclusive, describing randomized controlled trials (RCTs), controlled clinical trials, controlled before and after studies with adjustment for confounders, and interrupted time series) [29] and 'other studies' (any studies that did not meet EPOC criteria for adequate research design).

We subdivided non-intervention publications based on primary content (editorial, descriptive study, or treatment guideline). 'Editorial' publications were defined as non-data-based studies offering commentary on a facet of the evidence-practice gap. 'Descriptive studies' were data-based examinations of the evidence-practice gap, such as studies of epidemiology or sociodemographic factors, but did not evaluate any intervention strategy. 'Treatment guidelines' were defined as publications that described current treatment recommendations or clinical guidelines and did not report original research. Studies that examined the efficacy of treatment recommendations were excluded. Ten percent of the abstracts were randomly selected and type of study independently re-coded to provide an estimate of inter-rater reliability.

Statistical Methods

We investigated: whether the total number of evidence-practice gap publications that evaluated intervention strategies designed to improve clinician adherence to best practices increased over time; whether the proportion of these publications that were intervention studies increased over time; the proportion of these interventions that would be adequate as defined by EPOC criteria for experimental design [29]; and clinician participation in well-designed intervention trials.

The number of publications for each topic and type of study are presented. Given the small number of publications, studies were collapsed across topic areas and analysis undertaken on the pooled studies for the remaining analyses.

We undertook linear regression analysis of the number of evidence-practice gap articles versus time. A regression coefficient that was statistically significantly different from zero indicated an increase in the number of publications over time. We used the Cochran-Armitage Trend Test to determine whether the proportion of selected evidence-practice gap publications that were classified as intervention studies increased over time.

The percentage of clinician-focused intervention studies that used an adequate design (by EPOC criteria) was calculated with a 95% confidence interval (CI). If the lower limit of the confidence interval is greater than 75%, then we can conclude that the proportion of intervention studies that are RCTs is greater than a hypothesized value of 75%. Seventy-five percent was pre-specified in this analysis by consensus of the authors that this figure represented a clear majority of studies.

The median clinician consent rate for all studies targeting clinician adherence to best practice was calculated with a 95% confidence interval and compared to a hypothesized value of 75%.

Inter-rater reliability

We calculated the Kappa statistic to assess agreement between the two raters on type of study.

No approval was required by a human-subjects review board.

Results

For the nine medical guidelines, we identified 1,151 relevant publications from January 1998 to December 2007.

Total number of evidence-practice gap studies over time

The number of studies on the evidence-practice gap in the defined areas varied from 85 in 1998 to a high of 145 in 2003 (Tables 2 and 3). The slope of the simple linear regression model for total number of evidence-practice gap studies versus year was 2.10 (95% CI, -2.46 to 6.66), indicating no statistically significant increase over time.
Table 2

Number of intervention studies by year and topic

Year

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

Total

Topic

           

Mammograms

7

8

3

2

9

5

5

11

4

8

62

Beta-blockers

2

4

0

3

1

1

3

0

3

2

19

Atrial fibrillation

0

3

0

0

1

0

1

1

2

4

12

ACE inhibitors

0

0

0

0

0

1

1

0

1

0

3

Pain in cancer patients

0

1

0

0

0

0

1

0

0

0

2

Cervical cancer screening

1

0

0

0

0

0

0

0

1

1

3

Smoking in pregnancy

1

0

4

3

4

3

2

0

3

2

22

Antibiotics for URTI

0

3

4

3

5

3

4

6

6

8

42

Asthma

0

1

0

0

1

0

1

0

1

0

4

TOTAL

11

20

11

11

21

13

18

18

21

25

169

Table 3

Proportion of pooled intervention studies by year

Year

Total evidence-practice gap studies

Evidence-practice gap intervention studies (percent of total studies)

1998

85

11 (13%)

1999

114

20 (18%)

2000

102

11 (11%)

2001

129

11 (8.5%)

2002

113

21 (19%)

2003

145

13 (9.0%)

2004

123

18 (15%)

2005

117

18 (15%)

2006

93

21 (23%)

2007

130

25 (19%)

Total

1151

169 (15%)

Proportion of intervention trials compared to total evidence-practice gap studies

We found 169 intervention studies (15%) and 982 non-intervention studies (85%) (Tables 2 and 3). The percentage of all evidence-practice gap publications that involved intervention studies ranged from a minimum of 8.5% in 2001 to a maximum of 23% in 2006, a trend over time that was marginally non-significant (Cochran Armitage Trend Test Z = 1.9514, p = 0.0510).

Proportion of intervention studies that were well-designed

Of the 169 intervention trials, 87 (51%) were intended for patients and 82 (49%) were intended for clinicians. Of the 82 interventions intended for clinicians, 28 (34%; 95% CI, 24% to 45%) met the EPOC criteria for well-designed studies. Thus, the majority of intervention studies for clinicians do not meet EPOC criteria for well-designed studies. Of the studies that met EPOC criteria, there were 14 RCTs, two controlled clinical trials, five controlled before-and-after studies, and seven interrupted time designs. The most common reason for failure to meet EPOC criteria for good design was the inclusion of only one data point measurement of adherence to best practices before and after introduction of a guideline (28 of 54 studies). The remaining studies that failed to meet EPOC criteria were surveys, interviews, pilot projects, and observational studies.

Clinician consent rate in well-designed intervention studies

Only 11 of the 28 clinician-focused evidence-practice gap intervention studies meeting the EPOC criteria were included in this analysis, as 13 studies did not mention consent rates and four studies listed consent at the level of the physician practice or peer group rather than the individual physician. The median consent rate for well-designed studies targeting clinician adherence to best practice was 60% (95% CI, 25-69%), which was not greater than the hypothesized value of 75%.

Inter-rater reliability

In all, 109 publications were independently re-classified by type of study resulting in a Kappa of 0.85 (95% CI, 0.77 to 0.93).

Discussion

We examined publications of the last ten years related to the persistent evidence-practice gap in nine medical topic areas. We chose these topic areas because each guideline is supported by well-designed clinical trials and has been accepted by the medical community for a minimum of ten years [14, 710]. Despite widespread support for routine use of these guidelines to decrease morbidity, mortality, and/or costs, an ongoing evidence-practice gap is identified for each guideline [5, 6, 1121].

If the evidence-practice gaps were closing, it would be reasonable that further study of interventions or how to adopt them would not be needed. Because we document that gaps in all nine clinical topic areas are persisting, we expect that meaningful research would be ongoing. This research should include trials of strategies to affect change [24]. In contrast, we document with this study that over time the number of articles about the nine defined evidence-practice gaps did not significantly increase (2.10, 95% CI, -2.46 to 6.66). Our analysis demonstrated a marginally non-significant increase over time in the proportion of evidence-practice gap studies that were intervention studies, indicating that there may be some evidence of an increasing trend. However, the total number of intervention studies remained surprisingly low (an average of 1.9 intervention trials per year per topic area). The majority (53%) of publications over a ten-year period fell instead into the 'descriptive' category (see Methods for classification parameters). Although data-based, descriptive studies can only define or highlight problems rather than test solutions.

Reasons for the limited number of interventions were not identified by this study, but other reviews suggest ethical concerns, funding restrictions, and degree of difficulty in completing controlled trials compared with descriptive studies [22, 23, 26, 27].

Moving beyond observational studies, pre-post evaluations, and pilot studies to well-controlled research is necessary to obtain valid and generalisable results. However, we found few of these high-quality studies. Only 28 studies in nine subject areas over a ten-year period were well-designed studies evaluating strategies that clinicians or health systems could use to improve adherence to best practices. These 28 studies represent 16% of the total intervention studies, and only 2.4% of all of the evidence-practice gap publications in the nine topic areas.

The research patterns we describe above are discouraging, and it would be easy as a practicing physician to lay blame on the researchers or funding agencies designing and/or choosing which grants to support. It is also reasonable that a practicing physician may decline to participate in sloppy or frivolous research. However, this study documents low participation rates in clinician intervention studies that met EPOC criteria for adequate design. The median physician consent rate in well-controlled trials was 60% (95% CI, 25 to 69%).

Physicians may not participate in intervention trials for many reasons, including financial or time constraints, failure to be invited to participate, lack of interest, and disagreement about the medical merits of the intervention or research goal. A clinician may also refuse due to more psychologically complex issues: trial participation requires an acknowledgement that evidence-practice gaps exist and a willingness to let others dictate one's behavior. Physicians may also lack confidence that they are suitable agents of change for these guidelines. However, the medical community expects patients to readily participate in clinical trials so that valid and generalisable results are obtained. Physicians should be held to the same standard of participation.

Limiting the search to the PubMed database may have resulted in missing some relevant publications, but it is likely that a high-quality study would be published in a peer-reviewed journal catalogued in PubMed. The choice of medical guidelines may also affect the search results, but a similar pattern of limited high-quality interventions was seen in every guideline examined.

Summary

Evidence-practice gaps for nine well-established medical guidelines have persisted for the past ten years. Publications regarding these gaps are consistently descriptive in nature or simply restate treatment recommendations, with few rigorous trials of methods for closing the evidence-practice gap. The scarcity of high-quality intervention trials and low physician participation in these trials decrease the likelihood of closing the evidence-practice gap. This research pattern is insufficient to create successful strategies for implementing best practices. Instead, physicians are left without reliable means to improve their patients' health or means to meet the demand for improved health outcomes from employers and insurers. A new research pattern of evaluating strategies for changing clinical behavior and the functioning of health care systems is needed. Individual clinicians should contribute to translational research by readily agreeing to participate in these trials.

Declarations

Authors’ Affiliations

(1)
Department of Family Medicine, University of Wisconsin School of Medicine and Public Health
(2)
Faculty of Health, School of Medicine and Public Health, University of Newcastle
(3)
Centre for Clinical Epidemiology and Biostatistics, School of Medicine and Public Health, Faculty of Health, University of Newcastle

References

  1. Lang DM, Sherman MS, Polansky M: Guidelines and realities of asthma management: the Philadelphia story. Archives of Internal Medicine. 1997, 157: 1193-1200. 10.1001/archinte.157.11.1193.View ArticlePubMedGoogle Scholar
  2. Laupacis A, Boysan G, Connelly S: Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation. Archives of Internal Medicine. 1994, 154: 1449-1457. 10.1001/archinte.154.13.1449.View ArticleGoogle Scholar
  3. Max M: American Pain Society quality guidelines for treatment of acute and chronic pain. Journal of the American Medical Association. 1995, 274: 1874-1880. 10.1001/jama.274.23.1874.View ArticleGoogle Scholar
  4. US Preventive Services Task Force: Guide to Clinical Preventive Services. 1996, Baltimore: Williams and Wilkins, 2Google Scholar
  5. Yan AT, Yan RT, Tan M, Huynh T, Soghrati K, Brunner LJ, DeYoung P, Fitchett DH, Langer A, Goodman SG: Optimal medical therapy at discharge in patients with acute coronary syndromes: temporal changes, characteristics, and 1-year outcome. American Heart Journal. 2007, 154: 1108-1115. 10.1016/j.ahj.2007.07.040.View ArticlePubMedGoogle Scholar
  6. Allen LaPointe NM, Governale L, Watkins J, Mulgund J, Anstrom KJ: Outpatient use of anticoagulants, rate-controlling drugs, and antiarrhythmic drugs for atrial fibrillation. American Heart Journal. 2007, 154: 893-898. 10.1016/j.ahj.2007.06.035.View ArticlePubMedGoogle Scholar
  7. The HOPE study investigators: The HOPE (Heat Outcomes Protection Evaluation) Study.: The design of a large, simple, randomized trial of an angiotensin-converting enzyme inhibitor (ramipril) and vitamin E in patients at high risk of cardiovascular events. Can J Cardiol. 1996, 12: 127-137.Google Scholar
  8. Yusef S, Peto R, Lewis J, Collins R, Sleight P: Beta blockade during and after myocardial infarction: an overview of the randomized trials. Prog Cardiovasc Dis. 1985, 27: 335-71. 10.1016/S0033-0620(85)80003-7.View ArticleGoogle Scholar
  9. Arroll B, Kenealy T: Antibiotics for the common cold and acute purulent rhinitis. Cochrane Database of Systematic Reviews. 2005, CD000247-10.1002/14651858.CD000247.pub2. 3
  10. US Preventative Services Task Force: Guide to clinical preventive services. 1996, Baltimore: Williams and Wilkins, 2Google Scholar
  11. Winkelmayer W, Fischer M, Schneeweiss S, Wang PS, Levin R, Avorn J: Underuse of ACE inhibitors and angiotensin II receptor blockers in elderly patients with diabetes. Am J Kidney Disease. 2005, 46: 1080-1087. 10.1053/j.ajkd.2005.08.018.View ArticleGoogle Scholar
  12. Boushey H, Stempl D: Forward. J Allergy Clin Immunol. 2002, 109: S479-S481. 10.1067/mai.2002.122713.View ArticlePubMedGoogle Scholar
  13. Beuken-van Everdingen van den MH, de Rijke JM, Kessels AG, Schouten HC, van Kleef M, Patijn J: Prevalence of pain in patients with cancer: a systematic review of the past 40 years. Ann Oncol. 2007, 18: 1437-49. 10.1093/annonc/mdm056.View ArticlePubMedGoogle Scholar
  14. Cantrell R, Young AF, Martin BC: Antibiotic prescribing in ambulatory care settings for adults with colds, upper respiratory tract infections, and bronchitis. Clin Ther. 2002, 24: 170-82. 10.1016/S0149-2918(02)85013-5.View ArticlePubMedGoogle Scholar
  15. DATA 2010, The Healthy People 2010 Database, The Centers for Disease Control and Prevention. (accessed 2 July 2008), [http://wonder.cdc.gov/data2010/]
  16. McBride D, Bruggenjurgen B, Roll S, Willich SN: Anticoagulation treatment for the reduction of stroke in atrial fibrillation: a cohort study to examine the gap between guidelines and routine medical practice. J Thromb Thrombolysis. 2007, 24: 65-72. 10.1007/s11239-006-0002-8.View ArticlePubMedGoogle Scholar
  17. Thiebaud P, Demand M, Wolf SA, Alipuria LL, Ye Q, Gutierrez PR: Impact of disease management on utilization and adherence with drugs and tests: the case of diabetes treatment in the Florida: a Healthy State (FAHS) program. Diabetes Care. 2008, 31: 1717-22. 10.2337/dc07-2118.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Well K, Pladevall M, Peterson EL, Campbell J, Wang M, Lanfear DE, Williams LK: Race-ethnic differences in factors associated with inhaled steroid adherence among adults with asthma. Am J Respir Crit Care Med. 2008, 178: 1194-201. 10.1164/rccm.200808-1233OC.View ArticleGoogle Scholar
  19. Joynt KE, Huynh L, Amerena JV, Brieger DB, Coverdate SG, Rankin JM, Soman A, Chew DP: Impact of acute and chronic risk factors on use of evidence-based treatments in patients in Australia with acute coronary syndromes. Heart. 2009, 95: 1442-8. 10.1136/hrt.2008.154781.View ArticlePubMedGoogle Scholar
  20. Håkonsen GD, Strelec P, Campbell D, Hudson S, Loennechen T: Adherence to medication guideline criteria in cancer pain management. J Pain Symptom Manage. 2009, 37: 1006-18. 10.1016/j.jpainsymman.2008.06.006.View ArticlePubMedGoogle Scholar
  21. Higashi T, Fukuhara S: Antibiotic prescriptions for upper respiratory tract infection in Japan. Intern Med. 2009, 48: 1369-75. 10.2169/internalmedicine.48.1893.View ArticlePubMedGoogle Scholar
  22. Rainbird K, Sanson-Fisher R, Buchan H: Identifying barriers to evidence uptake. 2006, National Institute of Clinical Studies, Melbourne, Australia, FebruaryGoogle Scholar
  23. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, Rubin HR: Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999, 282: 1458-65. 10.1001/jama.282.15.1458.View ArticlePubMedGoogle Scholar
  24. Grol R, Grimshaw J: Evidence-based implementation of evidence-based medicine. Joint Commission Journal on Quality Improvement. 1999, 25: 503-513.PubMedGoogle Scholar
  25. Sung NS, Crowley WF, Genel M, Salber P, Sandy L, Sherwood LM, Johnson SB, Catanese V, Tilson H, Getz K: Central challenges facing the national clinical research enterprise. JAMA. 2003, 289: 1278-1287. 10.1001/jama.289.10.1278.View ArticlePubMedGoogle Scholar
  26. Woolf SH: The meaning of translational research and why it matters. JAMA. 2008, 299: 211-213. 10.1001/jama.2007.26.PubMedGoogle Scholar
  27. Woolf SH, Johnson RE: The break-even point: when medical advances are less important than improving the fidelity with which they are delivered. Ann Fam Med. 2005, 3: 545-552. 10.1370/afm.406.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Hanckak NA: Managed care, accountability, and the physician. Med Clin North Am. 1996, 80: 245-61.View ArticleGoogle Scholar
  29. Checklist of the Cochrane Effective Practice and Organisation of Care (EPOC) Review Group. (accessed 2 Dec 2008), [http://www.epoc.cochrane.org]

Copyright

© Evensen et al; licensee BioMed Central Ltd. 2010

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.