Trends in guideline implementation: a scoping systematic review

Numerous strategies have been used to implement guidelines, but their impact is inconsistent. There is currently no reliable way to choose strategies that are most appropriate for implementing guidelines to address determinants of practice in different contexts. To generate knowledge that would provide guidance for choosing and tailoring implementation strategies, this study examined trends in guideline implementation over a 10-year period to identify whether and how strategies and their impact differed by guideline topic as a proxy for context-specific determinants. Overall, few studies were identified, and most focused on the implementation of diabetes guidelines, so the identification of trends in guideline implementation by comparing the use and impact of strategies by guideline topic was not possible. Most studies employed either professional or patient education though these were often combined with other strategies. Choice of strategy did not appear to be associated with guideline topic or change over time. The majority of studies achieved positive impact on patient and professional knowledge and behaviour and on patient outcomes. This did not appear to be associated with guideline topic, use of theories or barrier assessment, or number or type of implementation approaches and strategies.

Few studies reported selecting or tailoring implementation strategies based on theory, assessing barriers or any other rationale. This finding also emerged in other systematic reviews on implementation [60,61] despite the existence of reporting standards that require a justification for strategy selection and clear description of intervention design [62-64]. Therefore, it could not be ascertained if this was associated with choice or tailoring of strategy or impact. Moreover, this did not change over time despite increasing awareness of the need to assess barriers or determinants and tailor strategies according to these factors [10] and a plethora of systematic reviews, taxonomies, theories, models and instruments for doing so [11-18]. Such resources may not be easy to use or broadly relevant [19] and do not objectively or reliably suggest appropriate implementation strategies, so further research is needed to generate guidance for identifying barriers and choosing implementation strategies suitable for addressing those barriers. The responsibility for implementation planning, of which barrier assessment is a part, also remains unclear. Most of the studies aimed to improve compliance with guidelines in the context of quality improvement rather than implementing new guidelines, and in most studies, personnel were either not specified or were health professionals. None of the studies explicitly mentioned involving knowledge brokers or facilitators, individuals with expertise in implementation, in the planning or delivery of interventions [65]. Our group recently generated a framework for implementation planning which also recommends that individuals with expertise in implementation be involved in guideline development from the outset [66].

This study and other research showed that most often guidelines are implemented using educational approaches such as workshops [13,21]. This is not surprising because, in comparison with more complex interventions such as organizational, financial or regulatory strategies which were rarely used by studies included in this review and require large-scale change and/or considerable funding, educational approaches may be less resource intensive and more easily employed by guideline developers or implementers with limited funding [21]. Educational meetings are known to have a small impact on practice [67]. Most studies included in this review achieved desired impact even though they did not report using theories or barrier assessment to inform strategies and employed educational materials and meetings directed at health professionals or patients. This may be due to the fact that educational approaches were often combined with other approaches and strategies. The use of single versus multiple implementation approaches and strategies remains controversial. For example, a recent systematic review found that multiple strategies were more likely to achieve beneficial outcomes, in particular when they combined strategies meant to reach many individuals in a wide variety of settings (post, email, electronic, mass media), motivate interest in the evidence (champions, opinion leaders, social networks) and enhance ability to apply the evidence (additional information or tools that show how to incorporate the evidence into practice) [68]. Yet another rigorous meta-review of 25 systematic reviews that compared direct and indirect effect size and dose–response of single and multifaceted strategies showed no benefit of multifaceted over single strategies [69].

Implementation approaches and strategies were categorized according to the Mazza et al. taxonomy of guideline implementation strategies [12]. It was generated by mapping strategies employed by those whose research was presented in posters or presentations at conferences hosted by the Guidelines International Network onto the Effective Practice and Organisation of Care (EPOC) group’s data collection checklist [70]. The Mazza et al. taxonomy was easy to apply, and most strategies used in eligible studies could be readily categorized according to its components. This research added to the Mazza et al. taxonomy by identifying additional professional (print material, tailored guideline, self-audit) and patient (education, counselling, social networks, print material, reminders) strategies employed in eligible studies. The extended taxonomy that appears in Table 1 can be used by guideline developers, implementers, users or researchers to plan and undertake implementation. Another useful resource that identified and defined 68 discrete strategies for implementing not only guidelines, but a range of evidence-based health innovations, was generated by Powell et al. who compiled a framework from the components of 41 reviews published between 1995 and 2011 [71]. This was further refined and expanded to 73 discrete implementation strategies using a three-round expert Delphi process [72].

Print material aimed at both professionals and patients emerged as a commonly used strategy distinct from the distribution of guideline material which was already included in the Mazza et al. taxonomy. Research on print material offers conflicting evidence of their impact. For example, a systematic review found that print material such as medical journals or guidelines have a small impact on provider behaviour [73]. Yet, other research shows that providers and patients are most likely to benefit from guidelines that offer summaries or other tools that support guideline implementation [68], and experts have called upon those who generate guidelines to develop implementation tools [6,7]. International developers that we interviewed said that they required guidance for developing implementation tools [21]. Our analysis of instructional manuals for developing and implementing guidelines found an absence of such guidance [74]. Therefore, we consulted with members of the international guideline community to generate criteria, methods and considerations for developing guideline implementation tools [75,76].

This study features both strengths and limitations. This study is unique among other systematic reviews that examined the effectiveness of particular implementation strategies [13]. In comparison, this study attempted to identify the types of strategies that were used to implement guidelines on particular topics that may be challenged by unique barriers. Most studies provided sufficient detail about interventions such that the implementation approaches and strategies could be readily ascertained. Screening and data extraction were undertaken independently by multiple individuals to optimize reliability. However, several issues may limit the interpretation and use of these findings. Few studies were eligible, limiting the comparison of implementation approaches and strategies across clinical topics and limiting the pooling of data to assess determinants of strategies used or the impact of those interventions. Other systematic reviews of guideline implementation included a larger number of studies but spanned many more years and healthcare diseases or issues [13,60]. Of 32 eligible studies, 14 were assessed to have a low risk of bias. Since more than half of the studies had a moderate or high risk of bias, findings must be interpreted with some caution. Although we searched the two most relevant medical databases, the literature search may not have identified all relevant studies. We did not search the grey literature, assuming that most empirical research on guideline implementation would be found in indexed databases. Publication bias, or the tendency for journals to publish positive results, may have influenced the number and type of studies that were retrieved. No trends in implementation over time or by guideline characteristics were identified. Since the examination of guideline characteristics was based on guideline topic (hence, indirectly on guideline users), it was therefore limited to details that were reported in eligible studies. Other guideline characteristics may be relevant, but their influence on use and reported impact of implementation strategies would need to be assessed through primary data collection. The focus on four guideline topics may also have limited the ability to identify trends in the use and impact of guideline implementation strategies, so future research could repeat this study for additional guideline topics. However, given the limited yield of this scoping review, it is unclear whether similar reviews on additional clinical topics are warranted. Instead, rigorous primary research should be undertaken to optimize the design of interventions that would ultimately offer insight on the ideal implementation approaches and strategies. Such research may pursue two broad themes. This review identified numerous approaches and strategies that have not been used to implement guidelines on the topics examine in this study but warrant further investigation. This review also identified that, despite the availability of evidence, taxonomies, theories, models and instruments by which to plan implementation, guidelines are most often implemented using educational strategies and print material. Most of the studies reviewed here achieved positive impact, which perhaps conflicts with the results of systematic reviews demonstrating that educational meetings and print material have a small impact on professional behaviour [67,73]. Therefore, ongoing research might focus on ways to optimize the design of educational strategies and print material.