We developed a Guideline Implementation Planning Checklist by conducting a comprehensive review of existing resources. This can be applied by guideline developers or users to prepare for and/or undertake implementation. Information was variable across documents, and, in general, quite sparse. In large part this was due to the fact that most resources focused on guideline development, and few were dedicated to guideline implementation. As a result, the Guideline Implementation Planning Checklist is brief. However, it provides developers or users with a framework for implementation planning that coincides with guideline development, can be adapted to context-specific factors and therefore may be broadly applicable, can be employed by guideline developers or users so its application is flexible and, as a synthesis of available information, it is more detailed than existing resources.

This research is limited in that our search strategy or screening may have failed to retrieve or identify all relevant resources, stringency of screening criteria may have excluded otherwise eligible documents, and we may not have accurately extracted or synthesized data. To mitigate these issues we performed a systematic search of previous comprehensive syntheses of guideline development and implementation documents, and of indexed databases for published healthcare research; and data extraction and synthesis was performed independently by two individuals which enhances reliability of the findings.

As noted, the content of the Guideline Implementation Planning Checklist is limited by the content of existing documents from which it was compiled which, in turn, is influenced by limited evidence for these processes. With respect to planning for implementation, there is no evidence on the effectiveness of planning steps or considerations, nor is it practical that such studies would be conducted. For example, the impact of forming an implementation team or developing an implementation plan on the conduct and outcomes of implementation planning is a logistical consideration, and would likely not be a priority for research funders. Evidence is mixed on the impact of assessing barriers as a means of choosing and tailoring interventions. For example, a Cochrane systematic review of 25 studies found that interventions tailored based on identified barriers were more likely to improve professional practice compared with no intervention or with dissemination of guidelines, however, methods used to identify barriers and tailor interventions were not described in eligible studies so how best to do this remains unclear [61]. Another systematic review of 63 studies across which the same instrument was used to assess barriers found that, although it identified numerous barriers, this was not reflected in the tailoring of implementation strategies, and was not associated with evidence adoption [62]. While several eligible documents recommended engaging stakeholders throughout the implementation planning process, few details were offered. Knowledge brokers can function as liaisons between those generating and applying evidence. Qualitative analysis of a randomized controlled trial revealed numerous ways in which knowledge brokers enhanced the capacity of public health officials to practice evidence-informed decision-making, however, the trial found their impact to be equivalent to that of evidence summaries so the authors recommended ongoing research to optimize their role [63]. The principles and practices of action research or integrated knowledge translation could be used to elaborate on how knowledge brokers could establish and nurture partnerships with stakeholders [64,65]. While evidence underlying the planning strategies identified in eligible documents may be limited, wide consultation with, and the consensus of guideline developers and implementers could be used to enhance the implementation planning component of the Guideline Implementation Planning Checklist until further evidence becomes available.

With respect to developing guideline implementation tools, this content could be expanded based on recent research in which Gagliardi et al. engaged the international guideline community to establish a 12-item framework of the desirable features of implementation tools, and identified 11 broad methodological steps, each with several sub-steps and considerations for developing implementation tools [66]. This knowledge can be applied to assess, and endorse or adapt existing implementation tools, or develop new implementation tools. Addition of these methods to the Guideline Implementation Planning Checklist may further enhance its relevance. However, further research is needed to develop guides or templates specific to different types of implementation tools.

With respect to dissemination, the list of options available in eligible documents and compiled in the Guideline Implementation Planning Checklist matches several items within the Professional Interventions domain of taxonomies of methods that have been used to implement guidelines including distribution and advertising of guideline material through mailing and mass media [16,17]. However, neither the taxonomies nor eligible documents comment on the comprehensiveness of the list of dissemination options, or on their effectiveness for achieving awareness and use of guidelines among different target audiences. In fact, the comparative effectiveness of such strategies is unclear. A Cochrane systematic review of printed educational materials, which included guidelines, reviewed data from 45 studies and concluded that, when used alone, such material can have a small impact on professional practice [67]. Another Cochrane systematic review of eight studies found that mass mailing of printed bulletins summarizing evidence may improve practice when there is a single, clear message, if the change is relatively simple to accomplish, and there is growing awareness by users of the evidence that a change in practice is required [68]. A systematic review of 42 articles reporting 38 dissemination studies by the Agency for Healthcare Research and Quality (AHRQ) similarly identified little comparative effectiveness research upon which to base the selection of dissemination strategies [69]. While evidence was limited, the AHRQ review found that reach strategies (dissemination using regular mail, email, or social or mass media) were as effective as ability strategies (providing users with additional resources or tools), and that multicomponent strategies involving a combination of research, ability and motivational strategies (opinion leaders, champions, etc.) appeared to be more effective than using one strategy alone, particularly for guideline adherence. Thus, some evidence exists to show that various means of dissemination can reach target audiences. This is further confirmed by a systematic review of studies that evaluated guideline implementation efforts which found that dissemination of guidelines through web sites, journal publications or other means resulted in awareness of, and agreement with guidelines among target users [5]. However, the same systematic review revealed that target users struggled with adoption, and other research involving interviews and focus groups with clinicians similarly revealed that they were frustrated with and uncertain about how to implement guidelines [14]. Therefore, guideline developers and users require implementation advice.

Eligible documents offered several implementation options but the compiled list was not as comprehensive as the options in published taxonomies of implementation strategies [16,17]. Moreover, the implementation strategies most frequently recommended in eligible documents were educational meetings and audit and feedback. Considerable evidence suggests that the impact of educational meetings is small and, while perhaps necessary as an initial means of raising awareness and engaging stakeholders, may need to be accompanied by other strategies that are informed by a contextual needs assessment [70]. Audit and feedback has also been investigated in a large number of trials and can achieve small to moderate impact when it is delivered by a supervisor or respected colleague, presented frequently, features both specific goals and action plans, aims to decrease the targeted behavior, baseline performance is low, and recipients are non-physicians [71]. In future research, the Guideline Implementation Planning Checklist could be improved by referring users to taxonomies of dissemination and implementation strategies [16,17], and systematic reviews that evaluate their effectiveness [6].

Given the conflicting evidence on the effectiveness of various implementation strategies, and the nuanced application of these strategies which requires contextual analysis and tailoring of strategies, it simply may not be possible to capture those details in a checklist or, indeed, even in a more detailed instructional manual such as those from which data were extracted. Therefore, the Guideline Implementation Planning Checklist could be applied in conjunction with additional strategies to develop implementation capacity among guideline developers and users. For example, a few eligible resources mentioned that the implementation team should include one or more knowledge translation experts. This may refer to researchers, managers or clinicians with expertise or experience in guideline implementation. It may also refer to knowledge brokers, who would liaise with both the guideline development or implementation group, and with target stakeholders to plan, develop and/or apply the most feasible and appropriate implementation strategies [72]. Alternatively, individuals charged with developing or implementing guidelines may require implementation training. Knowledge translation training programs have been established in the United States, United Kingdom and Canada [73-75]. Gagliardi et al. conducted a systematic review to generate guidance on how to develop a mentorship program for developing implementation capacity [76]. Through the G-I-N Implementation Working Group, we are pilot-testing the use of remote coaching by implementation experts as a means of providing guideline developers and implementers with guidance on an as-needed basis during the course of their initiatives.

In summary, this research identified that many of the recommended methods for planning guideline implementation may be intuitive but are not supported by evidence, and that lists of suggested options for dissemination and implementation are incomplete and lack descriptions of, or references to supporting evidence which would help developers or implementers choose which to apply. Further research is needed to generate evidence on how to assess barriers and use that information for implementation planning; describe the role of knowledge brokers or other knowledge translation experts in implementation planning; create guides or templates for developing specific types of implementation tools; and identify strategies that may be needed by developers and implementers in conjunction with the Guideline Implementation Planning Checklist to better plan guideline implementation. In ongoing research we will update the Guideline Implementation Planning Checklist by including references or links to relevant resources such as taxonomies of dissemination and implementation strategies, and systematic reviews that describe the effectiveness of those strategies. We will also consult with international guideline developers, implementers, and users to elaborate and improve the methods included in the Guideline Implementation Planning Checklist based on their needs and experiences. We may also conduct research in partnership with guideline developers, implementers and users to evaluate either concurrently or retrospectively their use of the Guideline Implementation Planning Checklist as a means of further refining its components.