Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Integrating guideline development and implementation: analysis of guideline development manual instructions for generating implementation advice

Implementation Science20127:67

https://doi.org/10.1186/1748-5908-7-67

Received: 29 January 2012

Accepted: 10 July 2012

Published: 23 July 2012

Abstract

Background

Guidelines are important tools that inform healthcare delivery based on best available research evidence. Guideline use is in part based on quality of the guidelines, which includes advice for implementation and has been shown to vary. Others hypothesized this is due to limited instructions in guideline development manuals. The purpose of this study was to examine manual instructions for implementation advice.

Methods

We used a directed and summative content analysis approach based on an established framework of guideline implementability. Six manuals identified by another research group were examined to enumerate implementability domains and elements.

Results

Manuals were similar in content but lacked sufficient detail in particular domains. Most frequently this was Accomodation, which includes information that would help guideline users anticipate and/or overcome organizational and system level barriers. In more than one manual, information was also lacking for Communicability, information that would educate patients or facilitate their involvement in shared decision making, and Applicability, or clinical parameters to help clinicians tailor recommendations for individual patients.

Discussion

Most manuals that direct guideline development lack complete information about incorporating implementation advice. These findings can be used by those who developed the manuals to consider expanding their content in these domains. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated. New approaches for guideline development and implementation may need to be developed. Use of guidelines might be improved if they included implementation advice, but this must be evaluated through ongoing research.

Keywords

Guideline development Guideline implementation Implementability

Background

Research, practice, and policy in the healthcare sector focus on improving the organization, delivery, and outcomes of care. Critical to achieving these objectives is the need for guidance based on currently available knowledge generated through research. This has led to an emphasis on evidence syntheses such as guidelines that are defined as ‘systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances’ [1, 2]. While guidelines are important decision-making tools, along with expert clinical judgment and patient preference, their impact remains variable. Perhaps the most convincing evidence of this comes from a population-based assessment of performance on 439 recommendations for 30 conditions spanning preventive, acute, and chronic services that found that only 55% of patients in the United States received recommended care [3]. Similar lack of adherence to practice guidelines continues to be identified worldwide across different conditions and settings of care [47].

Numerous factors influence whether and how guidelines are used. Some of these are extrinsic factors, such as: the nature of the newly recommended practice or technology itself; characteristics of healthcare providers; organizational capacity to collect, adapt, share, and apply evidence; and system-level environmental factors [810]. Because these manifest downstream of guideline development, single and combined interventions can be applied to address these barriers and improve compliance with guideline recommendations, although their impact can be variable and inconsistent [11]. Other factors are intrinsic to guidelines and perhaps best addressed at the time of guideline development. For example, guidelines may be biased through conflict of interest, of variable methodological quality, poorly written and ambiguously presented, viewed as not applicable to individual patients or reducing clinician autonomy, and the volume of guidelines now available may be overwhelming, particularly given that recommendations for the same clinical indication may be inconsistent across different guidelines [1214].

To promote the development and use of high quality guidelines, the Appraisal of Guidelines Research and Evaluation (AGREE) instrument was developed by an international collaboration of guideline development experts [15]. A revised version, AGREE II, was recently issued [16]. It can be used to score guidelines based on their reported scope and purpose, stakeholder involvement, rigour of development, clarity of presentation, editorial independence, and applicability. The Guideline Implementability Appraisal (GLIA) instrument is another tool that similarly considers the impact of guideline recommendations on individuals and organizations, and whether measurable indicators are provided by which to evaluate implementation [17]. Still, many guidelines fail to comply with these recommended standards. Shaneyfelt et al. evaluated 279 guidelines published from 1985 to June 1997 by 69 different developers, and found that mean overall adherence to standards for guideline format, methods, identification and summary of evidence, and formulation of recommendations was 43% [18]. When the AGREE instrument was used to evaluate 86 guidelines from 11 countries published between 1992 and 1999, they were found to vary considerably in whether and how they addressed each domain [19].

Turner et al. proposed that variability in manuals describing guideline development methods may be causing the mismatch between standards for guideline quality and the actual quality of guidelines [20]. They identified and examined six prominent guideline development manuals and found strong concordance in recommended guideline development methods across all manuals. However, the instructions varied in level of detail, so the authors hypothesized that manuals may not provide sufficient guidance on how to complete all steps of the guideline development process.

It is now well recognized that simply producing guidelines does not itself lead to use, and that active implementation is needed to encourage their uptake. The Knowledge to Action Cycle offers a framework for promoting the use of guidelines [21]. It recommends an iterative process comprised of several steps, including adapting guidelines to local context, identifying barriers of guideline use, selecting and implementing tailored interventions to promote guideline use, monitoring guideline use, evaluating outcomes associated with guideline use, and sustaining guideline use. Regardless of whether guideline developers are mandated and resourced to implement the guidelines they produce, or whether it is the responsibility of other individuals or organizations to implement guidelines, detailed instructions for guideline implementation are needed. Turner et al. found that four of six manuals described instructions for developing implementation advice that could be included in guidelines. However, the framework they used to examine manuals was based largely on the AGREE instrument, supplemented by a literature search for other key methods related to the development of guidelines. The framework consisted of 14 elements organized in four domains: preparing for guideline development, systematically reviewing the evidence, drafting the guideline, and reviewing the guideline. Only one element pertained to implementation, and they only looked for presence or absence of any implementation information, and not the details of instructions for developing implementation advice.

We investigated how to make guidelines more implementable by modifying their content and format [22]. We first conducted a review of the medical literature for features of guidelines desired by guideline users, or that are positively associated with guideline use. The framework was validated through review by health professionals and researchers with various clinical and disciplinary perspectives, and refined by using it to examine the content of published guidelines judged to be high quality by trained experts [22]. The final framework consisted of 22 elements within eight domains, including adaptability, usability, validity, applicability, communicability, accommodation, implementation, and evaluation (Table 1). Most guidelines we examined contained a large volume of graded evidence (validity) and numerous tables featuring complementary clinical information (applicability), but few contained additional features representing the other six domains that may facilitate application of guidelines by users. As proposed by Turner et al., guidelines may offer little support to users for implementing the recommendations because guideline development manuals lack instructions for developing such implementation advice. To confirm this and more thoroughly describe the nature of instructions for developing implementation advice in guideline development manuals, the purpose of this research was to re-examine the manuals sampled by Turner et al. according to our framework of implementability, which reflects guideline components shown by research to be associated with intent to, and actual use of guidelines. Use of a more detailed framework serves to pinpoint specific implementation information that may be lacking, thereby informing future updating of such manuals, or development of adjunct products.
Table 1

Guideline implementability framework

Domain

Definition

Element

Examples

Adaptability

The guideline is available in a variety of versions for different users or purposes

Sources

Internet, peer reviewed journal

Versions

Full text, summary, print, digital

Users

Tailored for patients or caregivers

Useability

Content is organized to enhancethe ease with which the guideline can be used

Navigation

Table of contents

Evidence

Narrative, tabulated or both

Recommendations

Narrative, graphic (algorithms) or both; Recommendation summary (single list in full or summary version rather than dispersed)

Validity

Evidence is summarized and presented such that its quantity and quality are apparent

Number of references

Total number of distinct references to evidence upon which recommendations are based

Evidence graded

A system is used to categorize quality of evidence supporting each recommendation

Number of recommendations

Total number of distinct recommendations

Applicability

Information is provided to help interpret and apply guidelines for individual patients

Clinical considerations

Information such as indications, criteria, risk factors, drug dosing that facilitates application of the recommendations explicitly highlighted as tips or practical issues using sub-titles or text boxes, or summarized in tables and referred to in recommendations or narrative

Communicability

Resources for providers or patients to inform, educate, support and involve patients

Inform, educate, support

Informational, educational or supportive resources for patients/caregivers, or contact information (phone, fax, email or URL) for such resources

Decision making

Questions or tools for clinicians to facilitate discussion with patients, or decision aids to support patient involvement

Relevance

The focus or purpose of the guideline is explicitly stated

Objective

Explicitly stated purpose of guideline (clinical, education, policy, quality improvement)

Stakeholders

Specify who would deliver (individuals, teams, departments, institutions, managers, policy makers, internal/external agents) and receive the services (specify type of patients)

Needs

Identification of stakeholder needs, perspectives, interests or values

Accommodation

Anticipated changes, resources and competencies required to adapt and accommodate guideline utilization are identified

Technical

Equipment or technology needed, or the way services should be organized

Regulatory

Industrial standards for equipment or technology, or policy regarding their use

Human resources

Type and number of health professionals needed to deliver recommended services

Professional

Education, training or competencies needed by clinicians/staff to deliver recommendations

Workflow

Anticipated changes in workflow or processes during/after adoption of recommendations

Costs

Direct or productivity costs incurred by acquiring resources or training to accommodate guidelines, or as a result of service reductions during transition from old to new processes

Implementation

Processes for planning and applying local strategies to promote guideline utilization are described

Identify barriers

Individual, organizational, or system barriers that could challenge adoption, or instructions for local needs assessment of guideline users

Tailor guideline

Instructions, tools or templates to tailor guideline/recommendations for local context

Integrated tools

Point-of-care templates/forms (clinical assessment, standard orders) to integrate guidelines within care delivery processes

Promote utilization

Possible mechanisms by which to promote guideline utilization

Evaluation

Processes for evaluating guideline implementation and utilization are described

Implementation

Methods for evaluating the implementation process

Utilization

Audit tools or performance measures/quality indicators to assess the organization, delivery and outcomes of guideline recommended care

Methods

Approach

Content analysis describes ideas in written, verbal, or visual communication to generate, extend, or validate a framework or model [23]. It involves preparation (select unit of analysis, review data), analysis (code, group text by category), and reporting (describe analysis, findings). There are three types of content analysis (conventional, directed, summative) that vary by inductive or deductive nature [24]. This study adopted a directed approach by examining the content of guideline development manuals based on domains and elements in the implementability framework, and a summative approach by synthesizing findings to compare the domains and elements included in each manual (summative). As with any qualitative research, the findings of content analysis can be quantified, for example, by counting the number of times a word is used or an idea expressed, or expressed more qualitatively by identifying themes and the way they are expressed [24]. We described the presence or absence of content reflecting implementability domains and elements quantitatively, and compared those findings across manuals.

Data collection

The unit of analysis is the guideline development manual. These were sampled by convenience. We examined the same manuals identified by Turner et al. because they used a comprehensive strategy to search for and identify the manuals, including the MEDLINE database, various guideline web sites, web sites of known guideline developers, a general search of the Internet using Google, and consultation with members of an evidence-based healthcare email list. They included manuals published in English language, produced by international organizations responsible for guideline development, and supporting the development of evidence-based guidelines, and excluded manuals issued by specialty societies for specific conditions.

Full-text manuals were retrieved from organization web sites, which were checked once again at the time of writing this manuscript to ensure the most recent versions were analyzed. A data extraction form reflecting the implementability framework domains and elements was developed. It was used to extract information about the presence or absence of each element anywhere within the guideline development manuals, plus producer, date of publication, number of pages, stated objective of the manual, implementation context (the stated purpose or setting of implementation), and how the implementation content was organized within the manual. Two individuals independently extracted data, and a third resolved differences and tabulated the data.

Data analysis

Tabulated findings were examined to describe general features of each manual, and enumerate the presence of implementation instructions reflecting elements from the implementability framework.

Results

Manual features are summarized in Table 2. The six manuals were published between 1998 and 2011 by the Council of Europe (COE), National Health and Medical Research Council (NHMRC), National Institute for Health and Clinical Excellence (NICE), New Zealand Guidelines Group (NZGG), Scottish Intercollegiate Guideline Network (SIGN), and the World Health Organization (WHO) [2530]. Their size ranged from 23 to 266 pages plus appendices. The stated objective of all manuals was similar: to inform guideline development. The stated implementation context was also similar across the six manuals, encompassing local patient care, regional management of services, and population-based policy setting. Five of the six manuals contained a specific chapter addressing implementation ranging from five to eleven pages [2529], and one of the five offered a 104-page supplement specific to implementation [26].
Table 2

Features of guideline manuals

Producer

Date published

Pages (#)

Manual objective

Implementation context

Section devoted specifically to guideline implementation

COE (25)

2002

77

Make proposals on the methodology to be used in developing guidelines

Daily practice, Managerial, Patient, Legal, ethical

11 pages in chapter on Dissemination and Implementation

Identify the practical, social, ethical and legal conditions for implementation of guidelines in daily practice

NHMRC (26)

1999, 2000 (supplement)

79

Put forward a method for developing clinical practice guidelines in Australia

Local conditions (population, setting, costs, constraints, patient values/preferences)

9 pages in chapter on Dissemination and Implementation

Reflect concern that greater emphasis should be placed on guideline implementation and evaluation

NICE (27)

2009

266 plus appendix

Explain how NICE develops clinical guidelines

Patient care and patient outcomes in the National Health Service as a whole

6-page chapter

Provide advice to guideline developers on technical aspects of guideline development and the methods used

Cost/resources

NZGG (28)

2001

86 plus appendix

To be used by guideline development teams, to assist them to produce evidence-base clinical practice guidelines

Legislative, administrative, clinical, industrial, consumer

10-page chapter on Dissemination and Implementation with focus on implementation

SIGN (29)

2011

104

Provide a reference tool that may be used by individual members of guideline development groups as they work through the development process

Local National Health Service board

5-page chapter

Allow users to see how SIGN guidelines are developed, and instill confidence … that the recommendations are both internally and externally valid, and feasible for practice

WHO (30)

2008

23

Document the recommended approach to development of WHO guidelines

Global, public health

COE, Council of Europe; NHMRC, National Health and Medical Research Council (Australia); NICE, National Institute for Health and Clinical Excellence; NZGG, New Zealand Guidelines Group; SIGN, Scottish Intercollegiate Guidelines Network; WHO, World Health Organization.

Implementability domains and elements addressed in manuals are enumerated in Table 3. Most manuals contained some representation of the majority of these domains and elements, though detail was minimal and did not lend itself to thematic analysis. COE included all domains but Applicability, but few elements within the domains of Relevance and Accommodation [25]. NHMRC [26] included all domains in the main manual or implementation supplement but few elements in the domain of Accommodation, as did NICE [27]. NZGG did not include the domains of Applicability and Communicability, and few elements in the domain of Accommodation [28]. SIGN did not include the domain of Applicability, and few elements in the domains of Accommodation and Implementation [29]. WHO did not include the domains of Adaptability and Comunicability, few elements in the domain of Accommodation, and cursory mention of Implementation and Evaluation [30].
Table 3

Domains of implementability addressed in guideline manuals

Domain/Element

Producer

COE

NHMRC

NICE

NZGG

SIGN

WHO

Total

Adaptability

       

 Sources

5

 Versions

5

 Users

5

Usability

       

 Navigation

2

 Evidence

5

 Recommendations

6

Validity

       

 # references

0

 Evidence graded

6

 # recommendations

0

Applicability

       

 Individualization

3

Patient support

       

 Patient resources

3

 Decision making

2

Relevance

       

 Objective

3

 Users

5

 Needs

6

Accommodation

       

 Technical

1

 Regulatory

0

 Human resources

0

 Professional

4

 Workflow

3

 Costs

  

4

Implementation

       

 Identify barriers

6

 Tailor recommendations

4

 Integrated tools

5

 Strategies to promote use

5

Evaluation

       

 Implementation

4

 Utilization

6

COE, Council of Europe; NHMRC, National Health and Medical Research Council (Australia); NICE, National Institute for Health and Clinical Excellence; NZGG, New Zealand Guidelines Group; SIGN, Scottish Intercollegiate Guidelines Network; WHO, World Health Organization.

Discussion

We examined the content of the same guideline development manuals reviewed by Turner et al. [20] using a more comprehensive framework to assess the degree to which they offered instructions for developing implementation advice. Our findings support the observation of Turner et al. that the manuals were similar but lacked sufficient detail. We did not identify trends in manual purpose, implementation context, or content by type of guideline developer. Use of a more comprehensive framework reflecting the multiple steps of implementation revealed specific topics not addressed. Most frequently this was Accomodation, or information that would help guideline users anticipate and/or overcome organizational- and system-level barriers. This may include: information about equipment or technology needed; industrial standards; policies governing their use; type and number of health professionals needed to deliver services; education, training, or competencies needed by staff to deliver services; anticipated changes in workflow or processes during or after adoption of guideline recommendations. In more than one manual, Communicability information was lacking, or resources that physicians and/or patients could use for education or to facilitate shared decision making, as was Applicability information, or clinical parameters to help physicians tailor guideline recommendations for individual patients.

While directions for creating Applicability information were absent in guideline development manuals, our previous work found that 90% of guidelines we examined did include clinical considerations by which to individualize recommendations [22]. This is likely because guidelines largely summarize data from clinical studies, and closely related to this is information such as diagnostic or risk criteria, pharmacologic dosing, indications for treatment or referral, and management options that often form the basis of Applicability content. The absence of Accomodation and Communicability information in guideline development manuals may not be surprising. Traditionally guidelines have often focused on questions relevant to clinicians, and when gaps in care emerged, the focus of quality improvement has been on changing clinician behaviour. Recognition of the need to consider a broader array of factors that influence guideline use and impact, and the role of guidelines as tools to direct quality improvement in this broader context is relatively recent [810, 21]. Similarly, the role of patients in the patient-provider dyad, clinical decision making, and even guideline development is evolving [31]. In keeping with the fact that new paradigms are emerging, our previous work found that 50% of the guidelines we examined contained Communicability information, and less than 50% contained Accomodation information [22].

The guideline enterprise is challenged to keep up with these changes and modify the traditional guideline development and implementation strategy. This may require consideration of different knowledge sources and evidence, involvement of different types of experts in the guideline development and implementation process either in staged fashion or as multidisciplinary or interprofessional teams, and partnerships between those who synthesize and interpret clinical effectiveness data and those who plan and/or undertake implementation. We know that guideline implementation is complex, and many developers lack the mandate and/or resources to implement the guidelines they produce [32, 33], so an overarching question that must be addressed is who bears responsibility for implementing guidelines, what resources are required to support implementation as conceived in this emerging, broader paradigm, and who will provide those resources.

Interpretation of these findings may be confounded by several issues. Not all existing guideline manuals were examined. We instead relied on the comprehensive search strategy applied by another research team to identify our sample. They chose to examine six manuals that were produced by major international guideline development organizations, and of a general nature and therefore of broad relevance. We did search for and examine newer versions of those manuals if they were available. We also ran an updated search for other general guideline development manuals, and found only one published by the Canadian Medical Association [34]. For consistency and comparative purposes, we examined only the six manuals that were included in the previous study but brief review of the CMA manual revealed that it consists of 34 pages, of which one eleven-page chapter on implementation partially addresses Implementation and a three-page chapter partially addresses Evaluation. Instructions reflecting the remaining implementability domains and elements is lacking, therefore this manual too provides guideline developers with little instruction for preparing implementation advice.

Our implementability framework, while formulated based on expressed needs among various types of guideline users and studies positively associating these features with actual guideline use, has not been thoroughly validated. We have yet to conduct experimental studies that would test whether inclusion of information or tools reflecting these implementability domains leads to guideline implementation, use, and beneficial healthcare outcomes. We also need to assess how different types of users (clinicians, managers, policy makers) would interpret and use implementability tools. First, however, we need to develop implementability tools because our most recent analysis of guidelines (not yet published) found that few contained such tools. In advance of such development, we are further validating the framework with input from international guideline developers, implementers, and researchers. Despite the need for further validation, our framework offers a more comprehensive way to evaluate implementation instructions in guideline development manuals that reflects the emerging, more complex multi-step paradigm of guideline implementation. Therefore, this study provides a more detailed evaluation than previously published on whether differences in intrinsic qualities of guidelines may be due to differences in the content of development manuals, and more precisely how manuals could be refined to facilitate the development of guidelines that provide users with implementation advice. These findings can be used by those who developed the manuals to consider expanding their content. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated.

Another limitation—not necessarily of our study, but in the evidence on guideline development and implementation—is the lack of criteria by which to assess the quality or actionability of instructions for developing guidelines or guideline implementation advice [35]. By using our more comprehensive implementability framework, we were able to comment on the completeness of the instructions. To judge the quality of the content of the instructions would require development of criteria, probably through expert consensus because little evidence is available, and then more detailed analysis of the content, both of which were beyond the resources available for the study described here. In ongoing research, we are searching for and examining the format and content of guidelines or adjunct products reflecting Accommodation, Implementation, and Evaluation to create ideal templates for tools offering implementation advice that will be vetted by both guideline developers and users.

In conclusion, most manuals that direct guideline development lack complete information about incorporating implementation advice for elements considered important by health professionals and associated with guideline use. These findings can be used by those who developed the manuals to consider expanding their content in these domains. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated. However, to embrace the emerging expanded paradigm of guideline development and implementation, new approaches may need to be considered, including use of different knowledge sources and evidence, involvement of different types of experts, and partnerships between those who synthesize and interpret clinical effectiveness data and those who plan and/or undertake implementation. Who bears responsibility for implementing guidelines, what resources are required to support implementation, and who will provide those resources must also be considered. Use of guidelines might be improved if they included implementation advice, but this must be evaluated through ongoing research.

Declarations

Acknowledgements

This study and the cost of this publication was funded by the Canadian Institutes of Health Research, which took no part in the study design or conduct, or decision to submit this manuscript for publication.

Authors’ Affiliations

(1)
University Health Network
(2)
McMaster University, Juravinski Hospital Site

References

  1. Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L: Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966–1998. J Gen Intern Med. 2006, 21 (Suppl 2): S14-S20.PubMedPubMed CentralGoogle Scholar
  2. Browman GP, Levine MN, Mohide A, Hayward RSA, Pritchard KI, Gafni A, Laupacis A: The practice guidelines development cycle: A conceptual tool for practice guidelines development and implementation. J Clin Oncol. 1995, 13 (2): 502-512.PubMedGoogle Scholar
  3. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. NEJM. 2003, 348 (26): 2635-2645. 10.1056/NEJMsa022615.View ArticlePubMedGoogle Scholar
  4. FitzGerald JM, Boulet LP, McIvor RA, Zimmerman S, Chapman KR: Asthma control in Canada remains suboptimal: the Reality of Asthma Control (TRAC) study. Can Respir J. 2006, 13 (5): 253-259.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Brown LC, Johnson JA, Majumdar SR, Tsuyuki RT, McAlister FA: Evidence of suboptimal management of cardiovascular risk in patients with type 2 diabetes mellitus and symptomatic atherosclerosis. CMAJ. 2004, 171 (10): 1189-1192. 10.1503/cmaj.1031965.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Roberts VI, Esler CN, Harper WM: What impact have NICE guidelines had on the trends of hip arthroplasty since their publication?. J Bone Joint Surg. 2007, 89 (7): 864-867. 10.1302/0301-620X.89B7.18593.View ArticleGoogle Scholar
  7. Milchak JL, Carter BL, James PA, Ardery G: Measuring adherence to practice guidelines for management of hypertension. Hypertension. 2004, 44: 602-608. 10.1161/01.HYP.0000144100.29945.5e.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Using Knowledge and Evidence in Health Care. Multidisciplinary Perspectives. Edited by: Louise L-C, Francois C. 2004, University of Toronto, Toronto, PressGoogle Scholar
  9. Francke AL, Smit MC, de Veer AJE, Mistiaen P: Factors influencing the implementation of clinical guidelines for health care professionals. BMC Med Inform Dec Mak. 2008, 8: 38-10.1186/1472-6947-8-38.View ArticleGoogle Scholar
  10. Baiardini I, Braido F, Bonini M: Why do doctors and patients not follow guidelines?. Curr Opin Allergy Clin Immunol. 2009, 9: 228-233. 10.1097/ACI.0b013e32832b4651.View ArticlePubMedGoogle Scholar
  11. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8 (6): 1-72.View ArticleGoogle Scholar
  12. Davis DA, Taylor-Vaisey A: Translating guidelines into practice: A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ. 1997, 157 (4): 408-416.PubMedPubMed CentralGoogle Scholar
  13. Grilli R, Magrini N, Penna A, Mura G, Liberati A: Practice guidelines developed by speciality societies: the need for critical appraisal. Lancet. 2000, 355: 103-106. 10.1016/S0140-6736(99)02171-6.View ArticlePubMedGoogle Scholar
  14. Ward JE, Grieco V: Why we need guidelines for guidelines: a study of the quality of clinical practice guidelines in Australia. Med J Aust. 1996, 156: 574-576.Google Scholar
  15. Collaboration AGREE: Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines. Qual Saf Health Care. 2003, 12 (1): 18-23.View ArticleGoogle Scholar
  16. Brouwers M, Kho ME, Browman GP: on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in health care. CMAJ. 2010, 182 (18): E839-E842. 10.1503/cmaj.090449.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, O’Connell R: The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005, 5: 23-10.1186/1472-6947-5-23.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J: Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999, 281 (20): 1900-1905. 10.1001/jama.281.20.1900.View ArticlePubMedGoogle Scholar
  19. Burgers JS, Cluzeau FA, Hanna SE, Hunt C, Grol R: Characteristics of high-quality guidelines: evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int J Technol Assess Health Care. 2003, 19 (1): 148-157.View ArticlePubMedGoogle Scholar
  20. Turner T, Misso M, Harris C, Green S: Development of evidence based clinical practice guidelines: comparing approaches. Implement Sci. 2008, 3: 45-10.1186/1748-5908-3-45.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell : Lost in knowledge translation: time for a map?. J Cont Ed Health Prof. 2006, 26: 13-24. 10.1002/chp.47.View ArticleGoogle Scholar
  22. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011, 6: 26-10.1186/1748-5908-6-26.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Elo S, Kyngas H: The qualitative content analysis process. J Adv Nurs. 2008, 62 (1): 107-115. 10.1111/j.1365-2648.2007.04569.x.View ArticlePubMedGoogle Scholar
  24. Hsieh HF, Shannon SE: Three approaches to qualitative content analysis. Qual Health Res. 2005, 15 (9): 1277-1288. 10.1177/1049732305276687.View ArticlePubMedGoogle Scholar
  25. Council of Europe: Developing a methodology for drawing up guidelines on best medical practice. 2002,http://www.leitlinien.de/mdb/edocs/pdf/literatur/coe-rec-2001-13.pdf,Google Scholar
  26. National Health and Medical Research Council: A guide to the development, evaluation and implementation of clinical practice guidelines. 1999, http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp30.pdf and How to put the evidence into practice: implementation and dissemination strategies, 2000. http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp71.pdf Google Scholar
  27. National Insitute for Health and Clinical Excellence: The guideline manual. 2009,http://www.nice.org.uk/media/5F2/44/The_guidelines_manual_2009_-_All_chapters.pdf,Google Scholar
  28. New Zealand Guidelines Group: Handbook for the preparation of explicit evidence-based clinical practice guidelines. 2001,http://old.nzgg.org.nz/download/files/nzgg_guideline_handbook.pdf,Google Scholar
  29. Scottish Intercollegiate Guidelines Network: A guideline developers’ handbook. 2011,http://www.sign.ac.uk/pdf/sign50.pdf,Google Scholar
  30. World Health Organization: Guidelines for WHO guidelines. 2008,http://www.searo.who.int/LinkFiles/RPC_Handbook_Guideline_Development.pdf,Google Scholar
  31. Legare F, Boivin A, van der Weijden T: Patient and public involvement in clinical practice guidelines. Med Decis Making. 2011, 31 (6): E45-E74. 10.1177/0272989X11424401.View ArticlePubMedGoogle Scholar
  32. Kryworuchko J, Stacey D, Bai N, Graham ID: Twelve years of clinical practice guideline development, dissemination and evaluation in Canada (1994 to 2005). Imp Sci. 2009, 4: 49-10.1186/1748-5908-4-49.View ArticleGoogle Scholar
  33. Lavis JN, Oxman AD, Moynihan R, Paulsen EJ: Evidence-informed health policy – interviews with the directors of organizations that support the use of research evidence. Implement Sci. 2008, 3: 55-10.1186/1748-5908-3-55.View ArticlePubMedPubMed CentralGoogle Scholar
  34. Davis D, Goldman J, Palda VA: Handbook on clinical practice guidelines. 2007, Canadian Medical Assocation,http://www.cma.ca/handbook.pdf,Google Scholar
  35. Schunemann HJ, Fretheim A, Oxman AD: Improving the use of research evidence in guideline development: Guidelines for guidelines. Health Research Policy and Systems. 2006, 4: 13-10.1186/1478-4505-4-13.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© Gagliardi and Brouwers; licensee BioMed Central Ltd. 2012

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement