- Open Access
- Open Peer Review
How can we improve guideline use? A conceptual framework of implementability
Implementation Sciencevolume 6, Article number: 26 (2011)
Guidelines continue to be underutilized, and a variety of strategies to improve their use have been suboptimal. Modifying guideline features represents an alternative, but untested way to promote their use. The purpose of this study was to identify and define features that facilitate guideline use, and examine whether and how they are included in current guidelines.
A guideline implementability framework was developed by reviewing the implementation science literature. We then examined whether guidelines included these, or additional implementability elements. Data were extracted from publicly available high quality guidelines reflecting primary and institutional care, reviewed independently by two individuals, who through discussion resolved conflicts, then by the research team.
The final implementability framework included 22 elements organized in the domains of adaptability, usability, validity, applicability, communicability, accommodation, implementation, and evaluation. Data were extracted from 20 guidelines on the management of diabetes, hypertension, leg ulcer, and heart failure. Most contained a large volume of graded, narrative evidence, and tables featuring complementary clinical information. Few contained additional features that could improve guideline use. These included alternate versions for different users and purposes, summaries of evidence and recommendations, information to facilitate interaction with and involvement of patients, details of resource implications, and instructions on how to locally promote and monitor guideline use. There were no consistent trends by guideline topic.
Numerous opportunities were identified by which guidelines could be modified to support various types of decision making by different users. New governance structures may be required to accommodate development of guidelines with these features. Further research is needed to validate the proposed framework of guideline implementability, develop methods for preparing this information, and evaluate how inclusion of this information influences guideline use.
Guidelines are syntheses of best available evidence that support decision making by clinicians, managers, and policy makers about the organization and delivery of healthcare, but continue to be underused. Numerous population-based studies demonstrate low compliance with guidelines produced by prominent government and professional agencies for chronic and acute conditions [1–7]. It has been proposed that for a condition such as cancer, a third of cases could be prevented, another third cured, and the remainder effectively treated if management consistently complied with existing guidelines . Thus, it is imperative that we better implement guidelines.
Many existing implementation strategies have limited effectiveness and are not routinely applied outside of experimental research due to their cost and complexity [9–18]. As a result many guidelines are passively distributed [19–21]. Surveys of international guideline developers found that few developers implemented their own guidelines, had dedicated implementation staff, or evaluated use of their guidelines, and many believed that target users should be responsible for implementation [22–25]. Accountability for guideline implementation may differ by jurisdiction and organization depending on the structure of the healthcare system and how programs are funded. Hence, there is a need to develop broadly applicable strategies for implementing guidelines.
Observational studies have shown that use of guidelines is associated with the complexity of the recommended clinical action so a promising, but untested option is to alter guideline recommendations to make them more easily implementable by target users [26, 27]. Implementability has been referred to as the characteristics of guideline recommendations that may enhance their implementation, and instruments have been developed to guide the formulation of implementable recommendations [28, 29]. However, research suggests that including information within guidelines to assist users with implementation of the recommendations may promote greater understanding of how users are to accommodate the recommendations, which may stimulate confidence in capacity to practice the recommended behaviour, leading to greater intent to use guidelines and possibly actual use .
For example, in a systematic review of 256 studies, 41 of them found that lack of comprehensible structure and local applicability were barriers of guideline use . Two randomized controlled trials (RCTs) examined the influence of guideline attributes on use. In one RCT, physicians of various specialties who received a guideline on electrodiagnostic tests (EDT) for patients with low back pain that was modified to include vignettes to illustrate use in patients with differing indications were more likely to use EDT appropriately compared with those who received the usual guideline . Another RCT found that wording a guideline in behaviourally specific terms enhanced patient attitude about, confidence in ability to use, and intention to use the recommendations [32, 33]. Furthermore, the information relevant to various guideline users may differ. Users include clinicians who deliver care, and managers and policy makers who must reconcile the competing interests of multiple stakeholders to make decisions about mobilizing organizational or system level resources . Research suggests that individual clinicians value an easy-to-use format, evidence clarity and validity, details about competency and training requirements, and guidance on how to blend experience with evidence when applying the recommendations to individual patients, and engage patients in shared decision making [35–39]. Managers and policy makers want guidelines to summarize resource or policy implications, and be publicly available in different versions for various purposes [40, 41].
It appears that guideline format and content may be important aspects of implementability that may influence use, and specific content may be required to support different types of decision making, including evidence-informed, experiential, shared, and resource allocation decision making. Including implementability information within guidelines to help users apply the recommendations represents a less-threatening, practice-relevant approach to guideline implementation compared with complex, costly, inconsistently effective implementation strategies often viewed negatively by guideline users . It may be easier to modify the content and format of guidelines rather than the clinical complexity of the recommendations. Finally, this approach may be more feasible for guideline developers to integrate with the processes they already use to create guidelines regardless of health system or funding structure. Therefore, further investigation of how to make guidelines more implementable is warranted.
To date, there has not been a systematic analysis of guideline features that may improve their use. The purpose of this study was to create a taxonomy of these attributes, and assess whether current guidelines contain these features, thereby identifying ways in which guidelines could be modified to potentially improve their use. This implementability framework could inform the development of modified guidelines or adjunct products, and evaluation of how various attributes influence perceptions about, and use of guidelines, prior to more definitive testing of whether their inclusion indeed improves use.
This study involved two key phases. The first phase was to develop an implementability framework of guideline format and content apart from clinical recommendations that are desired by users, or influence use of guidelines. The second phase was to use the framework assembled in phase one to examine the content of current practice guidelines, and refine or extend the framework.
Development of an implementability framework
Given the lack of controlled and observational studies on this topic, the methods were based on a modified meta-narrative approach . The meta-narrative approach is more suitable than a systematic review for conceptually examining literature that may be limited in quantity and quality, and vary in disciplinary focus and study design. It involves periodic input from a multidisciplinary research team to define the objectives and interpret the findings from a variety of conceptual perspectives. In this case, we used a modified approach that focused on healthcare literature rather than other disciplines, but were inclusive of a variety of study designs.
One individual with health librarian training and experience conducting different types of reviews (ARG) performed searches in MEDLINE and EMBASE from 1996 to August 2009. We purposely used a broad search strategy based on few terms (practice guidelines as topic AND guideline adherence AND attitude of health personnel or decision making, organizational or policy making) knowing that sensitivity and specificity would be limited. Two individuals independently selected eligible items. Eligible titles included empirical quantitative (meta-analyses, surveys, observational studies, randomized trials) or qualitative (reviews/conceptual analyses, interviews, focus groups) studies published in English language peer-reviewed journals describing guideline features desired by, or influencing the knowledge, decision making, or behaviour of health professionals. Studies were ineligible if they focused on guideline-informed tools such as clinical pathways; guidelines for non-medical interventions; clinical effectiveness of medical interventions; involved students, trainees, or patients as participants; investigated guideline use without examining views about guideline features that influenced use; evaluated interventions to promote guideline use; concluded that guideline features could be improved to promote their use without evaluating those features; or were in the form of abstracts, letters, commentaries, or editorials. All items selected by at least one individual were retrieved, and one individual extracted data. Quality assessment of studies was not undertaken to be inclusive of all relevant implementability elements.
Desirable or influential features potentially associated with guideline use were annotated in eligible studies, then tabulated. This tabulated list included the features of guidelines identified in each study as desirable or influencing guideline use. From this list, common items were categorized and defined. Findings were reviewed independently, and as a group by the study team, which included individuals with clinical, management, and research perspectives; grounding in the disciplines of knowledge translation/implementation science, psychology, and organizational behaviour; and experience in guideline development, guideline implementation, and performance improvement. The research team met in person and by teleconference to review and refine the draft framework. This largely involved minor edits to domain definitions. The draft framework was used to guide content analysis of guidelines, which expanded the number of elements in framework domains. This extended framework was reviewed and refined by the research team in person and by teleconference.
Application of the implementability framework
Manifest content analysis was used to examine guidelines for the presence of implementability elements. This is a method that describes explicit content as reported in written, verbal, or visual communication qualitatively and/or quantitatively, without interpretation of its underlying meaning . We selected a directed approach, which seeks to validate and extend elements in a framework . This means data are coded using elements in the draft framework, and data that cannot be coded are analyzed to assess if they represent a new element.
Individual guidelines were chosen as the unit of analysis. Guidelines on topics representing a high burden of illness in primary (diabetes, hypertension) and institutional care (chronic ulcer, heart failure) were selected from among those evaluated by the Guidelines Advisory Committee (GAC, http://www.gacguidelines.ca), a program that systematically appraised and summarized guidelines. Eligible guidelines included all those identified by GAC using a comprehensive search strategy and judged by trained experts using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument to be high quality that covered comprehensive management of these conditions and were publicly available .
Full versions of selected guidelines and adjunct products were retrieved from sponsor web sites. A form was developed to extract content from each guideline according to the implementability framework. Round one extraction was performed by ARG. This produced an expanded, revised framework, used by ARG to again extract data from each guideline. A research assistant independently reviewed the features in all guidelines, and a physician (VAP) independently reviewed coding of the elements for two guidelines on each clinical topic. ARG met with both independent reviewers to compare findings and resolve differences through discussion.
Data analysis and interpretation
Extracted data was tabulated. The presence of implementability elements within sampled guidelines was described using summary statistics including number, proportion, and mean or median. Detailed content was analyzed using Mays' narrative review method, based on verbatim reporting, rather than statistical summary or conceptual analysis of information . Data were examined to discuss the number of guidelines addressing each element overall and by topic, thereby identifying opportunities for modifying guideline format or content to enhance implementability. Findings were reviewed by the research team in person and by teleconference.
A total of 18 studies were reviewed from among 1,348 (441 MEDLINE, 907 EMBASE) identified by the literature search (Table 1). The vast majority of literature search results were ineligible because they evaluated the clinical effectiveness of medical interventions or interventions to promote guideline use. Eligible studies included one RCT, one observational study, four systematic reviews, three surveys, two modified Delphi studies, and six qualitative studies involving either focus groups or interviews. Based on features desired by, or influencing guideline use among primarily physicians, a preliminary taxonomy of eight implementability domains emerged, including adaptability, usability, validity, applicability, communicability, accommodation, implementation, and evaluation (Table 2).
Based on the implementability framework, data were extracted from 20 guidelines on the management of diabetes (n = 8), hypertension (n = 4), leg ulcer (n = 3) and heart failure (n = 5) from eight different countries, including Australia (n = 2), Brussels (n = 1), Canada (n = 4), the Netherlands (n = 1), New Zealand (n = 1), Singapore (n = 1), the United Kingdom (n = 4), and United States (n = 6) (Table 3). Most were produced by professional associations or government agencies (75%). The final framework derived through content analysis of guidelines included 22 elements organized within eight domains (Table 4).
Format elements that may facilitate guideline use are summarized in Table 5. One-half of the guidelines were published in journals or available in a summary version, and one-quarter were available as downloadable digital or patient versions. Many were very large documents with median pages of 72.5 (range 21.0 to 878.0), median number of references of 230.5 (range 15.0 to 3,487.0), and median number of recommendations of 41.5 (range 8.0 to 214.0). Most featured a table of contents (75.0%), and just over one-half included a recommendation summary (55.0%) or algorithm (65.0%). Nearly all guidelines used an evidence grading system (95.0%). Few summarized the evidence in tabular format (25.0%).
Content elements that may facilitate guideline use are summarized in Table 6. Clinical considerations by which to individualize recommendations were available in most guidelines (90.0%). For many guidelines this largely consisted of tables that summarized diagnostic or risk criteria, pharmacologic dosing, indications for treatment or referral, and management options (75.0%). All four hypertension guidelines included specific skill-based instructions for measuring blood pressure. Some guidelines featured sections explicitly labelled considerations for either special populations (two diabetes guidelines) or by health system capacity (one diabetes guideline). Two heart failure guidelines graphically highlighted considerations within text boxes or balloons labelled practice points or tips. Less than one-half of the guidelines included information to educate or engage patients (50.0%). Of these, five provided information to help clinicians discuss relevant issues with patients, two included information sheets for patients, and seven provided contact information (phone number or web site) where information for patients could be obtained.
No guidelines identified stakeholder needs or values, or costs or impact associated with use. Few included technical (45.0%), regulatory (15.0%), human resources (5.0%), or professional competency (20.0%) information required to accommodate guideline use. When included, this content was generally limited in detail. For example, technical guidance included: 'organization of care to deliver the above recommendations is largely concerned with putting registration, recall and record systems in place to ensure care delivery occurs for all people with diabetes, and having the healthcare professionals trained and available (D12)' or 'multidisciplinary care programs improve patients' quality of life, satisfaction with care, and the risk of unplanned hospitalization for heart failure (HF23).' Regulatory instructions included: 'blood pressure instruments must be properly validated and regularly recalibrated according to manufacturer instructions (H04).' Guidance for human resources included: 'interdisciplinary team comprised of family physician, diabetes educators (nurse, dietician), and community health support (D15).' Professional competency criteria included: 'compression bandages should be applied by a practitioner who has received training in their application (LU05).'
One-half of the guidelines included performance measures that could be used to monitor recommended clinical care. While 45.0% mentioned the need to actively promote guideline use, none thoroughly described how to undertake or evaluate this process. For example: 'implementation may be supported by a variety of activities including continuing education and training, and clinical audit (LU07)' or 'implementation programs are needed because it has been shown that the outcome of a disease may be favourably influenced by thorough application of clinical recommendations (HF26).' Less than a third included templates such as order forms or assessment checklists (30.0%), and fewer than this offered cursory instructions for identifying barriers of use (15.0%) or tailoring the guidelines to suit local circumstances (2.0%).
Relatively few studies published over the last 15 years specifically examined guideline features desired by, or associated with use among health professionals, most of these focused on physicians, and it does not appear that studies were informed by preceding research to build a cumulative body of knowledge. Considerable research has examined other factors influencing guideline use such as physician and organizational characteristics, but these studies were not eligible for this review, nor were numerous studies that examined general attitudes to guidelines on specific clinical topics. Review of 18 eligible studies revealed several features related to format or content that may positively influence guideline use, and this was expanded by reviewing the content of high quality international guidelines on various clinical topics. Most guidelines we examined contained a large volume of graded evidence and numerous tables featuring complementary clinical information to the point of being cumbersome, despite the presence of navigational features such as tables of contents. Few contained additional features specified by users or suggested by research to improve guideline use. Guideline use could potentially be improved by developing alternate versions for different purposes, incorporating summaries of evidence and recommendations, including information to facilitate interaction with and involvement of patients, outlining resource implications, and describing how to locally plan, promote, and monitor guideline use. There were no consistent trends by guideline topic.
Our findings simply suggest that more guidelines could be modified to include implementability content, but it remains unclear how various implementability features might influence guideline use. A recent analysis recommended that the reliability, relevance, and readability of knowledge resources be improved to support evidence-based decision making . Evidence is just one of several factors that inform decisions about guideline use . In reality, clinicians must often draw upon expertise and experience to consider what is best for and desired by those receiving care, but have expressed uncertainty about how to balance evidence with professional judgment and patient preferences, and the need for guidance to support these decisions [50, 51]. Furthermore, clinical decisions about guideline use are influenced by the availability and mobilization of organizational or system level resources, which are governed by managers and policy makers who must reconcile the competing interests of multiple stakeholders . Further insight could be gained by drawing upon decision science to examine the cognitive processes underlying guideline use. Considerable research has established that humans are not rational decision makers who identify alternative options, compare them on the same set of evaluative dimensions, and generate probability and utility estimates for different courses of action . Instead, it appears that a combination of intuitive (based on experience) and analytic (based on mental simulation) mechanisms are employed . This is particularly true in 'naturalistic' situations where decisions are complex; the quantity of information may be large or its implications ambiguous; goals may be shifting, poorly defined, or competing; and decisions have high stakes and are made within a dynamic environment under time constraints, as is true of the healthcare sector . It has been suggested that guidelines include content that mediates decisions among different stakeholders in a manner consistent with these cognitive processes . Thus, elements in the proposed framework may have impact on two dimensions: support for different types of decision making (evidence-informed, experiential, shared, allocation/policy) by providing particular information and/or tools, and support for different types of decision-making processes (intuitive, analytic) by making explicit the options for, and implications of alternate choices. This may influence attitudes about guideline relevance and confidence in choosing a course of action, which may be associated with use . While the concept of implementability is not new, the proposed framework is unique because it includes features that may be relevant not only to individuals, but to the managers and policy makers that govern the environment within which individuals function, and because it offers a novel way to improve guideline use by considering how to support different types and processes of decision making [28, 57, 58].
Interpretation of the findings may be limited in several ways. We studied guidelines relevant to primary and institutional care. Other guidelines relevant to specialty care may differ in their implementability characteristics. However, while we reviewed few guidelines, they were specifically selected to represent different topics, countries, and types of developer. Each element may not have been relevant to all guidelines reviewed, but this exercise serves as an exploratory, baseline effort to develop the framework according to content available in a range of guidelines. The literature on this topic is sparse, and referred to conceptually in a variety of ways and therefore not consistently indexed in literature databases; the search strategy used was purposely broad in an attempt to identify all relevant studies, but it may not have retrieved all studies describing guideline features desired by, or influencing the behavior of health professionals. We are currently in the process of conducting a systematic conceptual review of theoretical and empirical research on the mechanisms by which implementability elements influencing decision making about guideline use. Still, by assembling a rudimentary implementability framework that was expanded by review of guideline content, numerous opportunities were revealed for potentially improving guideline development and use.
Prior to testing these hypotheses, practical issues must be considered. Robust methods by which to operationalize concepts more specifically to enable accurate data capture would require further development. New governance structures may be required to accommodate the development of guidelines with these features. Future research should validate the framework by applying it to different types of guidelines, and by soliciting feedback from guideline development and implementation experts, clinicians, managers, policy makers, and patients/caregivers to further clarify and expand on its elements. Research is also required to examine precisely how the elements of implementability influence guideline use. Based on an expanded stakeholder-defined implementability framework, the cost-effectiveness of tailored guidelines or adjunct products could be established by examining short-term outcomes predictive of guideline use such as recall, attitude to, confidence in, and adoption intention, then long-term objective outcomes reflecting the adoption of processes and associated patient care outcomes.
Numerous opportunities were identified by which guidelines could be modified to potentially facilitate their use. New governance structures may be required to accommodate development of guidelines with these features. Further research is needed to validate the proposed framework of guideline implementability, develop methods for preparing this information, and evaluate how inclusion of this information influences guideline use.
Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L: Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Intern Med. 2006, 21 (Suppl 2): 14-20.
Browman GP, Levine MN, Mohide A, Hayward RSA, Pritchard KI, Gafni A, Laupacis A: The practice guidelines development cycle: A conceptual tool for practice guidelines development and implementation. Am J Clin Oncol. 1995, 13: 502-512.
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348: 2635-2645. 10.1056/NEJMsa022615.
FitzGerald JM, Boulet LP, McIvor RA, Zimmerman S, Chapman KR: Asthma control in Canada remains suboptimal: the Reality of Asthma Control (TRAC) study. Can Respir J. 2006, 13: 253-259.
Brown LC, Johnson JA, Majumdar SR, Tsuyuki RT, McAlister FA: Evidence of suboptimal management of cardiovascular risk in patients with type 2 diabetes mellitus and symptomatic atherosclerosis. CMAJ. 2004, 171: 1189-1192. 10.1503/cmaj.1031965.
Latosinsky S, Fradette K, Lix L, Hildebrand K, Turner D: Canadian breast cancer guidelines: have they made a difference?. CMAJ. 2007, 176: 771-776. 10.1503/cmaj.060854.
Francke AL, Smit MC, de Veer AJE, Mistiaen P: Factors influencing the implementation of clinical guidelines for health care professionals: A systematic meta-review. BMC Med Inform Decis Mak. 2008, 8: 38-10.1186/1472-6947-8-38.
World Health Organization: Cancer. Fact Sheets 297, 2006. access September 8, 2009, [http://www.who.int/mediacentre/factsheets/fs297/en/index.html]
Davis D, O'Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey AL: Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior?. JAMA. 1999, 282: 867-874. 10.1001/jama.282.9.867.
O'Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2001, 1: CD003030-
O'Brien MA, Oxman AD, Davis DA, Haynes RB, Freemantle N, Harvey EL: Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 1997, 4: CD000409-
Doumit G, Gattellari M, Grimshaw J, O'Brien MA: Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007, 1: CD000125-
Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L: Toward evidence-based quality improvement. Evidence of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Intern Med. 2006, 21 (Suppl 2): 14-20.
Wolff AM, Taylor SA, McCabe JF: Using checklists and reminders in clinical pathways to improve hospital inpatient care. Med J Aust. 2004, 181: 428-431.
Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes. J Am Med Assoc. 2005, 293: 1223-1238. 10.1001/jama.293.10.1223.
Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD: Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006, 2: CD000259-
Mehrotra A, Damberg CL, Sorbero ME, Teleki SS: Pay for performance in the hospital setting: What is the state of the evidence?. Am J Med Qual. 2009, 24: 19-28. 10.1177/1062860608326634.
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wending M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8: 1-72.
Armstrong R, Waters E, Crockett B, Keleher H: The nature of evidence resources and knowledge translation for health promotion practitioners. Health Promot Int. 2007, 22: 254-260. 10.1093/heapro/dam017.
Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, West P, Wright D, Wright J: What's the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients' notes, and interviews. BMJ. 2004, 329: 999-10.1136/bmj.329.7473.999.
Grunfeld E: Canadian breast cancer guidelines are as effective as possible under the circumstances. CMAJ. 2007, 176: 785-786. 10.1503/cmaj.070164.
Carter AO, Battista RN, Hodge MJ, Lewis S, Basinski A, Davis D: Report on activities and attitudes of organizations active in the clinical practice guidelines field. CMAJ. 1995, 153: 901-907.
Kryworuchko J, Stacey D, Bai N, Graham ID: Twelve years of clinical practice guideline development, dissemination and evaluation in Canada (1994 to 2005). Implement Sci. 2009, 4: 49-10.1186/1748-5908-4-49.
Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J: Towards evidence based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003, 15: 31-45. 10.1093/intqhc/15.1.31.
Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J: How can research organizations more effectively transfer research knowledge to decision makers?. Milbank Q. 2003, 81: 221-248. 10.1111/1468-0009.t01-1-00052.
Grilli R, Lomas J: Evaluating the message: the relationship between compliance rate and the subject of a practice guideline. Med Care. 1994, 32: 202-213. 10.1097/00005650-199403000-00002.
Foy R, MacLennan G, Grimshaw J, Penney G, Campbell M, Grol R: Attributes of clinical recommendations that influence change in practice following audit and feedback. J Clin Epidemiol. 2002, 55: 717-22. 10.1016/S0895-4356(02)00403-1.
Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, O'Connell R: The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005, 5: 23-10.1186/1472-6947-5-23.
Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J: Towards evidence based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003, 15: 31-45. 10.1093/intqhc/15.1.31.
Cochrane LJ, Olson CA, Murray S, Dupuis M, Tooman T, Hayes S: Gaps between knowing and doing: Understanding and assessing the barriers to optimal health care. J Contin Educ Health Prof. 2007, 27: 94-102. 10.1002/chp.106.
Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M: Are non-specific guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res. 2000, 34: 1429-1448.
Michie S, Johnston M: Changing clinical behaviour by making guidelines specific. BMJ. 2004, 328: 343-345. 10.1136/bmj.328.7435.343.
Michie S, Lester K: Words matter: increasing the implementation of clinical guidelines. Qual Saf Health Care. 2005, 14: 367-3670. 10.1136/qshc.2005.014100.
Lemieux-Charles Louise, Champagne Francois: Using Knowledge and Evidence in Health Care - Multidisciplinary Perspectives. 2004, Toronto: University of Toronto Press
Rashidian A, Eccles MP, Russell I: Falling on stony ground? A qualitative study of implementation of clinical guidelines' prescribing recommendations in primary care. Health Policy (New York). 2008, 85: 148-161.
Harris JS, Mueller KL, Low P, Phelan J, Ossler C, Koziol-McLain J, Glass LS: Beliefs about and use of occupational medicine practice guidelines by case managers and insurance adjusters. J Occup Environ Med. 2000, 42: 370-6. 10.1097/00043764-200004000-00007.
Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H: Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. Br Med J. 1998, 317: 858-861.
Michie S, Pilling S, Garety P, Whitty P, Eccles MP, Johnston M, Simmons J: Difficulties implementing a mental health guideline: an exploratory investigation using psychological theory. Implement Sci. 2007, 2: 8-10.1186/1748-5908-2-8.
Nuckols TK, Lim YW, Wynn BO, Mattke S, MacLean CH, Harber P, Brook RH, Wallace P, Garland RH, Asch S: Rigorous development does not ensure that guidelines are acceptable to a panel of knowledgeable providers. J Gen Intern Med. 2008, 23: 37-44. 10.1007/s11606-007-0440-9.
Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers' perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002, 7: 239-44. 10.1258/135581902320432778.
Ouimet M, Landry R, Amara N, Belkhodja O: What factors induce health care decision-makers to use clinical guidelines? Evidence from provincial health ministries, regional health authorities and hospitals in Canada. Soc Sci Med. 2006, 62: 964-976. 10.1016/j.socscimed.2005.06.040.
Carlsen B, Glenton C, Pope C: Thou shalt versus thou shalt not: a meta-analysis of GPs attitudes to clinical practice guidelines. Br J Gen Pract. 2007, 57: 971-8. 10.3399/096016407782604820.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R: Storylines of research in diffusion of innovation: a meta-narrative approach to a systematic review. Soc Sci Med. 2005, 61: 417-30. 10.1016/j.socscimed.2004.12.001.
Elo S, Kyngas H: The qualitative content analysis process. J Adv Nurs. 2008, 62: 107-115. 10.1111/j.1365-2648.2007.04569.x.
Hsieh HF, Shannon SE: Three approaches to qualitative content analysis. Qual Health Res. 2005, 15: 1277-1288. 10.1177/1049732305276687.
Cluzeau F, Burgers J, Brouwers M, Grol R, Makela M, Littlejohns P, Grimshaw J, Hunt C: Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003, 12: 18-23. 10.1136/qhc.12.1.18.
Mays N, Pope C, Popay J: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005, 10 (Suppl 1): 6-20. 10.1258/1355819054308576.
Straus S, Haynes RB: Managing evidence based knowledge: the need for reliable, relevant and readable resources. Can Med Assoc J. 2009, 180: 942-945. 10.1503/cmaj.081697.
Denis JL, Hebert Y, Langley A, Lozeau D, Trottier LH: Explaining diffusion patterns for complex health care innovations. Health Care Manage Rev. 2002, 27: 60-73.
Towle A, Godolphin W, Grams G, Lamarre A: Putting informed and shared decision making into practice. Health Expect. 2006, 9: 321-332. 10.1111/j.1369-7625.2006.00404.x.
Legare F, Ratte S, Gravel K, Graham ID: Barriers and facilitators to implementing shared decision making in clinical practice: Update of a systematic review of health professionals' perceptions. Patient Educ Couns. 2008, 73: 526-35. 10.1016/j.pec.2008.07.018.
Klein G: Naturalistic decision making. Hum Factors. 2008, 50: 456-460. 10.1518/001872008X288385.
Bucknall T: A gaze through the lens of decision theory toward knowledge translation science. Nurs Res. 2007, 56 (Suppl 4): 60-66.
Currey J, Botti M: Naturalistic decision making: A model to overcome methodological challenges in the study of critical care nurses' decision making about patients' hemodynamic status. Am J Crit Care. 2003, 12: 206-211.
Patel VL, Kaufman DR, Arocha JF: Emerging paradigms of cognition in medical decision making. J Biomed Inform. 2002, 35: 52-75. 10.1016/S1532-0464(02)00009-6.
Azjen I: The theory of planned behaviour. Organ Behav Hum Decis Process. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.
Hysong SJ, Best RG, Pugh JA: Audit and feedback and clinical practice guideline adherence: Making feedback actionable. Implement Sci. 2006, 1: 9-10.1186/1748-5908-1-9.
Carey M, Buchan H, Sanson-Fisher R: The cycle of change: Implementing best evidence clinical practice. Int J Qual Health Care. 2009, 21: 37-43. 10.1093/intqhc/mzn049.
This study and the cost of this publication is funded by the Canadian Institutes of Health Research through an operating grant and New Investigator in Knowledge Translation award (ARG) who took no part in the study design or decision to submit this manuscript for publication; and who will take no part in the collection, analysis and interpretation of data; or writing of subsequent manuscripts.
The authors declare that they have no competing interests.
ARG conceptualized and designed this study and obtained funding. She performed primary data collection, analysis, interpretation and report writing. MCB, LLC and JMG assisted with design of this study and data interpretation. VAP assisted with design of this study, independently reviewed data extracted from guidelines, and assisted with interpretation. All co-investigators contributed to report writing, and read and approved the final version of this manuscript.