This article has Open Peer Review reports available.
An exploration of how guideline developer capacity and guideline implementability influence implementation and adoption: study protocol
© Gagliardi et al; licensee BioMed Central Ltd. 2009
Received: 20 April 2009
Accepted: 02 July 2009
Published: 02 July 2009
Practice guidelines can improve health care delivery and outcomes but several issues challenge guideline adoption, including their intrinsic attributes, and whether and how they are implemented. It appears that guideline format may influence accessibility and ease of use, which may overcome attitudinal barriers of guideline adoption, and appear to be important to all stakeholders. Guideline content may facilitate various forms of decision making about guideline adoption relevant to different stakeholders. Knowledge and attitudes about, and incentives and capacity for implementation on the part of guideline sponsors may influence whether and how they develop guidelines containing these features, and undertake implementation. Examination of these issues may yield opportunities to improve guideline adoption.
The attributes hypothesized to facilitate adoption will be expanded by thematic analysis, and quantitative and qualitative summary of the content of international guidelines for two primary care (diabetes, hypertension) and institutional care (chronic ulcer, chronic heart failure) topics. Factors that influence whether and how guidelines are implemented will be explored by qualitative analysis of interviews with individuals affiliated with guideline sponsoring agencies.
Previous research examined guideline implementation by measuring rates of compliance with recommendations or associated outcomes, but this produced little insight on how the products themselves, or their implementation, could be improved. This research will establish a theoretical basis upon which to conduct experimental studies to compare the cost-effectiveness of interventions that enhance guideline development and implementation capacity. Such studies could first examine short-term outcomes predictive of guideline utilization, such as recall, attitude toward, confidence in, and adoption intention. If successful, then long-term objective outcomes reflecting the adoption of processes and associated patient care outcomes could be evaluated.
Research, practice, and policy in the health care sector focus on improving the organization, delivery, and outcomes of care, while optimizing efficiency. Critical to achieving these objectives is the need for compliance with best practice according to currently available knowledge generated through research. Knowledge syntheses such as practice guidelines provide the evidence base for health care decision making [1, 2]. Their development, dissemination, and implementation are intended to improve quality of care. Unfortunately, their impact remains limited as there continue to be many documented circumstances where they have not been adopted into practice [3–6]. Several issues challenge guideline adoption, including their intrinsic attributes, and whether and how they are implemented.
The Appraisal of Guidelines Research and Evaluation (AGREE) instrument assesses guidelines based on their scope and purpose, stakeholder involvement, rigour of development, clarity of presentation, editorial independence, and applicability . The criteria for applicability specify that, to improve uptake, guidelines should include information about anticipated organizational barriers, costs associated with adoption, and measures for audit and monitoring. The Guideline Implementability Appraisal (GLIA) instrument also recommends that guidelines explicitly identify the anticipated impact of adoption on individuals and organizations, and include measures by which performance of the recommended medical interventions or services can be evaluated . Both tools were proposed by guideline experts, and may not reflect the features important to target guideline users, including clinicians, managers, and policy makers.
Studies eliciting clinician views on guideline attributes that influence utilization are few. Interviews were conducted with 25 general practitioners in the United Kingdom to understand guideline qualities associated with adoption of recommendations for asthma, coronary heart disease, depression, epilepsy, and menorrhagia . In addition to credibility of the source and content, they also desired information about the resources required to deliver recommended care, and recommendations formatted in step-wise fashion to highlight how and when to deliver care. During focus groups, target users of the American College of Occupational and Environmental Medicine practice guidelines stated that the guidelines were too complicated to use quickly, and suggested a variety of easier-to-read formats . A single observational study examined the association between guideline attributes and use by general practitioners in the Netherlands . Over a three-month period, 61 general practitioners documented the details of patient care visits during which one of ten national guidelines was relevant. Out of 12,880 decisions made by physicians, 61% complied with guidelines. Recommendations that had been categorized as evidence-based, provided clear and specific advice on actions, and that did not require a change in existing practice routines, including re-organization of staff, acquisition of extra resources, and learning of new knowledge or skills achieved higher compliance. Self-doubt and training needs were identified as the two issues most influencing adoption by primary care teams of the National Institute for Health and Clinical Excellence's Schizophrenia guideline . An expert panel engaged to consider five guidelines for musculoskeletal disorders that had been judged by the AGREE instrument to have excellent technical quality found them to be only moderately acceptable, citing lack of relevance to usual practice . Notably the applicability domain scored low for most of the musculoskeletal guidelines (range 0.17 to 0.76 out of 1.00). In Ontario, Canada a total of 488 clinicians were sent 1,494 new questionnaires regarding attitude to 34 clinical practice guidelines produced between 1999 and 2002 . Endorsement of, and intent to use the guidelines were predicted by applicability, acceptability, and comparative value. Thus, in addition to the elements in the AGREE and GLIA tools, clinicians appear to also value ease of use, clarity of evidence, competency and training requirements, and identification of other practice changes required to accommodate the recommendations.
Fewer studies have investigated the guideline attributes considered important, or that lead to guideline utilization by managers and policy makers. A systematic review of 24 studies involving 2,041 interviews with health policy makers found that inclusion of summaries with policy recommendations was commonly suggested as a factor that could enhance guideline utilization . A survey of 899 managers and policy decision makers from across Canada revealed that accessibility through the internet increased guideline utilization by all decision makers in government, regional health authorities, and hospitals, while adaptability influenced guideline utilization by hospital managers .
Limited utilization of guidelines may depend on whether and how they are implemented. Recent syntheses of guideline implementation research found that there is considerable variation in the observed effects within and across interventions by condition and setting of care . Outside of experimental research there are few evaluations of whether and how guidelines are actively implemented . Those available suggest that the responsibility for guideline implementation is unclear, resources for implementation are lacking and, as a result, many guidelines are passively distributed. Interviews and focus groups with 47 government policy officers, agencies, practice guideline developers, and practitioners in Australia about the implementation of six practice guidelines revealed that no uniform strategy had been employed apart from mailing and posting on a web site . Telephone interviews with health professionals in the United Kingdom revealed they experienced difficulty in acquiring resources to fund guideline implementation, often turning to 'soft' money from pharmaceutical companies for educational meetings, a traditional type of continuing education that is considered largely ineffective .
Lack of knowledge about implementation processes may also contribute to the reliance on passive distribution methods. Health professionals have acknowledged that they are unfamiliar with, or confused about the concept and practice of implementation . Emergency medicine professionals from 16 countries highlighted their lack of skill in implementation . Interviews with individuals from 33 international research funding agencies revealed a widespread need to increase our knowledge about, and the practice of implementation .
To date there has not been a systematic analysis of guideline features that may improve adoption, or the factors that influence whether and how guidelines are implemented by sponsoring organizations. Their examination may reveal opportunities to improve guideline adoption. The purpose of the proposed research is to: develop a conceptual framework of guideline attributes that could be used to characterize the ease with which they can be adopted; and explore how sponsoring organizations implement guidelines, describing factors that influence these processes.
Clinicians, managers, and policy makers have suggested various guideline attributes that may enhance their 'implementability' [9–16]. Based on these studies, it appears that implementable features may improve attitude to the guidelines and confidence in decision making about adoption. Evidence is just one of several factors that inform clinical decision making . Clinicians must often use experiential or shared decision making to consider what is best for and desired by those receiving care, but have expressed uncertainty about how to balance professional judgment with patient preferences, and the need for informational resources to support these processes. Clinician decisions about guideline adoption are also influenced by the availability and mobilization of organizational- or system-level resources, which are governed by the decisions of managers and policy makers who must consider not only evidence, but the benefits and risks associated with adoption, and the competing interests of multiple stakeholders, a process called naturalistic decision making . Format elements of implementability are those that influence accessibility and ease of use, which may overcome attitudinal barriers of guideline adoption, and appear to be important to all stakeholders. Content elements of implementability are those that facilitate evidence-informed, experiential, shared and naturalistic decision making, stimulating confidence in whether and how to adopt guideline recommendations by different stakeholders.
Developing a conceptual framework of guideline implementability
The attributes hypothesized to facilitate adoption will be assessed and expanded by thematic analysis of the content of current guidelines. Published practice guidelines will be selected from among those reviewed (or the most recent version) by the Guidelines Advisory Committee http://www.gacguidelines.ca, a program in Ontario, Canada that identifies, appraises using the AGREE instrument, endorses and synthesizes guidelines, and were judged as high quality for two topics reflecting primary care (diabetes, hypertension) and two topics reflecting institutional care (chronic ulcer, chronic heart failure). Eligible guidelines include those that cover comprehensive management of these conditions, and are publicly available. Full versions of selected guidelines and adjunct products will be obtained. Data on presence of format and content features identified in the conceptual framework, or additional such features will be noted. Two individuals will independently extract data, then meet to compare findings and resolve differences. Extracted data will be tabulated. Most elements will be summarized quantitatively with mean, median, or frequency. Findings will be examined to discuss the number of guidelines addressing each element of implementability overall and by topic. Details of implementability content will be analyzed using Mays' narrative review method, based on verbatim reporting of information rather than statistical summary or conceptual analysis .
Exploring factors influencing guideline implementation
Individuals affiliated with organizations that issue practice guidelines will be interviewed to explore the factors that influence guideline implementation. Standard methods of qualitative research will be used for sampling, data collection, and data analysis . Individuals involved in sponsoring, developing, or implementing Canadian guidelines for four topics examined by content analysis will be identified on organizational web sites and through preliminary discussions with key contacts at those organizations (known sponsor approach). Ten consecutive individuals at each organization will be purposively recruited to represent different roles and perspectives, including sponsors, executives, managers, members of guideline development panels, and other individuals involved in coordinating guideline development or implementation, both internal and external to the involved programs, for a minimum total of 40 interviews. During interviews participants will be asked to recommend additional stakeholders that could provide relevant information (snowball sampling). Detailed information from representative, rather than a large number of cases is needed in qualitative research. Sampling is concurrent with data collection and analysis, and proceeds until no further unique themes emerge from successive interviews (grounded approach). If after 40 interviews new information continues to emerge, further interviews will be pursued. Data will be collected by conducting semi-structured telephone interviews with consenting participants. To enhance validity, a single investigator will conduct the interviews for internal consistency. They will be audio-recorded, then transcribed verbatim by an external professional. An interview guide will be pilot tested on one manager and clinician. Participants will be asked about their knowledge and perceptions of implementation; resources that were consulted to guide implementation decisions; their organization's incentives and capacity to implement guidelines; processes actually used for implementation; and suggestions for improving implementation capacity and processes. Unique themes will be identified in an inductive, iterative manner as previously described. Coded transcript text will be tabulated by theme and professional role.
Guideline implementability and implementation have not been systematically investigated to identify how they could be modified to improve guideline adoption. The development of a conceptual framework for implementability will be based on international guidelines for a variety of topics and therefore broadly applicable. Factors influencing the capacity for guideline implementation will be explored among a relatively small sample of participants in Ontario, Canada so those findings may not be relevant to guideline developers or sponsors in other settings with different types of health care systems, or where the organization of guideline development may differ from that in Ontario. Still, health systems worldwide experience non-compliance with guideline-recommended care, and seek novel insight into, and mechanisms for improving guideline utilization. The results of this study may provide a useful framework by which others can examine their capacity for guideline development and implementation.
With respect to policy and practice, this research may highlight that guideline development programs are not equipped to undertake implementation. Development of implementation capacity may be required to ensure that guideline sponsors and other groups seeking to improve quality of care have the required resources to achieve implementation. With respect to research, the findings will be used to refine the proposed conceptual framework, which could then inform ongoing studies. By identifying factors amenable to modification, for example, incorporation of actionable content within or as an adjunct product to guidelines, we establish a theoretical basis upon which to conduct experimental studies to compare the cost-effectiveness of such processes on the thoroughness of guideline implementation. Such studies could first examine short-term outcomes predictive of guideline utilization, such as recall, attitude toward, confidence in, and adoption intention . If successful, then long-term objective outcomes reflecting the adoption of processes and associated patient care outcomes could be evaluated.
This study and the cost of this publication is funded by the Canadian Institutes of Health Research through an operating grant and New Investigator in Knowledge Translation award (ARG) who took no part in the study design or decision to submit this manuscript for publication; and who will take no part in the collection, analysis and interpretation of data; or writing of subsequent manuscripts.
- Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L: Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966–1998. J Gen Intern Med. 2006, 21 (Suppl 2): S14-20.PubMedPubMed CentralGoogle Scholar
- Browman GP, Levine MN, Mohide A, Hayward RSA, Pritchard KI, Gafni A, Laupacis A: The practice guidelines development cycle: A conceptual tool for practice guidelines development and implementation. J Clin Oncol. 1995, 13: 502-512.PubMedGoogle Scholar
- McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348: 2635-2645. 10.1056/NEJMsa022615.View ArticlePubMedGoogle Scholar
- FitzGerald JM, Boulet LP, McIvor RA, Zimmerman S, Chapman KR: Asthma control in Canada remains suboptimal: the Reality of Asthma Control (TRAC) study. Can Respir J. 2006, 13: 253-259.View ArticlePubMedPubMed CentralGoogle Scholar
- Brown LC, Johnson JA, Majumdar SR, Tsuyuki RT, McAlister FA: Evidence of suboptimal management of cardiovascular risk in patients with type 2 diabetes mellitus and symptomatic atherosclerosis. CMAJ. 2004, 171 (10): 1189-1192. 10.1503/cmaj.1031965.View ArticlePubMedPubMed CentralGoogle Scholar
- Latosinsky S, Fradette K, Lix L, Hildebrand K, Turner D: Canadian breast cancer guidelines: have they made a difference?. CMAJ. 2007, 176 (6): 771-776. 10.1503/cmaj.060854.View ArticlePubMedPubMed CentralGoogle Scholar
- Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J: Towards evidence based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003, 15: 31-45. 10.1093/intqhc/15.1.31.View ArticlePubMedGoogle Scholar
- Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, O'Connell R: The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005, 5: 23-10.1186/1472-6947-5-23.View ArticlePubMedPubMed CentralGoogle Scholar
- Rashidian A, Eccles MP, Russell I: Falling on stony ground? A qualitative study of implementation of clinical guidelines' prescribing recommendations in primary care. Health Policy. 2008, 85: 148-161. 10.1016/j.healthpol.2007.07.011.View ArticlePubMedGoogle Scholar
- Harris JS, Mueller KL, Low P, Phelan J, Ossler C, Koziol-McLain J, Glass LS: Beliefs about and use of occupational medicine practice guidelines by case managers and insurance adjusters. J Occup Environ Med. 2000, 42: 370-376. 10.1097/00043764-200004000-00007.View ArticlePubMedGoogle Scholar
- Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H: Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998, 317 (7162): 858-861.View ArticlePubMedPubMed CentralGoogle Scholar
- Michie S, Pilling S, Garety P, Whitty P, Eccles MP, Johnston M, Simmons J: Difficulties implementing a mental health guideline: an exploratory investigation using psychological theory. Implement Sci. 2007, 2: 8-10.1186/1748-5908-2-8.View ArticlePubMedPubMed CentralGoogle Scholar
- Nuckols TK, Lim YW, Wynn BO, Mattke S, MacLean CH, Harber P, Brook RH, Wallace P, Garland RH, Asch S: Rigorous development does not ensure that guidelines are acceptable to a panel of knowledgeable providers. J Gen Intern Med. 2008, 23: 37-44. 10.1007/s11606-007-0440-9.View ArticlePubMedGoogle Scholar
- Brouwers MC, Graham ID, Hanna SE, Cameron DA, Browman GP: Clinicians' assessments of practice guidelines in oncology. Int J Technol Assess Health Care. 2004, 20: 421-426. 10.1017/S0266462304001308.View ArticlePubMedGoogle Scholar
- Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers' perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002, 7: 239-244. 10.1258/135581902320432778.View ArticlePubMedGoogle Scholar
- Ouimet M, Landry R, Amara N, Belkhodja O: What factors induce health care decision-makers to use clinical guidelines? Evidence from provincial health ministries, regional health authorities and hospitals in Canada. Soc Sci Med. 2006, 62: 964-976. 10.1016/j.socscimed.2005.06.040.View ArticlePubMedGoogle Scholar
- Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8 (6): iii-iv. 1-72View ArticlePubMedGoogle Scholar
- Ellis P, Robinson P, Ciliska D, Armour T, Brouwers M, O'Brien MA, Sussman J, Raina P: A systematic review of studies evaluating diffusion and dissemination of selected cancer control interventions. Health Psychol. 2005, 24: 488-500. 10.1037/0278-6184.108.40.2068.View ArticlePubMedGoogle Scholar
- Armstrong R, Waters E, Crockett B, Keleher H: The nature of evidence resources and knowledge translation for health promotion practitioners. Health Promot Int. 2007, 22: 254-260. 10.1093/heapro/dam017.View ArticlePubMedGoogle Scholar
- Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell , Robinson N: Lost in knowledge translation: time for a map?. J Contin Educ Health Prof. 2006, 26 (1): 13-24. 10.1002/chp.47.View ArticlePubMedGoogle Scholar
- Arnold LK, Alomran H, Anantharaman V, Halpern P, Hauswald M, Malmquist P, Molyneux E, Rajapakse B, Ranney M, Razzak J: Knowledge translation in international emergency medical care. Acad Emerg Med. 2007, 14 (11): 1047-1051.View ArticlePubMedGoogle Scholar
- Tetroe JM, Graham ID, Foy R, Robinson N, Eccles MP, Wensing M, Durieux P, Legare F, Nielson CP, Adily A, Ward JE, Porter C, Shea B, Grimshaw JM: Health research funding agencies' support and promotion of knowledge translation: an international study. Milbank Q. 2008, 86: 125-155. 10.1111/j.1468-0009.2007.00515.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Rogers EM: Diffusion of innovations. 1995, New York: The Free Press, 4Google Scholar
- Denis JL, Hebert Y, Langley A, Lozeau D, Trottier LH: Explaining diffusion patterns for complex health care innovations. Health Care Manage Rev. 2002, 27: 60-73.View ArticlePubMedGoogle Scholar
- Using Knowledge and Evidence in Health Care – Multidisciplinary Perspectives. Edited by: Louise Lemieux-Charles, Francois Champagne. 2004, Toronto: University of Toronto PressGoogle Scholar
- Mays N, Pope C, Popay J: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005, 10 (Suppl 1): 6-20. 10.1258/1355819054308576.View ArticlePubMedGoogle Scholar
- Strauss A, Corbin J: Basics of qualitative research. Techniques and procedures for developing grounded theory. 1998, Thousand Oaks, California: Sage PublicationsGoogle Scholar
- Azjen I: The theory of planned behaviour. Org Behav Human Dec Proc. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.