Evaluation of an implementation facilitation strategy for settings that experience significant implementation barriers
© Ritchie et al. 2015
Published: 14 August 2015
Though several implementation and quality improvement strategies have been shown to be effective in implementing programs and practices in routine clinical settings [1, 2], little work has been done in developing and testing implementation strategies in settings that experience significant implementation barriers. This VA funded study evaluated a highly partnered implementation facilitation (IF) strategy  within the context of a Department of Veteran Affairs (VA) mandate for implementation of Primary Care-Mental Health Integration (PC-MHI) . The IF strategy consisted of a national expert external facilitator (EF) and two internal regional facilitators (IRFs) who partnered with regional, medical center, and clinic leadership and staff in two VA regional networks to implement PC-MHI. Facilitators helped partners design/adapt their PC-MHI programs, develop site-specific implementation plans, and identify/address implementation barriers. They also identified and engaged key stakeholders at all organizational levels; conducted academic detailing, marketing, staff training, patient education, formative evaluation, and audit and feedback; assisted with technical issues; and established learning collaboratives. The EF had expertise in the evidence-base for PC-MHI and implementation activities. The IRF had protected time to support implementation activities, was embedded within the clinical organization at the regional level, and was familiar with local and regional organizational structures, procedures, culture, and clinical processes. We used a quasi-experimental, Hybrid Type III design  and mixed methods to test effectiveness of the IF strategy and document IF activities. National VA MH leadership has adopted this IF strategy for sites facing challenges to adopting evidence-based practices [4, 6, 7]. This panel presents findings from the project's three components: A quantitative study of facilitation outcomes, a qualitative study of the facilitation process and its outcomes, and a qualitative study of facilitation skill transfer.
Quantitative outcomes of using facilitation in implementing primary care - mental health integration
The study tested the effectiveness of the implementation (IF) strategy hypothesizing that, compared to national technical assistance support alone, national support plus IF would improve implementation of PC-MHI. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) Framework guided testing of the IF strategy's effectiveness [8, 9].
Two regions were recruited to receive the IF strategy and two matched regions were recruited for comparison. Regional MH leadership identified primary care (PC) clinics unlikely to implement PC-MHI without assistance. PC clinics in comparison regions were matched to clinics in IF regions. The sample included 14 PC clinics, 174 PC providers and 98,758 PC patients. To evaluate implementation outcomes, administrative data was extracted for two six month periods, 9-15 months and 21-27 months following completion of an implementation plan at IF clinics. Generalized estimating equations were used to control for observations clustered within sites.
For the first six month period, PC patients at clinics receiving IF were more likely to be seen by PC-MHI providers (Reach) (OR = 8.93, p<0.001) than patients at comparison clinics. PC providers were more likely to refer at least one patient to PC-MHI (Adoption) (OR = 7.12, p = 0.029) than providers at comparison clinics and a greater proportion of PC providers' patients were referred to PC-MHI (Adoption) (β=0.027, p<0.001) at IF clinics. There was no difference between IF and comparison clinics in the likelihood of patients being referred for a first time visit to specialty mental health care (Effectiveness) or the percentage of patients receiving same day access to PC-MHI (Implementation). Similar results occurred during the second six month period (Maintenance).
This study documents the effectiveness of IF compared to technical assistance in the VA PC-MHI mandate and provides an evidence-based intervention for sites unable to implement programs without additional assistance. VA MH leadership has adopted the IF strategy for sites facing challenges adopting evidence-based practices.
Examining inside the black box of implementation facilitation: process and effects on program quality
We explored inside the IF black box to document how the process changes over time and in response to circumstances. Specifically, we conducted monthly qualitative debriefings with the facilitators and semi-structured interviews with the facilitators and clinicians and managers at clinics, affiliated medical centers and regional networks over a two and half year period. Additionally, we asked experts to rate program quality and fidelity to evidence at the IF clinics and their matched comparisons [10, 11].
Based on a qualitative content analysis, we determined that although certain IF activities tended to occur predominantly during particular periods, ever evolving context dictated the presence and intensity of most activities at particular times and in particular places. We observed systematic differences between the two regions and identified both regional and facilitator characteristics that may explain these differences. We also examined what facilitators, clinicians, and managers valued most about facilitation and found systematic variation. We explored the widely held assumption that facilitation activities fall into two broad categories, "doing" for others and "enabling" others to do things for themselves [1, 12–15]. We found that although some activities appear to fit exclusively into one category, most do not.
Additionally, we examined the effect of IF on clinics' ability to implement evidence-based and high quality programs. Midway through the study, seven IF but only three comparison clinics had implemented a program; experts rated IF clinic programs' quality and adherence to evidence most highly. At the end of the study, all IF but only five comparison clinics had programs. All but one IF clinic had a higher rated program than its comparison.
In summary, we found that IF can foster implementation of high quality and evidence-based practices. We also found that facilitation activities do not occur according to a defined series of stages but rather flexibly in response to local circumstances.
Transferring implementation knowledge and skills to improve healthcare delivery systems
We explored how experts in implementation facilitation (IF) can help healthcare system change agents learn how to facilitate implementation of evidence-based programs. For two and a half years, we conducted monthly debriefing interviews with a national expert external facilitator (EF) who was mentoring and coaching two internal regional facilitators (IRFs) in facilitating implementation of a VA policy initiative for Primary Care-Mental Health Integration (PC-MHI) at eight primary care clinics. Interviews focused primarily on the EF's efforts to help the IRFs become experts in IF processes. We also conducted two semi-structured qualitative interviews with each facilitator, midway through and at the end of the intervention.
Our qualitative content analysis revealed that although the EF helped IRFs learn general implementation facilitation knowledge and skills, the EF also identified IRFs' individual strengths and weaknesses and tailored mentoring and coaching activities to their characteristics. The EF used a variety of methods to help IRFs learn IF skills, including both active methods (providing information, modeling and coaching) and participatory ones. She also used cognitive supports (making thinking visible, using heuristics, sharing IF experiences) and psychosocial supports, as well strategies to promote self-learning. Additionally, the EF tailored the process to sites' implementation needs. Over time, the EF pulled back from IRFs, increasingly turning responsibility for IF activities over to them. IRFs responded differently to this process with one IRF independently "breaking away" and the other being "pushed out of the nest." In addition to helping IRFs learn the skills they needed for facilitating PC-MHI implementation, the EF helped IRFs to identify and modify interpersonal styles that could hinder success of facilitation efforts.
This study addresses the critical but understudied area of how implementation scientists can transfer facilitation skills that incorporate evidence-based implementation interventions and strategies to internal change agents to help healthcare organizations implement effective programs.
This presentation was based on work supported by the U.S. Department of Veterans Affairs Health Services Research and Development Service SDP 08-316.
- Stetler CB, Legro MW, Rycroft-Malone J, Bowman C, Curran G, Guihan M, et al: Role of external facilitation in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science. 2006, 1: 23-10.1186/1748-5908-1-23.View ArticlePubMedPubMed CentralGoogle Scholar
- Kirchner J, Edlund CN, Henderson K, Daily L, Parker LE, Fortney JC: Using a multi-level approach to implement a primary care mental health (PCMH) program. Families, Systems & Health. 2010, 28 (2): 161-174.View ArticleGoogle Scholar
- Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney JC: Outcomes of a partnered facilitation strategy to implement primary care-mental health. J Gen Intern Med. 2014, 29 Suppl 4: 904-912.View ArticlePubMedGoogle Scholar
- Pomerantz AS, Kearney LK, Wray LO, Post EP, McCarthy JF: Mental health services in the medical home in the Department of Veterans Affairs: Factors for successful integration. Psychological Services. 2014, 11 (3): 243-253.View ArticlePubMedGoogle Scholar
- Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C: Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care. 2012, 50 (3): 217-226. 10.1097/MLR.0b013e3182408812.View ArticlePubMedPubMed CentralGoogle Scholar
- Kirchner JE, Kearney LK, Ritchie MJ, Dollar KM, Swensen AB, Schohn M: Lessons learned through a national partnership between clinical leaders and researchers. Psychiatric Services. 2014, 65 (5): 577-579. 10.1176/appi.ps.201400054.View ArticlePubMedGoogle Scholar
- Ritchie MJ, Dollar KM, Kearney LK, Kirchner JE: Responding to needs of clinical operations partners: Transferring implementation facilitation knowledge and skills. Psychiatric Services. 2014, 65 (2): 141-143. 10.1176/appi.ps.201300468.View ArticlePubMedGoogle Scholar
- Glasgow RE, McKay HG, Piette JD, Reynolds KD: The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management?. Patient Education and Counseling. 2001, 44 (2): 119-127. 10.1016/S0738-3991(00)00186-5.View ArticlePubMedGoogle Scholar
- Glasgow RE: Translating research to practice. Lessons learned, areas for improvement, and future directions. Diabetes Care. 2003, 26 (8): 2451-2456. 10.2337/diacare.26.8.2451.View ArticlePubMedGoogle Scholar
- Parker LE, dePillis E, Altschuler A, Rubenstein LV, Meredith LS: Balancing participation and expertise: A comparison of locally and centrally managed health care quality improvement within primary care practices. Qual Health Res. 2007, 17 (9): 1268-1279. 10.1177/1049732307307447.View ArticlePubMedGoogle Scholar
- Rubenstein LV, Parker LE, Meredith LS, Altschuler A, dePillis E, Hernandez J, et al: Understanding team-based quality improvement for depression in primary care. Health Services Research. 2002, 37 (4): 1009-1029. 10.1034/j.1600-0560.2002.63.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Heron J: The complete facilitators' handbook. 1989, LondonGoogle Scholar
- Harvey G, Wensing M: Methods of evaluation of small scale quality improvement projects. Quality and Safety in Health Care. 2003, 12 (3): 210-214.View ArticlePubMedPubMed CentralGoogle Scholar
- Dogherty E, Harrison M, Baker C, Graham I: Following a natural experiment of guideline adaptation and early implementation: A mixed-methods study of facilitation. Implementation Science. 2012, 7: 9-10.1186/1748-5908-7-9.View ArticlePubMedPubMed CentralGoogle Scholar
- Lombarts MJ, Klazinga NS, Redekop KK: Measuring the perceived impact of facilitation on implementing recommendations from external assessment: lessons from the Dutch visitatie programme for medical specialists. Journal of Evaluation in Clinical Practice. 2005, 11 (6): 587-597. 10.1111/j.1365-2753.2005.00595.x.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.