Skip to main content
Download PDF
Download ePub
  • Systematic review
  • Open access
  • Published:

Application of the Expert Recommendations for Implementing Change (ERIC) compilation of strategies to health intervention implementation in low- and middle-income countries: a systematic review

Implementation Science volume 18, Article number: 56 (2023) Cite this article

Abstract

Background

The Expert Recommendations for Implementing Change (ERIC) project developed a compilation of implementation strategies that are intended to standardize reporting and evaluation. Little is known about the application of ERIC in low- and middle-income countries (LMICs). We systematically reviewed the literature on the use and specification of ERIC strategies for health intervention implementation in LMICs to identify gaps and inform future research.

Methods

We searched peer-reviewed articles published through March 2023 in any language that (1) were conducted in an LMIC and (2) cited seminal ERIC articles or (3) mentioned ERIC in the title or abstract. Two co-authors independently screened all titles, abstracts, and full-text articles, then abstracted study, intervention, and implementation strategy characteristics of included studies.

Results

The final sample included 60 studies describing research from all world regions, with over 30% published in the final year of our review period. Most studies took place in healthcare settings (n = 52, 86.7%), while 11 (18.2%) took place in community settings and four (6.7%) at the policy level. Across studies, 548 distinct implementation strategies were identified with a median of six strategies (range 1–46 strategies) included in each study. Most studies (n = 32, 53.3%) explicitly matched implementation strategies used for the ERIC compilation. Among those that did, 64 (87.3%) of the 73 ERIC strategies were represented. Many of the strategies not cited included those that target systems- or policy-level barriers. Nearly 85% of strategies included some component of strategy specification, though most only included specification of their action (75.2%), actor (57.3%), and action target (60.8%). A minority of studies employed randomized trials or high-quality quasi-experimental designs; only one study evaluated implementation strategy effectiveness.

Conclusions

While ERIC use in LMICs is rapidly growing, its application has not been consistent nor commonly used to test strategy effectiveness. Research in LMICs must better specify strategies and evaluate their impact on outcomes. Moreover, strategies that are tested need to be better specified, so they may be compared across contexts. Finally, strategies targeting policy-, systems-, and community-level determinants should be further explored.

Trial registration

PROSPERO, CRD42021268374.

Peer Review reports

Background

The past two decades have been marked by rapid growth in the field of implementation science to address the large research-to-practice gap across contexts and health areas [1]. In recent years, the field’s focus has shifted from defining barriers and facilitators of implementing evidence-based practices to identifying strategies that effectively address and overcome these barriers. Implementation strategies are generally defined as the approaches or techniques used to enhance the adoption, implementation, sustainment, and scale-up of an evidence-based practice [2, 3]. These strategies vary in complexity and can target determinants at the intervention-, patient-, provider-, organization-, community-, policy-, and funding levels [4, 5].

Though the evidence base on implementation strategies is growing, current data on strategy effectiveness is mixed, with high variation in strategy effects observed across studies and outcomes [6,7,8,9,10,11]. Several reasons may contribute to this variation. It may be that certain strategies are not sufficient to improve implementation outcomes across contexts; it may also be that strategies were not appropriately matched to the contextual determinants or tailored to the setting [12]. However, reporting on implementation strategies often lacks the necessary information to determine why a strategy was or was not effective; for example, information on how a strategy was selected, adapted, and operationalized and whether or not the strategy was carried out as intended [7, 13,14,15,16,17]. As such, calls for consistent, detailed reporting of implementation strategies have emerged in tandem with calls for increased research on strategy effectiveness [2, 3, 18,19,20].

To aid reporting efforts, the field has developed taxonomies of implementation strategies [21, 22] and a methodology for specifying implementation strategies [3]. One strategy taxonomy, the Expert Recommendations for Implementing Change (ERIC) project, built upon a narrative review of the literature [23] and used a modified Delphi process to develop a compilation of implementation strategies, comprising 73 discrete strategies [21] which can be further grouped into nine thematic clusters [24]. The ERIC compilation has become the most commonly used taxonomy of strategies in the field of implementation science, with over 3000 citations. It has enabled a standardized language for naming implementation strategies that have been used to characterize implementation efforts both in prospective and retrospective analyses [25,26,27,28,29,30]. To support standardized strategy specification, Proctor et al. [3] developed guidelines to help stakeholders operationalize strategies based on specific domains, including the strategy actor, action, action target, temporality, dose, implementation outcome affected, and justification. These specification guidelines are consistent with the Patient Centered Outcomes Research Institute's Standards for Studies of Complex Interventions [31], and have the potential to not only improve our understanding of implementation strategy mechanisms, but also the required parameters for replication in other research and practice settings.

In low- and middle-income countries (LMICs), implementation science has become a key tool for helping to bridge the research-to-practice gap that is larger than that of high-income countries [32, 33]. As such, the need to identify effective implementation strategies for efficient, effective delivery of evidence-based practices is imperative in these settings. In recent years, formal implementation research in LMICs has expanded as funding sources increasingly recognize the utility of this work. There is now a growing evidence base for using implementation frameworks [34, 35] and measures [36, 37], and specific determinants to implementation [38] in LMICs have been identified. However, little is known about effective implementation strategies in these settings [39].

The purpose of the present study is to report on the application of the ERIC compilation of implementation strategies in LMICs and provide recommendations to the field for improving its application moving forward. Our aims are twofold: (i) to systematically review the literature on the use of ERIC strategies in LMICs, including which specific strategies have been included in the research, how they were selected, how the strategies were used (i.e., specification and targeted intervention/health condition/population), and how they were adapted; and (ii) to assess evidence for the effectiveness of specific ERIC implementation strategies in LMICs.

Methods

We registered our systematic review protocol in the International Prospective Register of Systematic Reviews (PROSPERO # CRD42021268374) and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [40]. See Additional file 1 for the completed PRISMA checklist.

Search strategy

We searched PubMed, CINAHL, PsycINFO, CINAHL, EMBASE, SCOPUS, and Web of Science until March 27, 2023, to identify original peer-reviewed research in any language that cited either (1) the original implementation strategies compilation paper [23], (2) the ERIC compilation [21], or (3) the strategy categorization [24], or that mentioned the ‘Expert Recommendations for Implementing Change’ or ‘ERIC’ in the title or abstract, and that took place within an LMIC. LMIC classification was determined based on World Bank criteria [41]. The full search strategy for all databases is presented in Additional file 2.

Study selection

Covidence was used to remove duplicate studies and to conduct study screening [42]. A mix of two authors from a team of five (KL, CK, CG, AG, and BW) independently screened all titles, abstracts, and full-text articles, and noted reasons for excluding studies during full-text review. Studies passed the title/abstract screening stage if the title or abstract referenced the implementation of a health-related intervention and if it was possible that the study had been conducted in an LMIC. Studies passed the full-text screening stage if all criteria above were met and the study described the use of the ERIC compilation in implementation strategy selection, development, or classification (i.e., manuscripts that cited ERIC—for example, in the introduction or discussion—without indicating the application to the study strategies were excluded). Discrepancies in eligibility assessments were resolved through discussion until a consensus was reached.

Data abstraction

Five authors (KL, CK, CG, AG, and BW) independently piloted a structured abstraction form with two studies each using a shared Google Sheets spreadsheet; all co-authors reviewed, critiqued, and approved the form. One of the two authors (CK and KL) then abstracted the study, intervention, and implementation strategy characteristics for the remaining studies (Additional file 3), while the other author-verified each abstraction, and then resolved any disagreement through discussion.

At the study level, we abstracted study settings, objectives, study design, and methods, whether the manuscript reported a study protocol or study results, any implementation research frameworks used, years of data collection, study populations, implementation outcomes reported, patient health and other outcomes reported, study limitations, and conclusions or lessons learned. We noted the types of independent variables represented in each study (i.e., intervention, implementation strategy, or context) based on which were systematically varied. Within each study, we also collected intervention names, intervention descriptions, associated health conditions, and target populations.

We then abstracted the discrete implementation strategies used and described in each study. At the implementation strategy level, we included descriptions of each strategy and noted whether the strategies were explicitly mapped to the ERIC strategy compilation, or to other strategy taxonomies (e.g., Behavior Change Wheel) [43]. We then abstracted each of the components of implementation strategy specification [3]; actor, action, action target, temporality, dose, implementation outcome(s) affected, and justification. We also noted any description of the hypothesized mechanism of action [19, 44], any description of adaptations to the implementation strategy [18, 45], and any assessment of implementation strategy fidelity [46]. Finally, we noted whether implementation strategy effect estimates were reported. Risk of bias was not assessed, as only one study evaluated strategy effectiveness and thus no meta-analysis of effectiveness was conducted.

Analysis

Percentages were calculated for all categorical variables; these were used to summarize study, intervention, and implementation strategy characteristics. Quantitative meta-analysis of study findings was not possible given the heterogeneity in research questions and outcomes as well as the insufficient numbers of studies evaluating implementation strategy effects.

Results

The database search yielded 659 articles, of which 441 were duplicates. We screened the remaining 218 article titles and abstracts and excluded 88, leaving 130 for full-text review. Of these, 41 were excluded that did not use ERIC (i.e., only cited ERIC in the manuscript introduction or discussion, without application to the present study), 12 that did not take place in an LMIC, 11 that did not meet multiple inclusion criteria, 5 that were not peer-reviewed, and 1 that was not primary research (Fig. 1).

Fig. 1
figure 1

PRISMA 2020 flowchart of systematic review

Full size image

The final sample included 60 studies (Table 1, see Additional file 3 for individual study characteristics), all published in English. The first study using ERIC in an LMIC was published in 2016, and the number of studies using ERIC in LMICs increased over the years, from just 1 in 2016 to 20 in 2022. Studies included data collected in all six WHO Regions, with the majority being conducted in the African region (n = 36, 60.0%). Most studies focused on healthcare settings (n = 52, 86.7%), while just 11 (18.3%) focused on community and four (6.7%) on policy-level settings. The most common health conditions targeted in studies were infectious diseases (n = 19, 31.7%), maternal and child health (n = 10, 16.7%), and mental health and substance use (n = 10, 16.7%). Two studies did not focus on a specific health condition, but rather on strategies for implementing clinical trial recruitment and building a national framework for research.

Table 1 Study-level descriptive statistics (n = 60)
Full size table

Nearly three-quarters of studies described empirical research (n = 43, 71.7%), the others being protocols for studies not yet completed (n = 17, 23.3%). Study populations included patients (n = 31, 51.7%), providers (n = 43, 71.7%), policymakers (n = 14, 23.3%), community members (n = 8, 13.3%), and researchers (n = 6, 10.0%); 34 (56.7%) studies included more than one of these study populations. Nineteen (31.7%) articles described formative implementation strategy design and nine (15.0%) described retrospective strategy specification. The majority (n = 36, 60%) of studies included an impact evaluation, the most common design being quasi-experimental with no control (n = 13, 21.7%) and a cluster randomized control trial (n = 9, 15.0%). Of the 17 studies that included a control, n = 9 tested the intervention and n = 7 tested the implementation strategy. Implementation frameworks were used by 47 (78.3%) studies, with 36 (60.0%) citing implementation determinant frameworks, 24 (40.0%) citing evaluation frameworks, and seven (11.7%) citing a process framework. A total of 44 (73.4%) studies evaluated implementation outcomes, most commonly adoption (n = 29), acceptability (n = 27), and fidelity (n = 26); 34 (56.7%) studies evaluated multiple implementation outcomes. Under half of studies (n = 25, 41.7%) evaluated health outcomes.

Across the 60 studies, 548 strategies were proposed, 282 (51.5%) of which were planned for future delivery and 266 (48.5%) had been delivered in the study. The total number of strategies described per study ranged from 1 to 46 (median = 6). Despite all 60 studies referencing the use of ERIC, just 32 (53.3%) studies explicitly matched specific implementation strategies used in the study to a specific ERIC strategy (Table 2). One study described strategy development being guided by other frameworks in addition to ERIC [47,48,49]; all other studies cited only ERIC as the guiding framework. The most commonly utilized ERIC strategies were (1) conduct educational meetings (n = 16, 2.9% of all 548 strategies proposed across studies); (2) audit and provide feedback (n = 15, 2.7%); (3) assess for readiness and identify barriers and facilitators (n = 13, 2.4%); and (4) build a coalition (n = 12, 2.2%). Of the 73 ERIC strategies, 9 (12.3%) were not cited at all. These include altering patient/consumer fees, changing liability laws, developing disincentives, making billing easier, preparing patients/consumers to be active participants, starting a dissemination organization, using capitated payments, using other payment schemes, and visiting other sites. Strategies from all ERIC strategy categories were described, though 5 of 9 (55.5%) strategies in the Utilize Financial Strategies category were not used.

Table 2 Descriptive statistics of implementation strategies
Full size table

Eight (13.3%) studies did not include any component of strategy specification (i.e., they named strategies but did not describe them at all), representing 88 (16.1%) of the 548 strategies described across studies. Among the strategies that were specified (Table 3), the most common components described were action (n = 412, 75.2%), action target (n = 333, 60.8%), and actor (n = 314, 57.3%). The study team itself comprised the majority of actors specified for implementation strategies, accounting for 185 (58.9%) of the 314 actors specified; action targets were most commonly healthcare providers, accounting for 196 (58.9%) of 333 action targets specified. The least commonly specified components were fidelity (n = 17, 3.1%), adaptation (n = 100, 18.2%), action mechanism (n = 129, 23.5%), and targeted implementation outcomes (n = 148, 27.1%). Only 12 (20.0%) studies described adaptation of implementation strategies, of which two (3.3%) described the use of Implementation mapping and one the use of human-centered design (1.7%) to guide adaptation, while nine (15.0%) described more general stakeholder engagement without the use of a specific framework of the process. Only one (0.5%) strategy was tested for independent effects: Gachau et al. found that audit and feedback significantly improved 24 of 34 indicators of pediatric guideline adherence in Kenya [50].

Table 3 Descriptive statistics of implementation strategy specification
Full size table

Discussion

In the present systematic review, we found 60 studies that cited the use of ERIC strategies in LMIC settings. These studies included data from all WHO regions and focused on diverse health issues, with over 35% published in the final year of our review period, indicating a growing application of ERIC in LMICs. However, just over half explicitly matched implementation strategies employed to an ERIC strategy and 16% of strategies did not include any components of strategy specification. Moreover, a minority of studies employed randomized trials or high-quality quasi-experimental designs with controls, and just one study evaluated implementation strategy effectiveness.

While nearly half of the included studies did not explicitly match implementation strategies used in the study to a specific ERIC strategy, several notable points arose from the ERIC strategies that were reported. First, a wide variety of ERIC strategies were identified, with 88% of the 73 ERIC strategies represented. This suggests that almost all ERIC strategies can be applied to LMIC contexts. Yet, several strategies that seem critical to implementation in LMIC contexts—including capturing and sharing local knowledge, conducting local needs assessment, providing local technical assistance, and tailoring strategies—were not commonly cited in any of the included studies. This may be because these strategies were considered irrelevant or incompatible [35], redundant with other ERIC strategies [34], or part of routine processes for implementing health interventions [51,52,53] rather than discrete implementation strategies. Additionally, other ERIC strategies rarely, if ever, cited included those that required systems-level changes—such as change fees/incentives/billing, change policy, and enhancing local data systems and analysis—despite data suggesting organizational- and policy-level strategies are effective [54] and a critical component to closing the research-to-practice gap [55]. Finally, a number of strategies not used were those related to privatized health systems (e.g., alter consumer fees, change liability laws, use other payment schemes), which are less common in LMICs than centralized, public health systems, and thus may not be applicable or require adaptation in LMIC contexts. Further research is needed to explore why certain ERIC strategies have not been used in LMICs and if there are additional implementation strategies relevant to LMIC settings not currently included in the ERIC compilation.

Strengthening the evidence base for implementation strategies requires that their operationalization be reported in detail and that modifications to implementation strategies be systematically documented. While the included studies’ lack of ERIC strategy matching complicates the interpretation of specific ERIC strategy applicability to LMICs in this review, strategy matching itself may not be a requirement for understanding if and how certain implementation strategies are effective. However, implementation strategy components need to be specified in a way that allows their replication in research and practice and comparison across contexts [56, 57]. Among the studies reviewed here, all but eight included some type of strategy specification. However, most strategies only included specification of their action (75%), action target (61%), and actor (57%). Strategy details, such as temporality and dose, are necessary to replicate them in further testing. Moreover, mechanisms of action and fidelity, described for just 24% and 3% of strategies, respectively, are required for generating theory and selecting strategies that appropriately target contextual determinants [19]. Also concerning are how few strategies (18%) included a description of their adaptation process, which is likely necessary to meet the specific needs of the study context [2, 58,59,60]. Of the strategies that did include specification, most relied heavily on providers or research teams as actors and providers as action targets. This echoes our finding that few strategies addressing the organizational and policy levels were used and further highlights the dearth of research on implementation strategies that support population-level health improvements previously observed in high-income [61] and LMIC settings [39]. Moreover, the reliance on research team members as strategy actors threatens the generalizability of implementation strategies used in the studies to real-world settings.

Finally, though we had hoped to be able to assess ERIC strategy effectiveness in LMIC, just one strategy was evaluated in the studies included in our review. This is likely related to few studies defining strategy justification, mechanisms, and targeted implementation outcomes in their research. Instead, most studies looked at implementation outcomes without directly linking them to an identified determinant and a theory of change, precluding them from evaluating the individual strategy’s impact. Research teams may have lacked the resources to conduct a randomized control trial to rigorously test a strategy, as less than 20% of studies employed this design. However, while randomized control trials are the gold standard for research on intervention effectiveness, research on implementation strategies can successfully employ alternative designs, including interrupted time series, factorial, adaptive, and rollout designs [62, 63]. These designs can provide more flexibility and feasibility in resource-limited settings, while simultaneously maximizing external validity [64, 65].

The present study has several limitations. For one, we only focused on articles that used the ERIC compilation to inform study strategies and not more broadly defined implementation strategies. Therefore, our results may not be generalizable to all implementation strategy research in LMIC. We chose to focus on ERIC strategies as it is the taxonomy most commonly used in the field of implementation science, with over 3300 citations for the original [23] and refined [21] ERIC strategy papers and was developed using an expert review of existing strategy compilations and reviews. While we recognize that this biases toward research connected to non-LMIC academics, we note that around two-thirds of included studies were published in the final 2 years of our review period, suggesting that the use of ERIC is becoming more widespread in LMICs and highlighting the need to understand how to promote improved application and testing of these strategies. Second, we only included articles published in peer-reviewed journals. Owing to factors such as publication cost and language of these journals, this likely biased our findings to represent international and well-funded research. Further research is needed to explore non-ERIC strategy application in LMICs across languages and in the grey literature. Third, as we included study protocols in our sample, we cannot say with certainty that all strategies proposed will be applied in the research phase. However, the objective of this review was to provide a comprehensive description of the current state of this rapidly growing field (e.g., 35% of included papers published in the last year, 28% of studies still in the protocol phase), and inclusion of study protocols allows us to capture the most recent data. Finally, as many strategies employed were not matched to ERIC by the study authors, we were unable to draw conclusions about which may be the most relevant ERIC strategy(ies) in LMICs. However, those that did match indicated the individual ERIC strategies are applicable in LMICs.

Conclusions

This systematic review demonstrated the broad and growing use of the ERIC strategy taxonomy in LMICs, with inconsistency in application and very limited testing of ERIC strategy effectiveness. Moving forward, we provide the following recommendations to promote the development of implementation strategies to more rapidly close the research-to-practice gap in LMICs. First, research in LMICs must move beyond merely describing strategies to evaluating their effects on implementation outcomes. Moreover, strategies that are tested need to be better specified so that their effectiveness may be compared across studies and contexts and their mechanisms of change can be understood. Researchers should also consider reporting how the strategy would be deployed under routine, non-research-related conditions as well to promote application beyond the study period. Finally, strategies targeting policy, organizational, and community-level determinants should be explored to encourage change that supports scale-up and sustainability of individual, research-based implementation efforts in LMICs. To catalyze these lines of research, there is a need for greater capacity-building among researchers in LMICs to gain training in implementation research. Moreover, this training should directly involve and/or emphasize methods for the engagement of diverse local stakeholders, such as policymakers and community members, who may be better situated to develop and implement strategies at the systems level. Research funders, governments, and other implementers should consider encouraging work that includes each of these components.

Availability of data and materials

All articles included in this systematic review are publicly available. The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

LMICs:

Low- and middle-income countries

ERIC:

Expert Recommendations for Implementing Change

WHO:

World Health Organization

References

  1. Shelton RC, Lee M, Brotzman LE, Wolfenden L, Nathan N, Wainberg ML. What is dissemination and implementation science?: An introduction and opportunities to advance behavioral medicine and public health globally. Int J Behav Med. 2020;27(1):3–20.

    Article  PubMed  Google Scholar 

  2. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7:3.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8(1):139.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23.

    Article  PubMed  Google Scholar 

  5. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Ariyo P, Zayed B, Riese V, Anton B, Latif A, Kilpatrick C, et al. Implementation strategies to reduce surgical site infections: A systematic review. Infect Control Hosp Epidemiol. 2019;40(3):287–300.

    Article  PubMed  Google Scholar 

  7. Baker R, Camosso‐Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database of Systematic Reviews. 2015(4).

  8. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Jones LK, Tilberry S, Gregor C, Yaeger LH, Hu Y, Sturm AC, et al. Implementation strategies to improve statin utilization in individuals with hypercholesterolemia: a systematic review and meta-analysis. Implement Sci. 2021;16(1):40.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Mills KT, Obst KM, Shen W, Molina S, Zhang H-J, He H, et al. Comparative effectiveness of implementation strategies for blood pressure control in hypertensive patients: a systematic review and meta-analysis. Ann Intern Med. 2018;168(2):110–20.

    Article  PubMed  Google Scholar 

  11. Powell BJ, Proctor EK, Glass JE. A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions. Res Soc Work Pract. 2014;24(2):192–212.

    Article  PubMed  Google Scholar 

  12. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177–94.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Lewis CC, Boyd MR, Walsh-Bailey C, Lyon AR, Beidas R, Mittman B, et al. A systematic review of empirical studies examining mechanisms of implementation in health. Implement Sci. 2020;15(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–94.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Prior M, Guerin M, Grimmer-Somers K. The effectiveness of clinical guideline implementation strategies–a synthesis of systematic review findings. J Eval Clin Pract. 2008;14(5):888–97.

    Article  PubMed  Google Scholar 

  16. Varsi C, Solberg Nes L, Kristjansdottir OB, Kelders SM, Stenberg U, Zangi HA, et al. Implementation Strategies to Enhance the Implementation of eHealth Programs for Patients With Chronic Illnesses: Realist Systematic Review. J Med Internet Res. 2019;21(9):e14255.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Wensing M, Bosch M, Grol R. Selecting, tailoring, and implementing knowledge translation interventions. Knowl Transl Health Care. 2009;94:113.

    Google Scholar 

  18. Haley AD, Powell BJ, Walsh-Bailey C, Krancari M, Gruß I, Shea CM, et al. Strengthening methods for tracking adaptations and modifications to implementation strategies. BMC Med Res Methodol. 2021;21(1):1–12.

    Article  Google Scholar 

  19. Lewis CC, Klasnja P, Powell B, Tuzzio L, Jones S, Walsh-Bailey C, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6:136.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Wilson PM, Sales A, Wensing M, Aarons GA, Flottorp S, Glidewell L, et al. Enhancing the reporting of implementation research. BioMed Central; 2017; 1–5.

  21. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Slaughter SE, Zimmermann GL, Nuspl M, Hanson HM, Albrecht L, Esmail R, et al. Classification schemes for knowledge translation interventions: a practical resource for researchers. BMC Med Res Methodol. 2017;17(1):161.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57.

    Article  PubMed  Google Scholar 

  24. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10(1):1–8.

    Article  Google Scholar 

  25. Boyd MR, Powell BJ, Endicott D, Lewis CC. A method for tracking implementation strategies: an exemplar implementing measurement-based care in community behavioral health clinics. Behav Ther. 2018;49(4):525–37.

    Article  PubMed  Google Scholar 

  26. Bunger AC, Powell BJ, Robertson HA, MacDowell H, Birken SA, Shea C. Tracking implementation strategies: a description of a practical approach and early findings. Health Res Policy Syst. 2017;15(1):1–12.

    Article  Google Scholar 

  27. Huynh AK, Hamilton AB, Farmer MM, Bean-Mayberry B, Stirman SW, Moin T, et al. A pragmatic approach to guide implementation evaluation research: strategy mapping for complex interventions. Front Public Health. 2018;6:134.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, Cohen DJ. Specifying and comparing implementation strategies across seven large implementation interventions: a practical application of theory. Implement Sci. 2019;14(1):1–13.

    Article  Google Scholar 

  29. Yakovchenko V, Miech EJ, Chinman MJ, Chartier M, Gonzalez R, Kirchner JE, et al. Strategy Configurations Directly Linked to Higher Hepatitis C Virus Treatment Starts: An Applied Use of Configurational Comparative Methods. Medical Care. 2020;58(5).

  30. Yakovchenko V, Morgan TR, Chinman MJ, Powell BJ, Gonzalez R, Park A, et al. Mapping the road to elimination: a 5-year evaluation of implementation strategies associated with hepatitis C treatment in the veterans health administration. BMC Health Serv Res. 2021;21(1):1348.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Institute PCOR. Standards for Studies of Complex Interventions 2019 [Available from: https://www.pcori.org/research/about-our-research/research-methodology/pcori-methodology-standards.

  32. Patel V, Saxena S, Lund C, Thornicroft G, Baingana F, Bolton P, et al. The Lancet Commission on global mental health and sustainable development. Lancet. 2018;392(10157):1553–98.

    Article  PubMed  Google Scholar 

  33. Theobald S, Brandes N, Gyapong M, El-Saharty S, Proctor E, Diaz T, et al. Implementation research: new imperatives and opportunities in global health. Lancet. 2018;392(10160):2214–28.

    Article  PubMed  Google Scholar 

  34. Lovero KL, Dos Santos PF, Adam S, Bila C, Fernandes ME, Kann B, et al. Leveraging stakeholder engagement and virtual environments to develop a strategy for implementation of adolescent depression services integrated within primary care clinics of Mozambique. Frontiers Public Health. 2022;10.

  35. Means AR, Kemp CG, Gwayi-Chore M-C, Gimbel S, Soi C, Sherr K, et al. Evaluating and optimizing the consolidated framework for implementation research (CFIR) for use in low- and middle-income countries: a systematic review. Implement Sci. 2020;15(1):17.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Aldridge LR, Kemp CG, Bass JK, Danforth K, Kane JC, Hamdani SU, et al. Psychometric performance of the Mental Health Implementation Science Tools (mhIST) across six low- and middle-income countries. Implement Sci Commun. 2022;3(1):54.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Haroz EE, Bolton P, Nguyen AJ, Lee C, Bogdanov S, Bass J, et al. Measuring implementation in global mental health: validation of a pragmatic implementation science measure in eastern Ukraine using an experimental vignette design. BMC Health Serv Res. 2019;19(1):262.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Esponda GM, Hartman S, Qureshi O, Sadler E, Cohen A, Kakuma R. Barriers and facilitators of mental health programmes in primary care in low-income and middle-income countries. Lancet Psychiat. 2019.

  39. Wagenaar BH, Hammett WH, Jackson C, Atkins DL, Belus JM, Kemp CG. Implementation outcomes and strategies for depression interventions in low- and middle-income countries: a systematic review. Glob Ment Health (Cambridge, England). 2020;7:e7.

    Article  Google Scholar 

  40. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. 2021;10(1):1–11.

    Article  Google Scholar 

  41. The World Bank. World Bank Open Data Washington, DC: The World Bank; 2022 [Available from: https://data.worldbank.org/.

  42. Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia. Available at www.covidence.org.

  43. Michie S, Atkins L, West R. The behaviour change wheel. A guide to designing interventions 1st ed Great Britain: Silverback Publishing. 2014:1003–10.

  44. Geng EH, Baumann AA, Powell BJ. Mechanism mapping to advance research on implementation strategies. PLoS Med. 2022;19(2):e1003918.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. 2021;16(1):36.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Akiba CF, Powell BJ, Pence BW, Nguyen MXB, Golin C, Go V. The case for prioritizing implementation strategy fidelity measurement: benefits and challenges. Transl Behav Med. 2021;12(2):335–42.

    Article  PubMed Central  Google Scholar 

  47. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(1):37.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Merrill M. First principles of Instruction in CM Reigeluth & Carr (Eds.) Instructional design Theories and Models: Building a Common knowledge Base (Vol. III). New York: Routledge Publishers; 2009.

  49. Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6(1):42.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Gachau S, Ayieko P, Gathara D, Mwaniki P, Ogero M, Akech S, et al. Does audit and feedback improve the adoption of recommended practices? Evidence from a longitudinal observational study of an emerging clinical network in Kenya. BMJ Glob Health. 2017;2(4):e000468.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Fernandez ME, ten Hoor GA, van Lieshout S, Rodriguez SA, Beidas RS, Parcel G, et al. Implementation mapping: using intervention mapping to develop implementation strategies. Frontiers in Public Health. 2019;7(158).

  52. Group AMHR. Design, implementation, monitoring, and evaluation of mental health and psychosocial assistance programs for trauma survivors in low resource countries; a user’s manual for researchers and program implementers. United States: Johns Hopkins University Bloomberg School of Public Health; 2013.

    Google Scholar 

  53. Peters DH, Tran NT, Adam T. Implementation research in health: a practical guide: World Health Organization. 2013.

    Google Scholar 

  54. Sarkies MN, Bowles K-A, Skinner EH, Haas R, Lane H, Haines TP. The effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare: a systematic review. Implement Sci. 2017;12(1):132.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Purtle J, Nelson KL, Counts NZ, Yudell M. Population-based approaches to mental health: history, strategies, and evidence. Annu Rev Public Health. 2020;41:201–21.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Kirchner JE, Smith JL, Powell BJ, Waltz TJ, Proctor EK. Getting a clinical innovation into practice: an introduction to implementation strategies. Psychiatry Res. 2020;283:112467.

    Article  PubMed  Google Scholar 

  57. Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. BioMed Central; 2009. 1–6.

  58. Eisman AB, Hutton DW, Prosser LA, Smith SN, Kilbourne AM. Cost-effectiveness of the Adaptive Implementation of Effective Programs Trial (ADEPT): approaches to adopting implementation strategies. Implement Sci. 2020;15(1):109.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Geng EH, Mody A, Powell BJ. On-the-Go Adaptation of Implementation Approaches and Strategies in Health: Emerging Perspectives and Research Opportunities. Annu Rev Public Health. 2023;44(1):21–36.

    Article  PubMed  Google Scholar 

  60. Quanbeck A, Brown RT, Zgierska AE, Jacobson N, Robinson JM, Johnson RA, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for opioid prescribing in primary care. Implement Sci. 2018;13(1):1–13.

    Article  Google Scholar 

  61. Colquhoun HL, Squires JE, Kolehmainen N, Fraser C, Grimshaw JM. Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review. Implement Sci. 2017;12(1):30.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB, Jones L, et al. An overview of research and evaluation designs for dissemination and implementation. Annu Rev Public Health. 2017;38(1):1–22.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Handley MA, Lyles CR, McCulloch C, Cattamanchi A. Selecting and improving quasi-experimental designs in effectiveness and implementation research. Annu Rev Public Health. 2018;39(1):5–25.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Mazzucca S, Tabak RG, Pilar M, Ramsey AT, Baumann AA, Kryzer E, et al. Variation in research designs used to test the effectiveness of dissemination and implementation strategies: a review. Frontiers in Public Health. 2018;6.

  65. Wagenaar BH, Sherr K, Fernandes Q, Wagenaar AC. Using routine health information systems for well-designed health evaluations in low-and middle-income countries. Health Policy Plan. 2016;31(1):129–35.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

KLL, ACG, and MGC were supported by a grant from the National Institute of Mental Health (K01 MH120285, K23 MH128742, K01MH129572). BJP was supported in part through grants from the National Institute of Mental Health (R25MH080916, P50MH126219, R01MH124914), National Institute on Alcohol Abuse and Alcoholism (R01AA030480), National Institute on Drug Abuse (R01DA047876, P50DA054072), National Institute of Child Health and Human Development (R01HD103902), National Cancer Institute (P50CA19006, R01CA262325), National Heart, Lung, and Blood Institute (U24HL154426, R01HL157255), and the Agency for Healthcare Research and Quality (R13HS025632). Funders did not play any role in the study design, data collection, analysis, interpretation, or writing the manuscript.

Author information

Author notes

  1. Kathryn L. Lovero and Christopher G. Kemp contributed equally to this work.

Authors and Affiliations

  1. Department of Sociomedical Sciences, Columbia University Mailman School of Public Health, New York, NY, USA

    Kathryn L. Lovero

  2. Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Christopher G. Kemp

  3. Department of Global Health, University of Washington, Seattle, WA, USA

    Bradley H. Wagenaar

  4. Department of Epidemiology, University of Washington, Seattle, WA, USA

    Bradley H. Wagenaar

  5. Department of Psychiatry, Columbia University Irving Medical Center, New York State Psychiatric Institute, New York, NY, USA

    Ali Giusto

  6. Program On Forced Migration and Health, Heilbrunn Department of Population and Family Health, Columbia University Mailman School of Public Health, New York, NY, USA

    M. Claire Greene

  7. Brown School, Center for Mental Health Services Research, Washington University in St. Louis, St. Louis, MO, USA

    Byron J. Powell & Enola K. Proctor

  8. Center for Dissemination & Implementation, Institute for Public Health, Washington University in St. Louis, St. Louis, MO, USA

    Byron J. Powell & Enola K. Proctor

  9. Division of Infectious Diseases, John T. Milliken Department of Medicine, School of Medicine, Washington University in St. Louis, St. Louis, MO, USA

    Byron J. Powell

Authors
  1. Kathryn L. Lovero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher G. Kemp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bradley H. Wagenaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ali Giusto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. Claire Greene
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Byron J. Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Enola K. Proctor
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KLL, CGK, MCG, BHW, and AG conceived of the review. CGK and KLL contributed to the study design; assisted with article screening, data extraction, and interpretation of results; and drafted the manuscript. MCG, BHW, and AG contributed to the study design; assisted with article screening, data extraction, and interpretation of results; and provided feedback on manuscript drafts. BJP and EKP provided guidance on study design and interpretation of results as well as feedback on manuscript drafts. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kathryn L. Lovero.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

PRISMA 2020 checklist.

Additional file 2.

ERIC LMIC search protocol 2023.07.15R2.

Additional file 3.

Study descriptive information.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lovero, K.L., Kemp, C.G., Wagenaar, B.H. et al. Application of the Expert Recommendations for Implementing Change (ERIC) compilation of strategies to health intervention implementation in low- and middle-income countries: a systematic review. Implementation Sci 18, 56 (2023). https://doi.org/10.1186/s13012-023-01310-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-023-01310-2

Keywords

Implementation Science

ISSN: 1748-5908