Where is “policy” in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example
Implementation Science volume 17, Article number: 80 (2022)
Implementation science aims to accelerate the public health impact of evidence-based interventions. However, implementation science has had too little focus on the role of health policy — and its inseparable politics, polity structures, and policymakers — in the implementation and sustainment of evidence-based healthcare. Policies can serve as determinants, implementation strategies, the evidence-based “thing” to be implemented, or another variable in the causal pathway to healthcare access, quality, and patient outcomes. Research describing the roles of policy in dissemination and implementation (D&I) efforts is needed to resolve persistent knowledge gaps about policymakers’ evidence use, how evidence-based policies are implemented and sustained, and methods to de-implement policies that are ineffective or cause harm. Few D&I theories, models, or frameworks (TMF) explicitly guide researchers in conceptualizing where, how, and when policy should be empirically investigated. We conducted and reflected on the results of a scoping review to identify gaps of existing Exploration, Preparation, Implementation, and Sustainment (EPIS) framework-guided policy D&I studies. We argue that rather than creating new TMF, researchers should optimize existing TMF to examine policy’s role in D&I. We describe six recommendations to help researchers optimize existing D&I TMF. Recommendations are applied to EPIS, as one example for advancing TMF for policy D&I.
(1) Specify dimensions of a policy’s function (policy goals, type, contexts, capital exchanged).
(2) Specify dimensions of a policy’s form (origin, structure, dynamism, outcomes).
(3) Identify and define the nonlinear phases of policy D&I across outer and inner contexts.
(4) Describe the temporal roles that stakeholders play in policy D&I over time.
(5) Consider policy-relevant outer and inner context adaptations.
(6) Identify and describe bridging factors necessary for policy D&I success.
Researchers should use TMF to meaningfully conceptualize policy’s role in D&I efforts to accelerate the public health impact of evidence-based policies or practices and de-implement ineffective and harmful policies. Applying these six recommendations to existing D&I TMF advances existing theoretical knowledge, especially EPIS application, rather than introducing new models. Using these recommendations will sensitize researchers to help them investigate the multifaceted roles policy can play within a causal pathway leading to D&I success.
Health policies, including the laws, regulations, and administration actions of governmental, public, and private organizations, play a critical role in influencing healthcare access , quality , and patient and public health outcomes , yet the role of policy in dissemination and implementation science (D&I) efforts is often understated or ignored [4, 5]. D&I research frequently conceptualizes health policy as a distal environmental factor within a broadly defined outer context rather than as central to the research and as a target of D&I strategies . Health policies can serve as implementation strategies that facilitate or mandate access to evidence-based health services [7,8,9]. Health policies can also represent the evidence-based law, rule, or “thing”  at the center of implementation efforts [5, 11,12,13,14], act as determinants that enable or constrain D&I strategies from achieving desired outcomes [5, 15,16,17], and potentially serve in other causal pathway roles (e.g., mechanisms).
Meaningful conceptualization of the multifaceted roles that policy plays in D&I research is critical to resolving persistent knowledge gaps about when, how, and why policymakers use evidence to inform policy, how evidence-based policies are implemented and sustained, and methods to de-implement policies that can harm individuals or society. Policy D&I can identify bidirectional roles of actors in the outer (e.g., policymakers) and inner context (e.g., healthcare delivery organizations) that can impact policy discourse and decision-making. Specifying health policy’s role in D&I research will improve empirical measurement of policy-related variables, can accelerate the impact of implementation, and lead to the development of strategies to de-implement outdated healthcare practices.
Argument for rethinking the role(s) of policy in D&I efforts
The fields of political science and public administration have repeatedly called on public health scientists to go beyond measuring the impact of a policy on health outcomes and intentionally examine how policy and politics influence the delivery of health services [18,19,20,21,22]. Doing so requires changes to how researchers conceptualize their work and the subsequent design and measurement choices that they make. Bernie and Clavier poignantly note that “health promotion research is inherently political” and caution researchers against conceptualizing a fictitious world where health interventions and scientific research are politically neutral topics . For example, recent policy changes in the USA demonstrate strong political influence, some to the detriment of public health and individual rights through restrictions in access to care [23,24,25]. Public health is also criticized for naïvely perceiving policymaking as a linear process with research serving as the strongest influence over decision-makers [20, 21, 26, 27]. Policy D&I research has the potential to address these criticisms by examining how complex, nonlinear policymaking and implementation processes are shaped by a plurality of interests including evidence, politics, personal and societal values, finances, and other factors of variable transparency [14, 28,29,30]. Early health policy implementation research focused solely on the mid-implementation process . Just as traditional D&I has increasingly focused on multiple phases of implementation , guidance is needed to support researchers in understanding health policy D&I decisions and activities across pre- and mid-implementation and sustainment phases. To address these critiques scientifically and practically, D&I research require theories, models, and frameworks (TMF) that are health policy conscious, that is, they meaningfully consider the dynamic nature of policy, polity structures, processes, political ideologies, and policymakers that shape implementation and sustainment.
D&I research emphasizes the importance of grounding studies in TMF to structure our understanding of how and why relationships between variables lead to certain outcomes . Delineating a TMF to inform study approach is critical to developing a successful D&I research proposal [34,35,36]. In 2012, Tabak et al.’s review identified nine policy-level D&I TMF . The online D&I models in health search now includes 26 TMF that address (albeit with varying degrees of depth) the socioecological level of policy in some way . However, many of these TMF are political science models without clear D&I processes described, content area specific [39,40,41], center on a population or setting , merely describe implementation phase activities , and focus on systems-level policy only , which limits their generalizability and utility for conceptualizing the multifaceted roles that policy can play. D&I TMF that are topic, setting, and population agnostic, useful across pre- and mid-implementation and sustainment phases, capable of addressing the determinants of, and policy implementation processes that occur across systems or within organizations are necessary to advance knowledge about the role health policy can play as the evidence-based “thing” to be implemented or within a causal pathway leading to successful dissemination or implementation .
We have two options to advance policy D&I research: (1) create new policy D&I-specific TMF that meet the above criteria or (2) adapt existing TMF to better conceptualize policy. We argue that a dearth of generalizable, policy-conscious TMF is not sufficient cause for developing new, untested TMF. Instead, researchers should optimize existing D&I TMF to more accurately capture policymaking and implementation processes over time and consider how policy (and its inseparable politics, polity structures, and policymakers) impacts the delivery of evidence-based practices (EBPs). In this article, we propose and describe practical recommendations to adapt the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework [32, 44] to meaningfully conceptualize policy, with the hope that researchers will apply these recommendations to enhance their empirical investigations with EPIS and other TMF.
Systematic scoping review of policy D&I studies that employed the EPIS framework
We conducted a systematic scoping review, guided by the PRISMA checklist  (Additional file 1) to identify examples of policy D&I research that used the EPIS framework, with the goal of generating recommendations for improved TMF optimization.
EPIS was chosen as the focus of this review for four reasons. First, EPIS is an influential TMF in the D&I field, having been cited in more than 1600 publications. Second, EPIS is one of few D&I TMF that directly investigates the temporal nature of D&I activities and determinants over time, suggesting that D&I stakeholders begin the exploration phase of any initiative by assessing the need for change and ways to accomplish change. The decision to adopt a new practice or policy propels them into preparation phase activities until active implementation of new or modified services commence. Stakeholders graduate to the sustainment phase when they can shift resources from implementation to maintenance activities. EPIS acknowledges that temporal phases may be recursive as priorities, resources, and policies shift over time . Third, EPIS is highly flexible and neutral in regard to topic, setting, population, and policy scope, making it a useful framework for investigating policy at different ecological levels and in diverse contexts [32, 44]. Fourth, EPIS includes domains (e.g., outer/inner context) and constructs found in other D&I TMF, which heightens the potential generalizability of recommendations presented below.
Most traditional political science conceptual models describe the temporal process of policymaking [47, 48], but do not provide insights on relationships between potential determinants and mechanisms across contexts. EPIS, like many D&I TMF (e.g., Consolidated Framework for Implementation Research; CFIR ), conceptualizes an outer context to describe a broad environment of influential factors, and an inner context comprised of local organizational factors that impact D&I efforts [32, 44]. These multi-level outer and inner contexts provide operational settings to investigate where a policy originates and exudes influence. In the 2011 EPIS introduction, Aarons et al. identified “sociopolitical/funding” as an outer context construct to acknowledge that legislative environments can influence stakeholders’ desire to explore potential changes and enhance or constrain available resources to adopt and sustain new practices . A later iteration of EPIS reconceptualized policy’s influence as “service environment/policies” and “funding/contracting” . The innovation factors domain is broadly defined, allowing for the study of one or more EBPs or policies [32, 44]. Bridging factors acknowledge the interrelated nature of outer and inner contexts and posit that specific structures, intermediaries, and activities (which can include policies and policy advocates) are needed to align contexts to support D&I success . The flexibilities afforded across EPIS domains provide space to conceptualize the dynamic and multiple roles policy can play. These reasons led us to conclude that conducting a scoping review of policy research guided by the EPIS would yield useful lessons about optimizing TMF.
Defining “policy” for the scoping review
Eligible studies employed EPIS to investigate the role of policy in D&I efforts. We defined “policy” and “policymaking” broadly using language from political science and public administration research. Policies are a series of interrelated decisions (e.g., legislation, rules) and purposive actions or inactions by decision-makers to execute agency goals . Studies could acknowledge policies of any scope including “big P” policies like federal, state, county, and city laws, regulations and administrative rules designed by government agencies, or “little p” policies including organizational rules and professional guidelines. Articles were also included if they mentioned the role of policy or the influence of policymakers and politics (e.g., describing the “sociopolitical environment”) in the policy environment. We focused on health policies but adopted a Health in All Policies approach to consider how any policy, regardless of its intended focus on health or any other issue, can be strategic tools that influence social determinants of health and have a direct impact on population health outcomes . Given the broad nature of scoping reviews, we did not exclude studies based on the topic (e.g., health, environment, taxes) of the EBPs/program/policy investigated, origin (outer or inner context), type (big P or little p), or influence of the policy addressed in a D&I effort.
Search strategy and results
We first reviewed results from a recent systematic review of EPIS  that identified 67 articles published between 2011 (when the original framework was published) and May 2017 that used EPIS to guide dissemination or implementation efforts . We obtained and reviewed the raw data from their systematic review to identify any articles that included policy considerations in their research. Simultaneously, we replicated Moullin et al.’s search criteria  (Additional file 2) in Web of Science, PsychINFO, and PubMed to identify more current relevant articles that were published between May 2017 and July 2022. Manual hand search methods were used to include relevant publications not yet indexed in databases.
Of the 1052 articles screened, 123 met criteria for full-text review. Most articles were excluded during the screening process because they cited the Aarons 2011 article but did not apply EPIS to their project (n = 719), or if they applied EPIS, they did not address policy in any way (n = 189). Articles were excluded after full-text review if they did not define any aspect of policy’s role in a D&I effort (n = 27). Ultimately, 96 articles were included in our qualitative synthesis (Additional file 3). Two researchers (ELC, RLH) performed screening and qualitative synthesis activities using a standardized scoping review template. We extracted data on policy characteristics (e.g., big P/little p), policy goals, the focal EBPs/policy, breadth and depth of EPIS use, and any policy-relevant adaptations to EPIS.
Recommendation development process
Five D&I scientists reviewed the extracted data and engaged in a consensus decision-making process to develop recommendations for optimizing TMF for policy D&I research. The authors have expertise in developing (GAA is one of the EPIS developers), advancing, and adapting TMF [(e.g., EPIS, CFIR, Practical, Robust, Implementation and Sustainability Model (PRISM)] to study D&I efforts and political science topics in US and global public sector health and allied health systems, community pharmacy, and criminal justice settings. Grounded in qualitative thematic analysis methods , we initially organized data from the scoping review to compare how prior research conceptualized the role of policy, policymakers, or politics as an outer or inner context variable, an innovation factor, bridging factor, or other variable. ELC identified similarities and differences in these conceptualizations and noted when policy’s role was ambiguously defined. ELC presented the qualitative synthesis and preliminary TMF recommendations to the coauthors. Coauthors reviewed the data and recommendations and then expanded on the suggestions, added additional ideas, and asked questions about the extracted data and role of policy. Revisions to recommendations were made over a 1-year period in a sequential process while maintaining a log of edits to capture the consensus decision-making process. Discrepancies in recommendations were resolved through team discussions. This process resulted in development of six recommendations for optimizing TMF for policy D&I research.
Recommendations for optimizing EPIS to investigate health policy D&I
We provide six recommendations to advance policy D&I research through EPIS optimization:
Specify dimensions of a policy’s function.
Specify dimensions of a policy’s form.
Identify and define the nonlinear phases of policy D&I.
Describe the temporal roles that stakeholders play in policy D&I over time.
Consider policy-relevant outer and inner context adaptations.
Identify and describe bridging factors necessary for policy D&I success.
Recommendations 1–2 optimize EPIS by defining key dimensions of a policy so that researchers can determine which domain/construct it should occupy and to understand where policy exists within a causal pathway. Recommendations 3–4 describe how researchers can use EPIS to conceptualize policy implementation activities over time and specify which policy-relevant stakeholders are represented in domains/constructs. Recommendations 5–6 acknowledge that existing domains/constructs may be underdeveloped for considering policy D&I factors and offer guidance for researchers to advance EPIS specification. Although recommendations are illustrated through EPIS application (Fig. 1) [32, 44], we provide examples of how they can be applied to other D&I TMF. We provide hypothetical research examples to illustrate the applicability of these recommendations to global settings, across different health topics, and roles of policy in D&I efforts.
Recommendation 1: Specify dimensions of a policy’s function
Few D&I studies specifically investigated policy as the evidence-based thing or as a strategy to be tested. Most alluded to policy as a factor in a vaguely described outer context but did not report on its purpose. Outer contexts were described generally as the “public and broader policy context,” “community,” and “outer system level of a broader environment.” Inner contexts were more clearly defined as specific state agencies, school districts, or healthcare provider organizations. Few articles defined the domain constructs (e.g., leadership, service environment agencies, funders, advocacy groups) responsible for creating and implementing policy or who might benefit from its passage.
The first recommendation is to assess the policy’s function describing the fundamental purpose of a policy [50, 54]. Function dimensions include the following: (1) policy goal(s), (2) policy type, (3) context, and (4) capital exchanged. Specifying these attributes will help researchers determine what role(s) a policy plays in D&I success and which domain/construct it occupies. Researchers should first ask, “what is the goal or intent of this policy?” This recommendation echoes early policy implementation research which argued that correctly identifying policy goals is critical to determining whether implementation was successful . Policies may aim to affect a broad or narrow scope of change or to formalize something that is already being done in practice. Policies with ambiguous goals may promote confusion around implementation activities and have little impact . Researchers should review legislative documents, government and organizational strategies, press releases and news articles, conduct legal mapping studies , or key informant interviews to specify policy goals. A single policy may have one or multiple goals; researchers should determine which goal(s) are critical to their D&I effort. Specifying the policy goal will help clarify if the policy is the evidence-based intervention, an implementation strategy to promote adoption of an EBP/program, a mechanism (series of events that promote the success of another implementation strategy), a precondition (i.e., factor necessary to activate the mechanism), determinant (i.e., barrier, facilitator), mediator (i.e., a variable that intervenes on the relationship between the implementation strategy and outcome), or moderator (i.e., a variable that alters the influence of another implementation strategy) . Lewis et al. (2018) provide a comprehensive description of these causal pathway terms, which can aid researchers in further identifying a policy’s goal . Specifying the policy goal can also reveal outcomes of interest (see “Recommendation 2: Specify dimensions of a policy’s form”) from the policy D&I effort — thereby advancing new policy-relevant implementation effects beyond traditional D&I outcomes (i.e., acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability) .
Researchers can then determine if the policy represents a “big P” or ‘little p’ policy type. Researchers can observe in-person or broadcasted hearings and/or document review of public policy records, policymaker meeting notes, white papers, and governmental strategies to help specify the functions of “big P” policies, although there may be many “behind the scenes” nuances to consider. Qualitative interviews with key informants may be needed to describe the function dimensions of “little p” policies if organizational documents (e.g., organizational strategy plans) are not publicly available. Researchers can investigate if/how “big P” policies turn into “little p” policies or vice versa over time.
Correctly identifying the policy goals and type will aid researchers in describing the outer and inner contexts where the policy originates and/or is implemented and potential implementation outcomes. The complexity of policymaking processes means that outer and inner contexts can be multi-level. The similarity in EPIS domain names makes this recommendation applicable to other TMFs including PRISM’s external/internal context  and CFIR’s outer/inner setting [49, 58]. Researchers should define all relevant policy contexts and levels to understand environmental factors that influence D&I processes. Finally, researchers need to identify the resources or capital exchanged (e.g., money, knowledge, data, training, political will) through the policy (then determine if those resources constitute a bridging factor, see “Recommendation 6: Identify and describe bridging factors necessary for policy D&I success”). Identifying the capital exchanged will help researchers understand why and when a policy is successfully implemented across multi-level contexts (i.e., “policy transfer”) .
Researchers need to specify a policy’s function to determine if their framework should include the policy of interest as an outer/inner context factor, bridging factor, implementation strategy, or as the innovation factor. This is critical because it will help guide researchers to hypothesize about potential contextual constructs and relationships that influence D&I processes and outcomes. An example scenario for applying Recommendation 1 is presented in Table 1.
Recommendation 2: Specify dimensions of a policy’s form
Few D&I studies have investigated policy as the evidence-based intervention to be implemented or as the implementation strategy. As a result, policy developers, their decision-making processes, and policy components are infrequently defined in D&I articles. To better conceptualize policy, researchers should clearly define the policy’s form: (1) its origin and creators, (2) structural components, (3) dynamism, and (4) (un)intended outcomes. Specifying a policy’s form will reveal the structures and processes that influenced how the policy was developed and can guide empirical research measuring how specific policy characteristics influence D&I outcomes . Knowledge about policy structure (i.e., what it specifically enforces) can help researchers investigate which role policy plays in a causal pathway for D&I efforts and where it should be placed in the TMF (e.g., outer/inner context, innovation factor).
Policy origin refers to how the policy was developed and the stakeholders involved in its creation. For example, was the policy developed by agency staff, an expert workgroup, via a collaborative process with the public or advocacy groups? Understanding the origin story creates transparency in the policymaking process  to reveal the nature of “evidence” (e.g., research vs. personal beliefs) used to inform decisions and the types of interests represented during policy development. Social network analysis can aid in identifying actors involved in the policy’s creation.
If policy is the evidence-based “thing” to be implemented, the EPIS innovation factors domain can be specified. In other TMFs, researchers can specify policy within the innovation [49, 58], evidence , or intervention domain . EPIS’ “innovation developers” construct can help define the policy’s origin. But policies might serve another role (e.g., as a determinant), and specifying where the policy developers reside (i.e., in outer or inner contexts and whether partisanship is part of the policies’ impetus) and their networks of influence can be useful to understanding which stakeholders need to be strategically engaged in the D&I effort or be the target of D&I strategies. For example, Purtle et al. identified US state legislators as a target group involved in policy decisions that impact children’s exposure to adverse childhood events (ACEs) . They found that democratic policymakers were more likely to engage with dissemination strategies that included projected lifetime costs to the public system associated with every nonfatal ACE case, while economic data did not alter republican’s engagement on this policy issue .
Specifying the policy structure requires asking whether the policy is enforceable or effective enough to impact implementation. Researchers should determine if the policy represents a funded or unfunded mandate, suggested guidelines, or some other structure that will impact the urgency and compliance of stakeholders. Document review of the policy itself should clarify structural components. Informational interviews with policy developers can also yield insights on policy structures.
Dynamism describes the policy’s intent and potential for permanence. Researchers should investigate if the policy has an expected lifetime (e.g., 5-year demonstration project). Time-limited policies may have temporary political/public support that diminishes over time, ultimately leading to the policy’s dissolution. For example, COVID-19 mask mandates were commonly implemented as time-limited policies that increasingly generated public backlash mounting political pressure on politicians and public health agencies to prevent mandate renewal . Policies without time limitations can face competing or supporting policies over time, political pressure, or advocacy from the outer and inner contexts that influence policy longevity. Researchers can investigate a policy’s dynamism by using legal mapping methods  or document review including white papers, government or organizational reports, legal, news, and social media sources. The prevalence of siloed health agencies [62, 63] suggests that competing or complementary health policy implementation efforts and political support exist, and qualitative interviews can help explain how these factors impact dynamism of the focal policy. Longitudinal media analyses and public opinion survey data can reveal how support for a policy changes over time and influences its permanence.
Identifying or measuring the intended and unintended outcomes of policy implementation represents the final form dimension. Policy outcome measurement can be the primary research aim or contribute to understanding the policy D&I process. For example, Crable et al. investigated implementation strategies used by Medicaid policymakers’ to encourage substance use treatment providers to adopt EBPs during each EPIS phase . Citing policy reach and fidelity outcomes from state evaluation projects helped contextualize the impact of implementation strategies used in Preparation and Implementation phases . Public testimony from constituents, advocacy groups, and lobbying firms can reveal potential unintended outcomes of policy implementation for researchers to investigate. Researchers should consider whether a policy is generating upstream and downstream outcomes and across which contexts. Upstream outcomes include the use of research evidence in policymaking and the overall fit of a policy with contextual factors. Downstream outcomes include how the evidence-based policy impacts quality, access, equity, and costs — which can be measured using large population health surveys or claims data. Qualitative descriptions and quantitative measures can be used to examine policy outcomes, and this methodological area is ripe for advancement [64, 65].
In EPIS, the innovation factors domain is commonly used to examine the developers, characteristics, and fit of an EBPs but can easily be adapted to investigate policy forms. Researchers should use “innovation developers” to describe the policy’s origin story, “innovation characteristics” to reveal its structure and its dynamism, while “innovation fit” describes the (un)intended consequences of a policy and its overall fit with contextual factors (Table 2). Policy forms can similarly be specified in RE-AIM/PRISM fit considerations regarding intervention/policy components or the overarching issues domain where policy representativeness, reasons, costs, benefits, and value can be defined . In CFIR, researchers can adapt the innovation domain to specify policy forms including its source (i.e., origin). Trialability, adaptability, and complexity can reveal the potential dynamism, and cost informs one outcome . Regardless of TMF used, researchers should specify if policy outcomes occur in outer and/or inner contexts.
Recommendation 3: Identify and define the nonlinear phases of policy D&I across contexts
Like policymaking, D&I processes are not linear [32, 59, 66, 67]. Our scoping review revealed few studies that examined D&I efforts across multiple EPIS phases. Most research focused on Implementation phase activities with little to no attention to how policy initially influenced or later modified implementation activities. Studying the nonlinear nature of policymaking and implementation processes is critical to understanding how and why evidence-based policies are adopted [21, 26].
Researchers should identify and define the nonlinear phases of policy D&I (Table 3). This process may require drawing different construct operationalizations within EPIS phases since contextual factors can yield different levels of influence and interaction over time. Researchers should identify the activities and stakeholders that characterize each D&I phase. Researchers can use EPIS phases or generic pre-, mid-, and post-implementation language to benchmark policy D&I activities. EPIS is particularly well-suited for achieving this recommendation given its temporal exploration, preparation, implementation, and sustainment phases and their dynamic relationship with other framework constructs. Researchers could integrate the use of group model building methods like causal loop diagrams to describe the role of policy over time, where reinforcing loops to indicate D&I momentum and balancing loops indicate stagnation . Causal loops might vary depending on the EPIS phase in which they are proposed to occur. Mixed methods can further illuminate the stories behind causal loop diagrams to reveal contextual factors that motivated each phase.
Recommendation 4: Describe the temporal roles that stakeholders play in policy D&I over time
Recommendation 3 highlights the need to understand how outer and inner contexts change over time, while Recommendation 4 advises researchers to specifically investigate how stakeholder roles and responsibilities in these contexts change over time. While some articles included in this review mentioned policy as a determinant of D&I efforts, they seldom described specific outer context “leadership” such as government officials charged with shaping or enforcing policy. “Interorganizational networks” of stakeholders were more frequently identified as having some distal influence over D&I processes, but their roles as implementation partners or intermediaries facilitating implementation efforts were not discussed. Several articles focused on the role that inner context “leadership” played in prioritizing and directing implementation efforts. Fewer articles addressed the role of stakeholders’ “individual characteristics” influencing implementation efforts.
Stakeholders involved in policy D&I efforts can enter, exit, and change positions over time. Researchers should document these positions, responsibilities, and movements in their framework to understand who is making decisions about policy development, dissemination, and implementation. Researchers can start by identifying the outer or inner context “leadership.” In addition, the role of outer context “interorganizational networks,” “advocacy groups,” “clients/patients,” and inner context frontline implementers as well as “intermediaries” who support the implementation of policy across contexts should also be considered. Some stakeholders will be involved throughout the entire policy lifetime (e.g., exploration, preparation, implementation, sustainment) or during time-limited phases where they make strategic contributions. Researchers should optimize their TMF to conceptualize the influence of all relevant stakeholders across outer and inner contexts and to determine if they serve in a bridging factor role, such as an “intermediary” aiming to align outer and inner contexts to promote policy implementation. EPIS includes multiple constructs describing stakeholders across domains, enabling researchers to capture how these roles change over time. If using other TMF, we recommend detailing individuals involved , specifically who is facilitating  policy D&I processes and the representativeness of stakeholders .
To conceptualize stakeholders’ roles over time, researchers can draw multiple time-bound versions of their EPIS framework. For example, researchers can specify stakeholder roles and responsibilities in outer and inner contexts, or as bridging factors during the exploration phase, and then re-specify those roles for the preparation phase to see which elements changed over time. Researchers can use multiple data collection methods to identify stakeholders including policy and meeting document review, social network analysis, ethnographic observation, stakeholder surveys, and qualitative interviews. Snowball sampling techniques  can reveal unexpected stakeholders across phases. An example scenario for applying Recommendation 4 is provided in Table 4.
Recommendation 5: Consider policy-relevant outer and inner context adaptions
TMF should guide the translation of research into policy and practice and elucidate and explain the relationship between contextual determinants, D&I strategies, and outcomes [33, 36]. Existing TMF present an incomplete organization of factors that impact policy D&I. Very few studies in the EPIS scoping review examined how specific policymakers (i.e., not just “leadership”), political institutions (i.e., polity structures), and politics played a role in D&I efforts. Most articles in the scoping review used a fraction of the EPIS constructs within outer and inner contexts, bridging factors, and innovation factors domains. Some articles did make adaptations to outer and inner contexts (Additional File 4). We argue that researchers should incorporate and define new policy-conscious constructs, as needed, to better understand the studied context or test new hypotheses about policy D&I processes, relationships, and causal pathways (Table 5). However, researchers should be careful not to include an unwieldly number of constructs that hinders meaningful investigation of the relationships between each.
Researchers should conduct literature reviews and speak with stakeholders in the study setting to identify relevant TMF adaptations that are necessary to conceptualize policy and guide empirical research. Potential adaptations to EPIS’ outer context include adding constructs like “political support” (to address partisanship), “societal stigma” (toward an issue or population targeted by the policy), “workforce capacity” (if implementing a policy that impacts provider responsibilities), and “news and social media attention” (which can sway societal and political support for a policy). Researchers should consider inner context adaptations which can include defining an organization’s “local service environment” (to describe how the existing service array might change due to policy D&I efforts). Adapting EPIS and other TMF to include relevant contextual influences helps to reveal new relationships between D&I strategies, mediating and moderating factors, and mechanisms that produce both desired and unintended outcomes .
Recommendation 6: Identify and describe bridging factors necessary for policy D&I success
Bridging factors represent structures, relationships, intermediaries, and processes that support outer-inner context alignment, policy transfer, and D&I success [44, 50, 69]. Like stakeholders, bridging factors may be omnipresent throughout all phases of dissemination or implementation or have a time-limited role [7, 50]. Although “bridging factors” language is specific to EPIS, these alignment enhancing factors can be conceptualized as domain-spanning linkages in other D&I TMF (e.g., boundary spanners that work across contexts to promote implementation outcomes). Recent research describes how contracts  and renegotiated reimbursement rates  between government agencies and clinical service providers are formal structures that function as bridging factors. Relational ties, like partnerships between government agencies and provider organizations, can also represent bridging factors. Stakeholders (e.g., lobbyists, consultants, advocates) who support the passage of a policy in the outer context and its implementation in the inner context serve in bridging factor roles [11, 50]. Researchers should investigate personal (e.g., financial) and professional (e.g., influence) gains individuals receive from serving as a bridging factor. Data and information sharing processes between outer and inner context entities (e.g., measurement-based care reporting) can also serve as bridging factors to promote cross-context alignment [69, 70] or policy transfer. Despite the important role bridging factors serve in achieving D&I success, their functions and forms are significantly understudied, and few studies in the scoping review enhanced our knowledge of their capacity to activate change.
Researchers should investigate and describe the presence or absence of necessary bridging factors for policy D&I success (Table 6). Such research would augment knowledge about how “big P” policies transfer from the outer to the inner context, how inner context “little p” policies are spread to the outer context, and how policies can diffuse across contextual levels [71,72,73]. Researchers can use qualitative methods to ask key informants about the nature and utility of structures, relationships, intermediaries, and processes supporting outer-inner context alignment and policy transfer processes. Snowball sampling techniques and social network analyses can help identify intermediaries and relational ties critical to policy implementation. Asking questions about how evidence is used to inform policymaking or how a policy is implemented can reveal when formal structures or processes serve as bridging factors.
TMF shape how researchers conceptualize studies, determine which variables will be measured, and where, when, and how EBPs and strategies are employed. Existing TMF do not sufficiently address health policy’s role in D&I, which limits advancement of this important field of research. Instead, policy-relevant constructs are frequently absent or treated as nuisance variables [5, 18, 19]. Table 7 summarizes six recommendations to help researchers improve how policy-relevant factors are conceptualized in EPIS so that empirical studies are better positioned to test and explain causal pathways that support the use of evidence in policymaking and the implementation of evidence-based policies. Advancing policy D&I research does not require “reinventing the wheel” with new TMF. Instead, researchers should apply these recommendations to EPIS and other TMF to define health policy’s role in D&I efforts and advance empirical policy D&I research. Enhanced specification of health policy’s role may support future work defining health policy D&I outcomes beyond those measured in traditional D&I efforts (i.e., acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability) . These TMF recommendations are not static but can serve as guidance for the growing body of policy-focused D&I. Future work should also investigate how these recommendations support calls for integrating D&I, public policy, and knowledge translation  to understand policy implementation processes outside of health and healthcare settings (e.g., criminal justice reform, housing and community development policies).
These recommendations are designed to build on each other, resulting in optimal specification of EPIS for policy D&I research. Researchers may be hesitant to consider every activity described within each recommendation or to apply all six recommendations in one study. Researchers should consider how each recommendation will impact the quality and scope of their study. Additionally, these recommendations advance a growing set of tools [36, 44, 50, 69] for researchers to test and advance EPIS in D&I efforts.
Availability of data and materials
Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.
Dissemination and implementation
Exploration, Preparation, Implementation, and Sustainment framework
Theories, models, or frameworks
World Health Organization
Aday LA. The impact of health policy on access to medical care. Milbank Mem Fund Q Health Soc. 1976;54(2):215–33.
Schweppenstedde D, Hinrichs S, Ogbu U, Schneider EC, Kringos DS, Klazinga NS, et al. Regulating quality and safety of health and social care: international experiences. Rand Heal Q. 2014;4(1):1.
Courtin E, Kim S, Song S, Yu W, Muennig P. Can social policies improve health? A systematic review and meta-analysis of 38 randomized trials. Milbank Q. 2020;98(2):297–371.
Nilsen P, Ståhl C, Roback K, Cairney P. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implement Sci. 2013;8(1):63.
Bullock HL, Lavis JN, Wilson MG, Mulvale G, Miatello A. Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis. Implement Sci. 2021;16(1):1–24.
Purtle J, Peters R, Brownson RC. A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014. Implement Sci. 2015;11(1):1.
Crable EL, Benintendi A, Jones DK, Walley AY, Hicks JM, Drainoni M. Translating Medicaid policy into practice: policy implementation strategies from three US states’ experiences enhancing substance use disorder treatment. Implement Sci. 2022;17(1):1–14.
Huguet N, Angier H, Marino M, McConnell KJ, Hoopes MJ, O’Malley JP, et al. Protocol for the analysis of a natural experiment on the impact of the Affordable Care Act on diabetes care in community health centers. Implement Sci. 2017;12(1):1–7.
Purtle J, Stadnick NA. Earmarked taxes as a policy strategy to increase funding for behavioral health services. Psychiatr Serv. 2020;71(1):100.
Curran GM. Implementation science made too simple: a teaching tool. Implement Sci Commun. 2020;1(1):27.
Crable EL, Jones DK, Walley AY, Hicks JM, Benintendi A, Drainoni M. How do Medicaid agencies improve substance use treatment benefits? Lessons from three states’ 1115 waiver experiences. J Health Polit Policy Law. 2022;47(4):497–518.
Tso P, Culyer AJ, Brouwers M, Dobrow MJ. Developing a decision aid to guide public sector health policy decisions: a study protocol. Implement Sci. 2011;6(1):1–5.
Nelson KL, Purtle J. Factors associated with state legislators’ support for opioid use disorder parity laws. Int J Drug Policy. 2020;82:102792.
Purtle J, Nelson KL, Horwitz SMC, McKay MM, Hoagwood KE. Determinants of using children’s mental health research in policymaking: variation by type of research use and phase of policy process. Implement Sci. 2021;16(1):13.
Lobczowska K, Banik A, Brukalo K, Forberger S, Kubiak T, Romaniuk P, et al. Meta-review of implementation determinants for policies promoting healthy diet and physically active lifestyle: application of the Consolidated Framework for Implementation Research. Implement Sci. 2022;17(1):1–16.
Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8(1):1–11.
Lui JHLL, Brookman-Frazee L, Lind T, Le K, Roesch S, Aarons GA, et al. Outer-context determinants in the sustainment phase of a reimbursement-driven implementation of evidence-based practices in children’s mental health services. Implement Sci. 2021;16(1):1–9.
Navarro V. Politics and health: a neglected area of research. Eur J Public Health. 2008;18(4):354–5.
Judge K. Politics and health: policy design and implementation are even more neglected than political values? Eur J Public Health. 2008;18(4):355–6.
Fafard P, Cassola A. Public health and political science: challenges and opportunities for a productive partnership. Public Health. 2020;1(186):107–9.
Bernier NF, Clavier C. Public health policy research: making the case for a political science approach. Health Promot Int. 2011;26(1):109–16.
Smith K. Beyond evidence based policy in public health: the interplay of ideas. London: Palgrave Macmillan; 2013.
Gordon MR, Coverdale J, Chervenak FA, McCullough LB. Undue burdens created by the Texas abortion law for vulnerable pregnant women. Am J Obstet Gynecol. 2022;226(4):529–34.
Tanne JH. Texas’s new abortion law is an attack on medical practice and women’s rights, say doctors. BMJ. 2021;374:n2176.
Grossman D, Baum S, Fuentes L, White K, Hopkins K, Stevenson A, et al. Change in abortion services after implementation of a restrictive law in Texas. Contraception. 2014;90(5):496–501.
Gagnon F, Bergeron P, Clavier C, Fafard P, Martin E, Blouin C. Why and how political science can contribute to public health? Proposals for collaborative research avenues. Int J Heal Policy Manag. 2017;6(9):495.
Oliver K, Wellstead A, Cairney P. Policy advice: irked by naivety about policymaking. Nature. 2015;527(7577):165.
Sheingold S, Zuckerman RB, De Lew N, Maddox KEJ, Epstein AM. Should Medicare’s value-based pricing be adjusted for social risk factors? The role of research evidence in policy deliberations. J Health Polit Policy Law. 2018;43(3):401–25.
Innvær S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002;7(4):239–44.
Purtle J, Lê-Scherban F, Nelson KL, Shattuck PT, Proctor EK, Brownson RC. State mental health agency officials’ preferences for and sources of behavioral health research. Psychol Serv. 2019;17(S1):93–7.
Matland RE. Synthesizing the implementation literature: the ambiguity-conflict model of policy implementation. J Public Adm Res Theory. 1995;5(2):145–74.
Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Heal Ment Heal Serv Res. 2011;38(1):4–23.
Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):1–13.
Crable EL, Biancarelli D, Walkey AJ, Allen CG, Proctor EK, Drainoni ML. Standardizing an approach to the evaluation of implementation science proposals. Implement Sci. 2018;13(1):71.
Proctor EK, Powell BJ, Baumann AA, Hamilton AM, Santens RL. Writing implementation research grant proposals: ten key ingredients. Implement Sci. 2012;7(1):96.
Moullin JC, Dickson KS, Stadnick NA, Albers B, Nilsen P, Broder-Fingert S, et al. Ten recommendations for using implementation frameworks in research and practice. Implement Sci Commun. 2020;1(1):42.
Tabak RG, Padek MM, Kerner JF, Stange KC, Proctor EK, Dobbins MJ, et al. Dissemination and implementation science training needs: insights from practitioners and researchers. Am J Prev Med. 2017;52(3):S322–9.
Dissemination and implementation models in health: an interactive webtool to help you use D&I models. https://dissemination-implementation.org/tool/. Accessed 25 Aug 2022.
Prihodova L, Guerin S, Kernohan WG. Knowledge transfer and exchange frameworks in health and their applicability to palliative care: scoping review protocol. J Adv Nurs. 2015;71(7):1717–25.
Owen N, Glanz K, Sallis JF, Kelder SH. Evidence-based approaches to dissemination and diffusion of physical activity interventions. Am J Prev Med. 2006;31(4):35–44.
Bauman AE, Nelson DE, Pratt M, Matsudo V, Schoeppe S. Dissemination of physical activity evidence, programs, policies, and surveillance in the international public health arena. Am J Prev Med. 2006;31(4):57–65.
Edgar L, Herbert R, Lambert S, MacDonald JA, Dubois S, Latimer M. The joint venture model of knowledge utilization: a guide for change in nursing. Nurs Leadersh. 2006;19(2):41–55.
Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Heal. 2018;6:136.
Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1.
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann of Intern Med. 2018;169:467–73.
Becan JE, Bartkowski JP, Knight DK, Wiley TRAA, DiClemente R, Ducharme L, et al. A model for rigorously applying the Exploration, Preparation, Implementation, Sustainment (EPIS) framework in the design and measurement of a large scale collaborative multi-site study. Heal Justice. 2018;6(1):9.
Kingdon JW. Agendas, alternatives, and public policies. Boston: Little, Brown & Co.; 1984.
Redman S, Turner T, Davies H, Williamson A, Haynes A, Brennan S, et al. The SPIRIT Action Framework: a structured approach to selecting and testing strategies to increase the use of research in policy. Soc Sci Med. 2015;136–137:147–55.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.
Lengnick-Hall R, Stadnick NA, Dickson KS, Moullin JC, Aarons GA. Forms and functions of bridging factors: specifying the dynamic links between outer and inner contexts during implementation and sustainment. Implement Sci. 2021;16(1):34.
Jenkins WI. Policy analysis: a political and organisational perspective. London: Martin Robertson; 1978.
World Health Organization. Health in all policies: Helsinki statement. In: Framework for country action. Geneva: WHO Press; 2014.
Green J, Thorogood N. In: Seaman J, editor. Qualitative methods for health research. 3rd ed. London, UK: SAGE Publications Inc.; 2014.
Perez Jolles M, Lengnick-Hall R, Mittman BS. Core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med. 2019;34(6):1032–8.
Ramanathan T, Hulkower R, Holbrook J, Penn M. Legal epidemiology: the science of law. J Law Med Ethics. 2017;45(Supp 1):69–72.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.
Glassgow R, Harden S, Gaglio B, Rabin B, Smith M, Porter G, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Heal. 2019;7:64.
Damschroder L, Reardon C, Opra Widerquist M, Lowery J. The updated consolidated framework for implementation research based on user feedback. Implement Sci. 2022;17:75.
Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11:35.
Purtle J, Nelson KL, Gebrekristos L, Lê-Scherban F, Gollust SE. Partisan differences in the effects of economic evidence and local data on legislator engagement with dissemination materials about behavioral health: a dissemination trial. Implement Sci. 2022;17:38.
Scoville C, McCumber A, Amironesei R, Jeon J. Mask refusal backlash: the politicization of face masks in the American public sphere during the early stages of the COVID-19 pandemic. Socius. 2022;8:1–22.
Grogan CM. The murky relationship between ideology and the role of government in health policy. J Heal Polit Policy Law. 2012;37(3):361–4.
Burris S, Hitchcock L, Ibrahim J, Penn M, Ramanathan T. Policy surveillance: a vital public health practice comes of age. J Health Polit Policy Law. 2016;41(6):1151–73.
Allen P, Pilar M, Walsh-Bailey C, Hooley C, Mazzucca S, Lewis CC, et al. Quantitative measures of health policy implementation determinants and outcomes: a systematic review. Implement Sci. 2020;15(1):47.
Brownson RC, Kumanyika SK, Kreuter MW, Haire-Joshu D. Implementation science should give higher priority to health equity. Implement Sci. 2021;16(1):1–16.
Simpson KM, Porter K, McConnell ES, Colón-Emeric C, Daily KA, Stalzer A, et al. Tool for evaluating research implementation challenges: a sense-making protocol for addressing implementation challenges in complex research settings. Implement Sci. 2013;8(1):1–12.
Rycroft-Malone J, Burton C. Paying attention to complexity in implementation research. Worldviews Evid Based Nurs. 2010;7(3):121–2.
Vennix J. Group model-building: tackling messy problems. Syst Dyn Rev. 2000;15(4):379.
Lengnick-Hall R, Willging C, Hurlburt M, Fenwick K, Aarons GA, Aarons GA. Contracting as a bridging factor linking outer and inner contexts during EBP implementation and sustainment: a prospective study across multiple US public sector service systems. Implement Sci. 2020;15(1):43.
Economou MA, Kaiser BN, Yoeun SW, Crable EL, McMenamin SB. Applying the EPIS framework to policy-level considerations: tobacco cessation policy implementation among California Medicaid managed care plans. Implement Res Pract. 2022;3. https://doi.org/10.1177/26334895221096289.
Evans M, Davies J. Understanding policy transfer: a multi-level, multi-disciplinary perspective. Public Adm. 1999;77(2):361–85.
Lavis JN, Røttingen JA, Bosch-Capblanch X, Atun R, El-Jardali F, Gilson L, et al. Guidance for evidence-informed policies about health systems: linking guidance development to policy development. PLoS Med. 2012. https://doi.org/10.1371/journal.pmed.1001186.
Dolowitz DP, Marsh D. Learning from abroad: the role of policy transfer in contemporary policy-making. Governance. 2000;13(1):5–23.
Drs. Crable and Lengnick-Hall are fellows, Dr. Stadnick is an alumnus, and Dr. Aarons is core faculty with the Implementation Research Institute (IRI, at the George Warren Brown School of Social Work, Washington University in St. Louis, through an award from the National Institute of Mental Health (R25 MH080916). Dr. Crable is also funded by NIDA 1K01DA056838. Dr. Lengnick-Hall is also funded by NIMH P50MH113662. Dr. Moullin is partially funded by a Medical Research Future Fund grant (GNT1168155). Drs. Aarons and Crable are supported by NIDA grant R01DA049891, and Drs. Aarons and Stadnick are supported by NIMH P50MH126231 Center for Team Effectiveness to Accelerate EBP Implementation in Children’s Mental Health Services.
Ethics approval and consent to participate
Consent for publication
GAA is a co-editor-in-chief, and JCM is on the editorial board of Implementation Science. All decisions on this paper were made by another editor. The other authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
PRISMA Checklist Applied to Scoping Review. Word document displaying how the PRISM checklist was applied to this narrative review.
Search String for the Systematic Scoping Review. Word document listing the search terms used for the systematic scoping review.
Flow Diagram of Search Strategy and Article Selection for the Scoping Review. Word document displaying a flow diagram summarizing the systematic scoping review search results.
Identified Additions to Outer and Inner Context Domains in the Exploration, Preparation, Implementation, and Sustainment Framework from the Scoping Review. Word document displaying a table of adaptations applied the outer and inner contexts of the EPIS framework.
About this article
Cite this article
Crable, E.L., Lengnick-Hall, R., Stadnick, N.A. et al. Where is “policy” in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example. Implementation Sci 17, 80 (2022). https://doi.org/10.1186/s13012-022-01256-x