Based on stakeholder feedback, the FRAME-IS is modular and includes both core (i.e., required) and optional modules to maximize its practicality across implementation projects with a variety of goals, priorities, and available resources. Core modules can be found in Fig. 1, and optional modules can be found in Fig. 2. The decision regarding which modules to designate as core versus optional was made by the authors based on consensus discussions, incorporating stakeholder feedback. Additional descriptive text can be found in the supplement. Completing core modules requires specifying: a brief description of the EBP, implementation strategy, and modifications (Module 1); what is modified (Module 2); the nature of the modification (content, evaluation, or training modifications only; Module 3); and the rationale for the modification (Module 4). Optional modules, which can be completed at the discretion of the study investigators or project team, involve specifying when the modification occurred, and whether it was planned (Module 5); who participated in the decision to modify (Module 6); and how widespread the modification is (Module 7). Module 3, while itself considered a core module, includes the option of documenting the extent to which the modification was considered fidelity-consistent with the original implementation strategy.
Module 1: Brief description of the EBP, implementation strategy, and modification(s)
To facilitate tracking modifications and to complete the remainder of the FRAME-IS, we recommend briefly describing the EBP in question, and the initially defined implementation strategy. The Expert Recommendations for Implementing Change (ERIC) compilation [11] may be useful in describing the implementation strategy. If it is feasible, more comprehensive guidance [16] may be used to describe the implementation strategy in more detail to delineate the core elements, processes, or functions of the strategy. We also suggest identifying potential initial modifications to the strategy. We note that many modifications may be “bundled”—i.e., may involve changes to multiple aspects of the implementation strategy. For example, the content and the length of a provider training may be modified simultaneously. In those cases, the research or implementation team can determine whether to complete the FRAME-IS separately for each modification or to document all of the separate modifications at once. Completing it separately for each individual modification may allow for finer-grained analysis of what was most helpful—but may also represent an undue documentation burden in cases where separate components of a modification cannot be disentangled. Similarly, depending on the goal of the project, users should determine whether it is best to use the FRAME-IS separately for each component of a multifaceted implementation strategy or to define the entire strategy and modification in Module 1. For example, Leadership and Organizational Change for Implementation (LOCI [36]) is a multi-component implementation strategy that includes coaching calls. If the modification in question involved changing the frequency of LOCI coaching calls, the research team would need to decide whether the change in coaching call frequency should be documented alongside modifications that may be made simultaneously to other LOCI components.
Module 2: What is modified?
The FRAME-IS includes four broad categories of modifications to implementation strategies: Content, Context, Evaluation, and Training. These four categories mirror those included in the FRAME, but with some key distinctions. A Context modification refers to changes to the setting or the way the implementation strategy is delivered. For example, if the implementation strategy being modified was implementation facilitation, the context could change if facilitation was provided virtually as opposed to in person. We note that there are some distinctions in Content and Training modifications to implementation strategies. For example, if the implementation strategy in question is a clinician training workshop, then modifications to the content of the workshop itself would qualify as a Content modification (because it is part of the package of implementation strategies). In contrast, changes to how implementers are trained—e.g., by modifying the ways that external facilitators are trained in the context of implementation facilitation—would qualify as a Training modification. Modifications to evaluation refer to changes in the way that an implementation strategy is evaluated. For example, recent work has aimed to uncover the “core activities” of implementation facilitation [37]; ongoing studies are incorporating assessments of these core activities into their implementation evaluation plans (e.g., [38]).
Module 3: What is the nature of the content, evaluation, or training modification?
For Content, Evaluation, and Training modifications, it is important to track the nature of the modification itself. These modifications can range from relatively small tweaks or more sweeping changes to the implementation strategy—or even abandonment of an implementation strategy or component altogether. The elements of the FRAME-IS in this domain closely mirror those of the FRAME, but with language specifying modifications made to an implementation strategy rather than an EBP. Adding or removing/skipping elements refer to specific aspects of a discrete implementation strategy (e.g., removing role plays from a training workshop) or package of strategies (e.g., removing feedback, adding incentives in a leadership program). Pacing may refer to pacing of training or frequency of feedback. Integration refers to incorporation of other implementation strategies or approaches (e.g., adding feedback or incentives to training and supervision when the first round of implementation suggests that results are not as robust as desired). Loosening structure might include coaching “on the fly” or immediately after a clinical interaction rather than at scheduled times. Substituting may include adaptations such as changing fidelity assessment from observer-rated to self-report.
As an optional portion of this module, practitioners or researchers may also be interested in documenting the extent to which the modification was conducted with fidelity [39]. In this context, fidelity may be defined as “the adherence to the intervention components, competence with which the intervention is delivered, and differentiation from other treatments” [1]. Fidelity-consistent modifications may be expected to have better outcomes than fidelity-inconsistent ones, which may represent “drift” as they remove core elements of the strategy. Core elements or functions, in this context, refer to components or topics considered essential to that implementation strategy [39]. For example, removing the feedback component from an audit and feedback implementation strategy would likely be fidelity-inconsistent. We acknowledge, though, that the relationships among fidelity, adaptation, and outcomes are complex and not fully understood [40]. Fundamental questions regarding balancing adaptation and fidelity have yet to be answered—especially for the substantial proportion of implementation strategies that are meant to be inherently adaptable, flexible, or modular (e.g., [13]). Thus, completion of this portion of the module may not be warranted in all cases.
Module 4: What is the rationale for the modification?
The goal of this module is to document why a given modification was made to a given implementation strategy. This may allow interested parties to determine what rationales are associated with more or less successful modifications. We break the rationale into two components. First, regarding the goal of the modification, we have derived answer options in part from the Reach – Effectiveness – Adoption – Implementation – Maintenance ((RE-AIM) Framework [35] and Proctor’s implementation outcomes [15], plus one option related to health equity. We note that some goals may be related to the implementation effort itself, while others may be more directly related to the EBP being implemented. We also note that many modifications may aim to achieve multiple goals. For example, shortening an EBP training may simultaneously reduce costs, increase adoption, and increase reach. In completing this Module, we recommend selecting the box corresponding to the primary goal of the modification or selecting multiple boxes in the context of several co-equal goals.
Second, we recommend documenting the level of the organization that most directly informed the modification. For example, modifications made to accommodate available staffing at a clinic would qualify as the organizational level, while modifications made to fit with the professional or cultural values of frontline staff delivering the EBP would qualify as the practitioner level.
Module 5: When the modification is initiated, and whether it is planned
The timing of modifications is crucial in implementation science, with the pre-implementation, implementation, and sustainment phases featuring distinct pressures, challenges, and goals [37, 41, 42]. Modifications made early in the implementation process may leave more time for implementers and practitioners to adjust to the change. Modifications made later (during the implementation or sustainment phases) may nonetheless be required to accommodate shifting priorities or resources (e.g., shifting initial training to a web-based format based on travel restrictions). Documenting the timing of modifications to implementation strategies will allow the field to develop a better understanding of how such timing affects implementation processes and outcomes.
Note that for our purposes here the primary goal is to document when a modification is initiated, rather than when it occurs, as many modifications to implementation strategies may happen over large portions of the implementation period. For example, consider an implementation strategy of provider training to increase uptake of an evidence-based psychotherapy. If the timing of those trainings is modified, the important question for this section is when the decision was made to change the timing rather than when the (modified) trainings were offered. Documenting when decisions regarding modifications are made may ultimately help shed light on whether modifications made early versus late in the implementation period are differentially successful.
Regardless of the phase in which modifications to implementation strategies are made, there is a conceptual distinction between those that are made in a planful versus reactive manner [43, 44]. For example, consider a hypothetical implementation project featuring a learning collaborative that was originally designed to meet face-to-face four times during the implementation year. Let us further imagine that, due to budget constraints, only two learning collaborative meetings could be funded. If the budget constraints were known early in the implementation year, then the implementer and/or content experts would likely be able to modify the curriculum or format to ensure that all core content of the learning collaborative is covered (i.e., an adaptation or planned modification). Such proactive changes could include, for example, establishing or expanding a virtual component for the learning collaborative, or lengthening the two face-to-face sessions that could be funded. In contrast, having two face-to-face learning collaborative meetings abruptly canceled midway through the implementation year (i.e., a reactive modification) might necessitate more substantial changes to the curriculum that leaves some core content unaddressed. Differentiating unplanned, reactive modifications from proactive and planful adaptations will allow implementation scientists to better understand the circumstances under which impromptu modifications to implementation strategies may be more or less helpful.
Module 6: Who participates in the decision to modify?
In some cases, modifications to implementation strategies may be made in a collaborative manner, with multiple stakeholders or “actors” (e.g., administrators, frontline clinicians, implementation specialists) agreeing that a given modification is needed in a given setting [16]. In other cases, the decision to modify an implementation strategy may be unilateral (as when a health system leader requires that a given implementation strategy be scaled back based on personnel changes or competing priorities). Documenting this distinction can inform future decisions regarding when broader consensus on certain types of modifications to implementation strategies is required for implementation success, consistent with the principles of stakeholder engagement [45]. As researchers start to grapple with the intersection of implementation science and health equity [46], carefully identifying who is suggesting the modifications may be an important aspect of tracking the co-creation of implementation strategies [21,22,23,24]. Identifying sources of power in the implementation process [47], and incorporating the voices of those in the community, will be important for the field as we move toward equitable practice in our science.
Module 7: How widespread is the modification?
For modifications documented in Module 2 as Content, Training, and Evaluation modifications, it may be important to document the breadth or scope of the modification to the implementation strategy. This can range from relatively narrow modifications (e.g., in the context of an individual consultation call for a clinician who missed a day of group consultation) to broad-based ones (e.g., modifications made by an entire health system that is using an implementation strategy to roll out an EBP).
Note that some of the answer options refer to the individuals receiving the EBP, while others refer to the practitioners delivering the EBP, and yet others refer to those tasked with supporting the use of the EBP. It is possible that boxes within all three of these categories could be checked. If a single implementation facilitator adds an audit and feedback component to an implementation facilitation strategy within one clinic (and no other facilitators are using the unmodified strategy within the clinics they oversee), then that would qualify as a modification at one clinic/unit and one specific implementer/facilitator.