Table 1 Summary of the Pragmatic Explanatory Continuum Indicator Summary (PRECIS) and the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework criteria
Framework | Domain | Criteria as described in published literature |
|---|---|---|
PRECIS [16] | Participant eligibility criteria | Extremely pragmatic studies would enroll all participants who have the condition of interest regardless of their responsiveness, past compliance, and co-morbidities. |
Extremely explanatory studies would use a variety of exclusion criteria to identify those individuals who would least likely respond to the intervention. | ||
Experimental intervention flexibility | Extremely pragmatic studies would leave the details of implementation of the experimental condition up to the practitioner. | |
Extremely explanatory studies would have strict instructions for every intervention element (e.g., timing of intervention delivery, tactics used, educational materials used). | ||
Experimental intervention practitioner expertise | Extremely pragmatic studies would be conducted by individuals from a variety of backgrounds and levels of experience and in a variety of settings. | |
Extremely explanatory studies would only be conducted by credentialed or seasoned interventionists. | ||
Comparison intervention | Extremely pragmatic studies would use ‘usual care’ or available alternative interventions. | |
Extremely explanatory studies would use a placebo condition as the comparison group. | ||
Comparison intervention practitioner expertise | Extremely pragmatic studies would have the comparison intervention delivered by individuals with full range of expertise levels, with only ordinary attention to training or experience. | |
Extremely explanatory studies would have the expertise of the comparison condition standardized at a high level so as to be able to detect whatever comparative benefits the experimental intervention might have. | ||
Follow-up intensity | Extremely pragmatic studies would not have formal follow-up visits. Instead administrative databases would be used to assess outcomes. | |
Extremely explanatory studies would have formal follow-up visits on a prescribed schedule and more extensive data collection than would occur in usual care or routine practice. | ||
Primary trial outcome | Extremely pragmatic studies would have a primary outcome that is objectively measured, has clinical significance, and is one that can be assessed under usual conditions. | |
Extremely explanatory studies would have a primary outcome that is known to be a direct and immediate relation to the intervention. The outcome may also require specialized training or testing not normally used to determine outcome status. | ||
Participant compliance with prescribed intervention | Extremely pragmatic studies would have no measurement of compliance and no special strategies to try to improve it. | |
Extremely explanatory studies would have study participants’ compliance monitored closely and would have strategies to enhance compliance. | ||
Practitioner adherence to study protocol | Extremely pragmatic studies would have no measurement of practitioner adherence and no special strategies to try to improve it. | |
Extremely explanatory studies would have close monitoring of how well the practitioners and study sites are adhering to the study protocol. | ||
Analysis of primary outcome | Extremely pragmatic studies would include all participants in the analyses regardless of compliance (e.g., intent-to-treat analysis). | |
Extremely explanatory studies would focus on analyses that allowed for estimating the maximum benefit of the intervention (e.g., analysis restricted to those considered ‘completers’). | ||
RE-AIM [23] | Reach | The absolute number, proportion, and representativeness of individuals who are willing to participate in a given initiative, intervention, or program. Reporting of exclusion criteria and percentage of potential participants excluded, Including the use of qualitative data to understand recruitment. |
Extremely pragmatic studies would include participants that are typical of those individuals with the specified condition (including hard-to-reach individuals). | ||
Extremely explanatory studies would include individuals who are not typical on most or all characteristics of those with the specified condition. | ||
Effectiveness | The impact of the intervention on outcomes. Measure of primary outcome, including potential negative effects, quality of life, and economic outcomes. Moderation analysis. Measure of short-term attrition and use of qualitative data to understand outcomes. | |
Extremely pragmatic studies would have primary outcomes that are meaningful to patients and providers. Explicit discussion of efforts to prevent harm to participants; report on unintended harmful or beneficial consequences of the intervention. | ||
Extremely explanatory studies would have primary outcomes that are meaningful to investigators. No efforts to capture unintended harmful or beneficial consequences of the intervention; only what is necessary for IRB requirements. | ||
Adoption | The absolute number, proportion, and representativeness of settings and the individuals within those settings who deliver the program who are willing to initiate a program. Reporting of setting and staff exclusion criteria and percentage excluded. Use of qualitative data to understand settling level adoption and staff participation. | |
Extremely pragmatic studies would have few or no setting exclusions. Sites are either randomly selected or purposely selected for variation. | ||
Extremely explanatory studies would have many exclusion criteria for settings and/or would try to get the ‘best’ sites to participate. | ||
Implementation | Fidelity to study/program protocol and adaptations made to intervention during study/program. Cost of intervention in terms of time and money. Consistency of implementation across staff, time, setting, and subgroups – focus is on process. Use of qualitative data to understand implementation. | |
Extremely pragmatic studies would provide detailed reporting of modifications made and rationale. Explicit discussion of efforts to contain costs and to make the intervention feasible for low resource settings would also be included. | ||
Extremely explanatory studies would have no mention of modifications to protocols or measures. No effort to contain costs; uses state of the art resources and procedures. | ||
Maintenance | The extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies. If and how the program/policy was adapted long-term. Some measure/discussion of alignment to organization mission or sustainability of business model. Use of qualitative data to understand setting level institutionalization. | |
Within the RE-AIM framework, maintenance also applies at the individual level. At the individual level, maintenance has been defined as the long-term effects of a program on outcomes 6 or more months after the most recent intervention contact. Measure of long-term attrition and differential rates by patient characteristics and/or treatment condition. Use of qualitative data to understand long-term effects. | ||
Extremely pragmatic studies would focus on explicit plans for handing off intervention to setting/site for continuation of the program/intervention after the completion of the study. | ||
Extremely explanatory studies would not have a report of efforts to continue the intervention after completion of the study. |