Skip to main content
Download PDF
Download ePub

Designing provider-focused implementation trials with purpose and intent: introducing the PRECIS-2-PS tool

Implementation Science volume 16, Article number: 7 (2021) Cite this article

Abstract

Background

First articulated by Schwartz and Lellouch (1967), randomized controlled trials (RCTs) can be conceptualized along a continuum from more explanatory to more pragmatic. The purpose and intent of the former is to test interventions under ideal contexts, and the purpose and intent of the latter is to test interventions in real-world contexts. The PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) is a validated tool that helps researchers make decisions about the elements of the trial to match the overall purpose and intent of the trial along the continuum. The PRECIS-2 tool has guided the design of hundreds of RCTs. However, a few aspects of the tool would benefit from greater clarity, including its application to provider-focused implementation trials rather than patient-focused intervention trials.

Main text

We describe the newly developed PRECIS-2-Provider Strategies (PRECIS-2-PS) tool, an extension of the PRECIS-2 tool, which has been adapted for trials testing provider-focused strategies. We elaborate on nine domains that can make a provider-focused trial more explanatory or more pragmatic, including eligibility, recruitment, setting, implementation resources, flexibility of provider strategies, flexibility of intervention, data collection, primary outcome, and primary analysis. We detail the complementary roles that researchers and stakeholders play in the trial design phase, with implications for generalizability of trial results to the contexts in which they are intended to be applied.

Conclusions

The PRECIS-2-PS tool is designed to help research and practice teams plan for provider-focused trials that reflect the overall intent and purpose of the trial. The tool has potential to help advance the science of provider-focused strategies across a range of trials, with the ultimate goal of facilitating the adoption, integration, and sustainability of provider-focused strategies outside the context of trials.

Peer Review reports

Background

Randomized controlled trials (RCTs), including individual-, cluster-, and stepped-wedge, are considered the “gold standard” study design for identifying effective health-related innovations [1]. Compared to other types of study designs (e.g., quasi-experimental), RCTs are best equipped to minimize the multitude of threats to internal validity that can compromise the integrity and interpretation of trial results [2]. However, RCTs are often criticized for having poor external validity—that is, trial results do not accurately reflect the circumstances to which the results should or could be applied once the study is completed. The potential for trials to impact patient outcomes, care delivery, and population health is limited if trial results are neither generalizable nor applicable to the contexts in which they are intended to apply.

In recognition of this criticism, methodologists have considered ways in which RCTs can be designed to better balance both internal and external validity. First introduced by Schwartz and Lellouch (1967), one way of conceptualizing trials with more or less emphasis on external validity is along a multi-axial continuum from more explanatory to more pragmatic [3]. On one end of the continuum, explanatory trials are those that emphasize internal validity. Explanatory trials seek to understand if an intervention is effective under ideal contexts, which are often characterized as highly resourced, tightly controlled, and conducted in somewhat artificial settings. These trials are mainly concerned with understanding and testing hypotheses on the existence of particular mechanisms of action for a given innovation. On the other end of the continuum are pragmatic trials. These trials emphasize a balance between internal and external validity. Pragmatic trials seek to understand if innovations work in real-world contexts that closely reflect the settings in which the innovation is intended to be used [4].

Interest in pragmatic RCTs (pRCTs) has increased substantially in recent years as researchers, practitioners, and funders continue to recognize the need for innovations that are both effective and generalizable beyond a single trial. Many opportunities now exist to support the conduct of pRCTs, including educational courses (e.g., massive open online course, Pragmatic Randomized Controlled Trials in Health Care, hosted by edX [5]), trainings and research networks (e.g., National Institutes of Health [NIH] Health Care Systems Research Collaboratory [6]), online resources (e.g., interactive eBook, Pragmatic Trials: A Workshop Handbook [7];), and funding opportunity announcements [8, 9]. Tools for helping research teams plan for trials along the explanatory-pragmatic continuum to match the overall intent and purpose of the trial are also available. One such tool is the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2 [10]).

The PRECIS tool was first developed by Thorpe and colleagues (2009) and revised by Loudon and colleagues (2015) through a collaborative, iterative process involving 80 international trialists [10, 11]. Briefly, the PRECIS-2 tool operationalizes elements or characteristics of trials that make them more or less pragmatic; in doing so, it encourages trialists to make purposeful decisions about the trial design to match the intent and purpose of the trial along the explanatory-pragmatic continuum. PRECIS-2 identifies nine domains of a trial that can make it more explanatory or more pragmatic: eligibility, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis.

The PRECIS-2 tool is intended to be used by the research team when planning for a trial. Through interactive, team-based discussions, each of the nine domains of the tool are scored on a five-point scale, with a score of one reflecting characteristics of that domain as very explanatory and a score of five reflecting characteristics of that domain as very pragmatic. Scores for each trial domain are represented on a wheel, where trials that are more explanatory have scores toward the center of the wheel, and trials that are more pragmatic have scores toward the periphery of the wheel. Scores for each domain can vary across the explanatory-pragmatic continuum, such that a score for one domain in one trial may be very pragmatic and a score for another domain within the same trial may be very explanatory. Rarely are all nine domains in a single trial scored as a one (very explanatory) or scored as a five (very pragmatic). Scores may also fluctuate throughout the duration of the trial, as planned or unplanned changes to specific domains may be necessary (e.g., recruitment, eligibility). As there are no objectively preferred individual or collective domain scores for a trial, each domain should match the intent and purpose of that particular trial.

The PRECIS-2 tool has been used to help design over 500 RCTs (personal communication, K. Loudon, August 15, 2020), with hundreds more retrospective assessments reported in published papers [12, 13]. It has demonstrated good interrater reliability and modest discriminant validity [14] and is often a major component of trainings and workshops for both research teams [5, 6] and funding agency staff [15]. With increased use and application of the PRECIS-2 tool to a wide range of trials, some users have identified a few aspects of the tool that would benefit from greater clarity and guidance.

One common source of confusion encountered by users of the PRECIS-2 tool is how to apply and interpret all nine domains to trials where participants are healthcare professionals (rather than patients) and trials are testing provider-focused strategies (rather than health interventions [16, 17]). While the eligibility and recruitment domains are similarly interpreted for trials involving patients or providers, it is less clear how to differentiate and operationalize all domains for trials testing health interventions that target patients compared to trials testing strategies that target healthcare professionals. Although the PRECIS-2 toolkit [10] states that “participants may be patients and/or healthcare professionals, interventions may target patients (e.g., medication) or healthcare professionals (e.g., continuing education),” it stops short of specifying how to conceptualize and interpret all domains for trials with healthcare professionals and provider-focused strategies. As trials designed to test provider-focused strategies vary from those that test patient-level interventions, the PRECIS-2 tool should vary, as well.

To clarify this issue, we propose an extension and adaptation of the PRECIS-2 tool tailored to trials that test strategies to change providers’ behavior. Such trials are quite common in implementation science, healthcare delivery research, and quality improvement research, where the overall goal is often to increase the adoption of evidence-based health interventions, de-implement ineffective interventions, and/or improve healthcare delivery and service provision. Consistent with the literature, we consider provider-focused strategies as those that target providers’ behavior; examples include audit and feedback, continuing education, and external coaching, among others [18].

The PRECIS-2-Provider Strategies (PS) tool was developed through a multi-step process involving experts from the original PRECIS tool (2009; MZ), revised PRECIS-2 tool (2015; KL, MZ), and implementation scientists (WEN, DAC). A series of 2-h discussions were held to deliberate the proposed domains, descriptions, and scores of PRECIS-2-PS. The PRECIS-2 worksheet and domain examples were used as points-of-comparison to contrast domains for trials where participants are patients and the target of health interventions vs. trials where participants are providers and the target of strategies. We leveraged our content expertise and trial experience to generate examples of a score of 1 (more explanatory) and a score of 5 (more pragmatic) for all nine domains. While formal reliability and validity testing was considered beyond the scope of the preliminary development of the tool, we nonetheless applied the PRECIS-2-PS tool to a small sample of four diverse trial protocols for pilot testing. Three of us independently read each trial protocol, scored each domain, and highlighted relevant text that informed the domain score. Scores and selected text were compared across coders; discrepancies were discussed until consensus was reached, and refinements to the tool were made accordingly (e.g., examples of domain scores). The implementation resources (#4) and data collection (#7) domains generated the most discussion. As the conceptualization of these two domains deviates the most from PRECIS-2, additional discussion of scores and refinement of PRECIS-2-PS was not unexpected. The remaining seven domains (e.g., eligibility, recruitment) generated fewer discrepancies and discussions.

We describe the PRECIS-2-PS tool below. We define and operationalize each of the nine domains of the PRECIS-2-PS tool and provide examples of characteristics that make that domain more explanatory (score 1) or more pragmatic (score 5). We provide guidance for how to use the tool during the trial planning phase, including a comprehensive toolkit (see Additional file). Finally, we build on our experience using the PRECIS-2 tool in training workshops and trial consultations to emphasize emergent issues and apply them to the PRECIS-2-PS tool. These include the complementary role of different stakeholder groups for trial design and the importance of understanding and describing implementation-as-usual.

Discussion

PRECIS-2-PS domains

Consistent with the PRECIS-2 tool, we include a total of nine domains for PRECIS-2-PS. We changed the domain name, key question, and/or description for all domains to match the specific nature of provider-targeted trials. A comparison on domain names, key questions, and score examples between PRECIS-2 and PRECIS-2-PS can be found in Table 1. We used the same wheel-and-spoke visualization of PRECIS-2 for the PRECIS-2-PS tool. A blank version of the PRECIS-2-PS wheel is shown in Fig. 1.

Table 1 Comparison of PRECIS-2 and PRECIS-2-PS on domains, key questions, and scores
Full size table
Fig. 1
figure 1

The PRagmatic-Explanatory Continuum Indicator Summary 2 Provider Strategies (PRECIS-2-PS) wheel

Full size image

Domain #1: Eligibility

The eligibility domain refers to characteristics of healthcare professionals who would qualify to participate in the trial. Healthcare professionals may include nurses, physicians, allied health professionals, specialists, patient navigators, social workers, community health workers, and other individuals who provide relevant services. Reflecting a very explanatory intent of the trial, a score of 1 for the eligibility domain would have extensive exclusion criteria. The sample of healthcare professionals eligible for and included in the trial would be a restricted subset of the full population of healthcare professionals to whom the findings of the trial would apply outside the context of a trial. Reflecting a very pragmatic intent of the trial, a score of 5 for this domain would have minimal exclusion criteria. The sample of healthcare professionals eligible for and included in the trial would be a representative subset of the full population of healthcare professionals to whom the findings of the trial would be expected to apply outside the context of a trial.

Domain #2: Recruitment

The recruitment domain refers to approaches used to recruit and enroll eligible healthcare professionals into the trial. The recruitment domain assesses to what extent additional time, effort, and resources are used to recruit participants into the trial compared to approaches that would be readily available in similar settings. Reflecting a very explanatory intent of the trial, a score of 1 would include extensive approaches in terms of additional time, effort, resources, and personnel for recruiting providers into the trial. Examples might include personalized invitation letters, free continuing medical education credits, monetary incentives, or other approaches that are uncommon or unavailable in routine care settings. Reflecting a very pragmatic intent of the trial, a score of 5 for this domain would include relatively few approaches. These approaches would be feasible in routine care settings, such as announcements at staff meetings, word-of-mouth, or flyers posted in break rooms.

Domain #3: Setting

The setting domain refers to characteristics of the setting (namely, organizations) in which the trial is conducted. Examples of organizations include hospitals, clinics, health centers, health departments, and community-based organizations, among others. Organizational characteristics may include location, size, resources, payment structures, culture, climate, and performance metrics. This domain assesses to what extent the organizations in the trial reflect organizations to which the results would apply; that is, are organizations included in the study a representative sample of organizations to which the results would apply, or are they unique in some way that makes them a poor representation of organizations to which the results are intended to apply?

Reflecting a very explanatory intent of the trial, a score of 1 for the setting domain would include a sample of organizations that are unusual or atypical in ways that make them a poor reflection of the full population of organizations to which the results are intended to be applied. Examples include organizations that are limited to a small geographic region, significantly under or over resourced, exceptionally high or low performers on quality metrics, or a single rarified delivery system. Reflecting a very pragmatic intent of the trial, a score of 5 for this domain would include organizations in the trial that closely represent the total population of organizations to which the trial is intended to be applied. Examples include organizations that are dispersed across a large geographic region, located in rural, urban, and peri-urban settings, and average performance on quality metrics.

Domain #4: Implementation resources

The implementation resources domain refers to the time, effort, and personnel needed to support the delivery of provider-focused strategies. This domain compares the intensity of strategies tested in the trial relative to what is or would likely be available to deliver those strategies outside the context of the trial. A very explanatory intent of the trial (score of 1) would reflect resources to support the delivery of strategies above and beyond what would be reasonable or available in similar care settings. This would include the use of strategies that are time-intensive, costly, extensive, and frequent, such as one-on-one in-person coaching sessions, and expensive expert-led external facilitation. A very pragmatic intent of the trial (score of 5) would use resources to deliver provider-focused strategies that are available or accessible in these settings, such as quality improvement teams and collaboratives, educational sessions, skill-building seminars or workshops, or electronic health record systems.

Domain #5: Flexibility of provider-focused strategies

The flexibility of delivery of provider-focused strategies domain refers to the flexibility in how, when, and by whom the strategies are delivered to providers within the trial. This domain reflects the extent to which individuals who deliver the strategies, and the detailed aspects of delivery of the strategies themselves, are pre-specified (e.g., training, experience, credentials), the format in which they can be delivered (e.g., in-person, online), the frequency and sequence in which they can be delivered, and the flexibility in selection, use, and adaptation of strategies. Reflecting a very explanatory trial, a score of 1 would include trials with strict protocols and structures that do not allow for or encourage adaptations to strategies, regardless of context. Reflecting a very pragmatic trial, a score of 5 would include trials with flexible guides, suggestions, and manuals for delivering provider-focused strategies that allow for or even encourage the adaptation of strategies to meet the context of the organization and the needs of healthcare professionals.

Domain #6: Flexibility of intervention

The flexibility of intervention domain refers to the degree to which healthcare professionals are able to adapt their use of the patient-focused intervention to their situation relative to the likely flexibility that would be available during usual care. A score of 1 on the flexibility of intervention domain, reflecting a very explanatory approach, would include strict protocols or structures in place to discourage or limit the extent to which the patient-focused intervention could be adapted by the provider. A score of 5 on the flexibility of intervention domain, reflecting a very pragmatic approach, would include explicit suggestions or encouragement for healthcare professionals to adapt the patient-focused intervention to the context.

Domain #7: Data collection

The data collection domain is a function of both the frequency and intensity of data collected throughout the duration of the trial from baseline through follow-up. This domain includes how often data are collected from participants as well as how extensive, intrusive, or time-consuming it is for participants to provide data compared to what would be considered routine within similar care settings. In a very explanatory trial (a score of 1), data would be collected quite often and require extensive time and effort on behalf of participants to complete. Very explanatory trials would also likely include (or be limited to) original data collection, including quantitative and/or qualitative data. In a very pragmatic trial (a score of 5), data would be collected less often and require little effort and time for completion. Very pragmatic trials would rely heavily on secondary data collection—that is, data that are readily available within routine care (e.g., electronic health records), with minimal or no original data collection.

Domain #8: Primary outcome

This domain is the extent to which the primary outcome is of interest and importance to healthcare professionals. Primary outcomes may include direct provider-level outcomes or indirect patient-level outcomes. Reflecting a very explanatory approach, a score of 1 on this domain would include trials where the primary outcome is some type of process or proxy variable, where the overall intent is to better explain or understand a mechanism or mediating variable that might predict the degree to which the provider-focused strategy is effective. Outcomes that would reflect a very explanatory trial include providers’ knowledge about an intervention, perceptions of leadership, or patients’ completeness of laboratory testing (where these tests may not be part of routine care). Reflecting a very pragmatic approach, a score of 5 on this domain would include trials where the primary outcome is of obvious interest and importance to healthcare professionals, such as their quality of care or job satisfaction, or their patients’ health status or quality of life.

Domain #9: Primary analysis

Primary analysis refers to the extent to which all data are used to assess the primary outcome of the trial. A trial that is more explanatory (a score of 1) would use an on-protocol approach to test the main study hypothesis, where only data from healthcare professionals who consented, received the provider-focused strategy (for experimental condition only), and completed all process and outcome measures of the study are included in the primary analysis. A more explanatory trial may also use less-robust data imputation techniques to account for attrition and missing data. Trials on the pragmatic end of the continuum (a score of 5) would use an intent-to-treat (ITT) or robust modified ITT analysis for comparing the trial arms on the primary outcome variable for all participants who consented into the trial [19,20,21,22].

Enhancing the use of the PRECIS-2-PS tool

The PRECIS-2-PS tool is intended to be used during the trial planning phase to help team members consider structured, contextual elements of the trial that would match the overall intent and purpose of the trial. Building on our collective experience with PRECIS-2 and the published literature [23,24,25], we make explicit two major recommendations for enhancing the use of the tool that were not fully articulated in previous versions. The first recommendation is to involve stakeholders throughout the planning, execution, and interpretation of the trial. The second recommendation is to describe implementation-as-usual at the outset of the trial and document major changes that may occur as the trial unfolds. Together, we believe these approaches strengthen the ability of the tool to better understand and characterize the context in which the trial will take place, ultimately providing better guidance to decision-makers. These two recommendations do not themselves constitute domains of PRECIS-2-PS, but do facilitate its effective use in the design of trials of provider-focused strategies.

Stakeholders

Individuals involved in the trial planning phase should include representatives from research and practice to ensure that the overall intent of the trial reflects important perspectives in the decision-making process of the trial [23, 24]. Stakeholders would include (but not be limited to) members of the interdisciplinary scientific team, organizational and healthcare professional partners participating in the trial, and representatives from the settings in which the trial results are intended to be applied and adopted. All stakeholders would provide unique yet complementary input during the trial planning phase [26]. Researchers and practitioners would rely on expert and tacit knowledge about specific topics that would otherwise be inaccessible or likely inaccurate if not provided by that group. Examples include healthcare professionals identifying priority topic areas and important outcomes; statisticians providing power calculations to determine sample size; and health system leaders describing the feasibility and acceptability of the proposed provider-focused strategies. See Table 2 for more examples of the roles that stakeholders may play when planning for the trial. By involving these stakeholder groups, one would be able to address another important aspect of using PRECIS-2-PS: understanding, describing, and measuring implementation-as-usual.

Table 2 Stakeholder questions for designing provider-focused trials to match the overall purpose and intent of the trial
Full size table

Implementation-as-usual

To date, most RCTs of multi-level implementation strategies (including but not limited to provider-focused strategies) use either an implementation-as-usual control condition or provide basic implementation support (e.g., 1-h training workshop, educational materials) as an enhanced implementation comparison condition. In reflecting generalizability to implementation-as-usual, it is important for trial teams to explicitly identify to what populations and settings the intent is to be generalizable, and to carefully consider the current approach to implementation in those settings. Given the importance of planning for and interpreting the results of trials, describing and defining implementation-as-usual is critically important [27, 28].

Of course, as with usual care for patients, implementation-as-usual is by no means the same across delivery sites or healthcare professionals. Improved characterization of implementation-as-usual at all trial sites at baseline and over time will offer greater understanding of to whom and where the results of the trial may be applicable. This also highlights the importance of tracking how implementation-as-usual may change over time as a result of changes to policies, reimbursement structures, or reorganization, which may enhance or limit the generalizability of study results to settings in which they are intended to be used. Note that all of these considerations similarly apply to usual care for PRECIS-2-PS, where applicable (i.e., flexibility of intervention).

Conclusion

The PRECIS-2-PS tool for provider-focused implementation trials aims to help research and practice stakeholders plan for trials where design decisions are matched with the overall intent and purpose of the trial. The PRECIS-2-PS tool builds on PRECIS-2 by adapting all nine domains to trials testing provider-focused strategies. In doing so, it enables planning for provider-focused trials along the explanatory-pragmatic continuum, with implications for advancing the field, and ultimately maximizing the impact of trial results on improving patient outcomes, care delivery, and population health. The PRECIS-2-PS tool also responds to recent calls for a better understanding of implementation strategies. We incorporate this thinking into the PRECIS-2-PS tool by encouraging trial teams to better specify strategies [29, 30], consider adaptation of strategies [31, 32], costs of strategies [33], and describe and track changes to strategies, usual care, and implementation-as-usual over time [34] before and during the trial.

Research is needed to further develop, validate, and apply the PRECIS-2-PS tool to diverse provider-focused implementation trials. The PRECIS-2-PS tool should be formally tested for interrater reliability and discriminant validity, following the sequence of development of PRECIS-2. Applying the PRECIS-2-PS tool prospectively during trial planning, and retrospectively to published protocols, would generate an abundance of examples of domain scores, and identify aspects of the tool that may need to be refined or further specified. In addition to focusing on aspects of the tool itself, research is needed on how best to communicate trial domain scores from the PRECIS-2-PS tool. Dissemination research can help elucidate what type of information generated from the PRECIS-2-PS toolkit may be most useful for decision-makers and in what format.

Compendiums of trial case studies of PRECIS-2-PS scores would also better enable calibration of scores in the planning process and facilitate training in how to use the tool. This would help identify which PRECIS-2-PS domains are most likely to change over time and in what direction along the continuum. It would help characterize the state of provider-focused trials along the explanatory-pragmatic continuum, and identify what additional trials are needed to enhance the generalizability and applicability of strategies. It is important to remember, however, that not all provider-focused trials should be pragmatic. Explanatory trials are not inherently “bad” and pragmatic trials are not inherently “good.” Rather, each type of trial serves a purpose. Given the relatively nascent state-of-the-science of implementation strategies, one might expect to see more explanatory trials testing mechanisms of change now and more pragmatic trials testing strategies in diverse, representative settings later.

The PRECIS-2-PS tool is intended to encourage research and practice stakeholders to design provider-focused trials that match the overall intent and purpose of trials. The tool creates an opportunity to enhance research-practice partnerships by making explicit the importance and complementary role of stakeholders in the trial design phase. It also highlights the need for more and better provider-centered research [35] by focusing on priority issues, questions, and outcomes that are of interest to them. This approach is necessary for maximizing the potential of provider-focused implementation trials to have an impact once studies have been completed and papers have been published. The PRECIS-2-PS tool provides a structured approach for bringing stakeholders together in designing elements of a trial to match the intent and purpose of that trial, whether it be more explanatory, more pragmatic, or somewhere in between.

Availability of data and materials

Not applicable.

References

  1. Bothwell LE, Greene JA, Podolsky SH, Jones DS. Assessing the gold standard--lessons from the history of RCTs. The New England journal of medicine. 2016;374(22):2175–81.

    Article  CAS  PubMed  Google Scholar 

  2. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference/William R. Shedish, Thomas D. Cook, Donald T. Campbell. Boston: Houghton Mifflin; 2002.

    Google Scholar 

  3. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967;20(8):637–48.

    Article  CAS  PubMed  Google Scholar 

  4. Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10:37.

    Article  PubMed  PubMed Central  Google Scholar 

  5. edX. Pragmatic randomized controlled trials in health care 2015 [Available from: https://www.edx.org/course/pragmatic-randomized-controlled-trials-in-health-c. Accessed 15 Aug 2020.

  6. NIH. Rethinking clinical trials: a living textbook of pragmatic clinical trials 2020 [Available from: https://rethinkingclinicaltrials.org/. Accessed 15 Aug 2020.

  7. ACCORDS D. Pragmatic trials: a workshop handbook: ACCORDS Dissemination and Implementation Science Program: ; 2019 [Available from: http://www.crispebooks.org/workbook-18OF-1845R.html#cover. Accessed 15 Aug 2020.

  8. DHHS. Accelerating colorectal cancer screening and follow-up through implementation science (ACCSIS, Second Wave) (UG3/UH3 Clinical Trial Required) 2019 [Available from: https://grants.nih.gov/grants/guide/rfa-files/rfa-ca-19-018.html. Accessed 15 Aug 2020.

  9. DHHS. HEAL initiative: pragmatic and implementation studies for the management of pain to reduce opioid prescribing (PRISM) (UG3/UH3, Clinical Trials Optional) 2019 [Available from: https://grants.nih.gov/grants/guide/rfa-files/rfa-at-20-004.html. Accessed 15 Aug 2020.

  10. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ : British Medical Journal. 2015;350:h2147.

    Article  PubMed  Google Scholar 

  11. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Journal of clinical epidemiology. 2009;62(5):464–75.

    Article  PubMed  Google Scholar 

  12. Sajobi TT, Li G, Awosoga O, Wang M, Menon BK, Hill MD, et al. A comparison of meta-analytic methods for synthesizing evidence from explanatory and pragmatic trials. Syst Rev. 2018;7(1):19.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Sepehrvand N, Alemayehu W, Das D, Gupta AK, Gouda P, Ghimire A, et al. Trends in the explanatory or pragmatic nature of cardiovascular clinical trials over 2 decades. JAMA Cardiol. 2019;4(11):1122–8.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Loudon K, Zwarenstein M, Sullivan FM, Donnan PT, Gágyor I, Hobbelen H, et al. The PRECIS-2 tool has good interrater reliability and modest discriminant validity. J Clin Epidemiol. 2017;88:113–21.

    Article  PubMed  Google Scholar 

  15. Norton WE, Zwarenstein M, Czajkowski S, Kato E, O'Mara A, Shelburne N, et al. Building internal capacity in pragmatic trials: a workshop for program scientists at the US National Cancer Institute. Trials. 2019;20(1):779.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, et al. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17:32.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Neta G, Johnson KE. Informing real-world practice with real-world evidence: the value of PRECIS-2. BMC Med. 2018;16(1):76.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Cochrane. Cochrane effective practice and organization of care 2020 [Available from: https://epoc.cochrane.org/our-reviews. Accessed 15 Aug 2020.

  19. Altman DG, Bland JM. Missing data. BMJ (Clinical research ed). 2007;334(7590):424.

    Article  Google Scholar 

  20. White IR, Horton NJ, Carpenter J, Statistics rim, social, Pocock SJ. Strategy for intention to treat analysis in randomised trials with missing outcome data. BMJ (Clinical research ed). 2011;342:d40.

    Article  Google Scholar 

  21. Montori VM, Guyatt GH. Intention-to-treat principle. Cmaj. 2001;165(10):1339–41.

    CAS  PubMed  PubMed Central  Google Scholar 

  22. Gupta SK. Intention-to-treat concept: a review. Perspect Clin Res. 2011;2(3):109–12.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Nicholls SG, Carroll K, Zwarenstein M, Brehaut JC, Weijer C, Hey SP, et al. The ethical challenges raised in the design and conduct of pragmatic trials: an interview study with key stakeholders. Trials. 2019;20(1):765.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Nicholls SG, Zwarenstein M, Hey SP, Giraudeau B, Campbell MK, Taljaard M. The importance of decision intent within descriptions of pragmatic trials. Journal of clinical epidemiology. 2020;125:30–7.

    Article  PubMed  Google Scholar 

  25. Zwarenstein M, Thorpe K, Treweek S, Loudon K. PRECIS-2 for retrospective assessment of RCTs in systematic reviews. Journal of clinical epidemiology. 2020.

  26. Vukov JM, Rempala K, Klug M, Hornewer M. From epistemic trespassing to transdisciplinary cooperation: the role of expertise in the identification of usual care. Am J Bioeth. 2020;20(1):52–4.

    Article  PubMed  Google Scholar 

  27. Nicholls SG, Zwarenstein M, Taljaard M. The importance of describing as well as defining usual care. Am J Bioeth. 2020;20(1):56–8.

    Article  PubMed  Google Scholar 

  28. Macklin R, Natanson C. Misrepresenting “Usual Care” in research: an ethical and scientific error. Am J Bioeth. 2020;20(1):31–9.

    Article  PubMed  Google Scholar 

  29. Waltz TJ, Powell BJ, Fernández ME, Abadie B, Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci. 2019;14(1):42.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177–94.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Chambers DA, Vinson CA, Norton WE. Advancing the science of implementation across the cancer continuum: Oxford University Press; 2018.

    Book  Google Scholar 

  32. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7:3.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Gold HT. Cost-effectiveness analysis in implementation science. Advancing Sci Implement Across Cancer Continuum. 2018;368:368–74.

  34. Bunger AC, Powell BJ, Robertson HA, MacDowell H, Birken SA, Shea C. Tracking implementation strategies: a description of a practical approach and early findings. Health research policy and systems. 2017;15(1):15.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Norton WE, Vinson CA, Chambers DA. Future directions in implementation science across the cancer continuum. In: Advancing the Science of Implementation across the Cancer Continuum. Eds. Chambers, Vinson, & Norton; 2018.

    Google Scholar 

Download references

Acknowledgements

We thank the reviewers for their thoughtful feedback on the initial version of this manuscript. We also thank the developers of the PRECIS and PRECIS-2 tools for their foundational work in advancing the conceptualization of trials along the explanatory-pragmatic continuum.

Funding

Drs. Merrick Zwarenstein (18BBCA0022) and Kirsty Loudon (18BBCA0025) receiving funding as consultants for developing the PRECIS-2-PS tool under prime contact number HHSN2612014000002B for ICF International Inc.

Author information

Authors and Affiliations

  1. Division of Cancer Control and Population Sciences, National Cancer Institute, 9609 Medical Center Drive, #3E424, Bethesda, MD, 20850, USA

    Wynne E. Norton & David A. Chambers

  2. Independent Contractor, Edinburgh, Scotland

    Kirsty Loudon

  3. Centre for Studies in Family Medicine, Department of Family Medicine, Western University, London, Ontario, Canada

    Merrick Zwarenstein

Authors
  1. Wynne E. Norton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kirsty Loudon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David A. Chambers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Merrick Zwarenstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors participated in the development of the PRECIS-2-PS tool. WEN, MZ, KL, and DAC engaged in structured discussions about adapting the PRECIS-2 tool domains, key questions, and example scores to provider-focused implementation trials. WEN, MZ, and KL piloted the PRECIS-2-PS tool with select trial protocols. All authors discussed any discrepancies in coding and refined PRECIS-2-PS domains, key questions, and example scores, where appropriate. WEN drafted the initial version of the manuscript. All authors contributed to edits and revisions. All authors read and approved the final version.

Corresponding author

Correspondence to Wynne E. Norton.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

WEN is on the Editorial Board of the journal Implementation Science.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

PRECIS-2-Provider Strategies Toolkit.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Norton, W.E., Loudon, K., Chambers, D.A. et al. Designing provider-focused implementation trials with purpose and intent: introducing the PRECIS-2-PS tool. Implementation Sci 16, 7 (2021). https://doi.org/10.1186/s13012-020-01075-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-020-01075-y

Keywords

Implementation Science

ISSN: 1748-5908