Skip to main content
Download PDF
Download ePub
  • Systematic review
  • Open access
  • Published:

Reporting of costs and economic impacts in randomized trials of de-implementation interventions for low-value care: a systematic scoping review

Implementation Science volume 18, Article number: 36 (2023) Cite this article

Abstract

Background

De-implementation of low-value care can increase health care sustainability. We evaluated the reporting of direct costs of de-implementation and subsequent change (increase or decrease) in health care costs in randomized trials of de-implementation research.

Methods

We searched MEDLINE and Scopus databases without any language restrictions up to May 2021. We conducted study screening and data extraction independently and in duplicate. We extracted information related to study characteristics, types and characteristics of interventions, de-implementation costs, and impacts on health care costs. We assessed risk of bias using a modified Cochrane risk-of-bias tool.

Results

We screened 10,733 articles, with 227 studies meeting the inclusion criteria, of which 50 included information on direct cost of de-implementation or impact of de-implementation on health care costs. Studies were mostly conducted in North America (36%) or Europe (32%) and in the primary care context (70%). The most common practice of interest was reduction in the use of antibiotics or other medications (74%). Most studies used education strategies (meetings, materials) (64%). Studies used either a single strategy (52%) or were multifaceted (48%). Of the 227 eligible studies, 18 (8%) reported on direct costs of the used de-implementation strategy; of which, 13 reported total costs, and 12 reported per unit costs (7 reported both). The costs of de-implementation strategies varied considerably. Of the 227 eligible studies, 43 (19%) reported on impact of de-implementation on health care costs. Health care costs decreased in 27 studies (63%), increased in 2 (5%), and were unchanged in 14 (33%).

Conclusion

De-implementation randomized controlled trials typically did not report direct costs of the de-implementation strategies (92%) or the impacts of de-implementation on health care costs (81%). Lack of cost information may limit the value of de-implementation trials to decision-makers.

Trial registration

OSF (Open Science Framework): https://osf.io/ueq32.

Peer Review reports

Background

Efficient use of health resources benefits both individuals and society — and one way to increase efficiency is to abandon obsolete and ineffective health interventions [1, 2]. De-implementation is typically aimed at reducing the use of low-value care, which has been described as providing little or no benefit, being potentially harmful, and leading to unnecessary costs to patients or wasting health care resources [3]. To achieve this, de-implementation strategies are needed. De-implementation, a process to reduce the use of a medical practice, can occur in four different ways, by removing, replacing, reducing, or restricting the use [4]. Each category has different underlying reasons, and therefore, different solutions may be needed [5]. It is easier to implement new interventions than it is to de-implement existing medical practices [6].

Different terms are used from these withdrawn actions, like de-implementation and disinvestment [7]. Public policy concepts, like disinvestment, are relevant to de-implementation study. Many de-implementation frameworks and models mention costs as a justification for de-implementation [8]. De-implementation has the potential to decrease health care costs [5, 7, 9], and bringing these out requires an evaluation of clinical practices and care pathways. De-implementation can increase health care costs but is still cost cutting to the society. Health technology assessment is one way to assess these changes in clinical practices and care pathways [10]. Differences between health care systems in different countries affect clinical practices and care pathways and costs, which must be taken into account when transferring information from one country to another.

Economic evaluation has been pointed out to be crucial part of implementation research [11,12,13,14,15,16], and costs are identified as a key outcome in implementation research [13, 17]. An economic evaluation can bring out whether using a strategy to improve the quality of health care is a cost-effective use of limited resource [13]. Without knowing the costs of implementation strategies, it is also difficult, or even impossible, to compare different strategies or even implement them [14]. Accordingly, implementation studies should report the relevant costs of an implementation strategy, the sources of costs data, and how costs are calculated. Costs should include all costs from development to execution, such as staff, material, and training costs [12]. However, a previous systematic review showed that the quantity of economic evaluation in the field of implementation research is modest and called for more systematic and comprehensive reporting of costs in implementation research [12]. In economic evaluation of guideline implementation, there are three distinct stages: development of the guidelines, implementation of the guidelines, and treatment effects and costs as a consequence of behavior change. Systematic review of these cost brought out that costs were reported in a quarter of studies (27%), methodological quality was poor, and none of the included studies gave reasonable complete information of costs [18].

The above considerations are equally valid when de-implementation is concerned [19] as de-implementation processes are observed to be difficult and resource-intensive and the actual costs and subsequent savings are not well understood [3, 4]. A study [19] conceptualized the outcomes of de-implementation and recommended a clear distinction between the target of de-implementation and the strategies used. The recommendations included several aspects, such as potential cost savings due to decreased use of the target intervention, the costs of de-implementation strategies, the impacts on health care providers, and time, which should be considered when measuring the costs of de-implementation.

Since the aim of implementation and de-implementation is similar — to improve the quality of health care and effective use of resources — de-implementation strategies need to measure clinically relevant outcomes but also to analyze whether a strategy leads to a change in health care costs. The potential savings in health care costs as well as the costs of de-implementation strategy itself must be taken into account.

The aim of this systematic scoping review was to analyze how de-implementation studies have reported both the costs of de-implementation strategies and the impacts (estimated or measured) of de-implementation on health care costs.

Methods

We used the PRISMA extension for scoping reviews (PRISMA-ScR) [20] to guide the conducting and reporting of this review (Additional file 1). This analysis of de-implementation costs and de-implementation impacts on health care costs was undertaken as part of a systematic scoping review of de-implementation randomized controlled trials (RCTs) [21]. This systematic scoping review was registered with Open Science Framework (OSF ueq32).

Data sources and searches

Literature searches for these economic analyses are drawn from the registered systematic scoping review and are described in detail elsewhere [21]. We searched for de-implementation RCTs in the MEDLINE and Scopus databases up to May 24, 2021, without language or publication date limitations. The search strategy (Additional file 2) was developed in consultation with a medical information specialist (T. L.). We based our search on a previous scoping review identifying de-implementation-related terms [7] and modified it iteratively based on systematic reviews [22, 23]. We searched the reference lists of systematic reviews identified by our search to find additional potentially eligible articles. We also followed up protocols and post hoc analyses and added their main articles to the selection process.

Study inclusion and exclusion criteria

The inclusion and exclusion criteria are described previously [21]. In brief, we included RCTs that aimed to reduce the use of a clinical practice. We included all de-implementation intervention types on any clinical practice and all target groups (patients, health care personnel, organizations, and citizens in general). We excluded articles on de-prescribing trials, because in our opinion the context is different (stopping a treatment already in use vs. not starting a treatment) [24]. We also excluded trials where one medical practice was used to de-implement another medical practice and trials where the reason to de-implementation was to reduce resource use (e.g., financial resources or clinical visits) [21].

Risk of bias

To assess the quality of the included studies, we used a modified Cochrane risk-of-bias tool (RoB2.0) for randomized trials [25]. The process of modification is described in detail elsewhere [21]. This modified tool includes six criteria, judging studies to be at either high or low risk of bias (Additional file 3). The six criteria are as follows: (1) randomization procedure, (2) allocation concealment, (3) blinding of outcome collectors, (4) blinding of data analysts, (5) missing outcome data, and (6) imbalance of baseline characteristics. Four of the researchers conducted the quality assessment independently and in duplicate.

Data collection and extraction strategy

Both independently and in duplicate, we used standardized forms with detailed instructions in identifying eligible articles (titles and abstract and full-text screening) and in data extraction. Disagreements were resolved through discussion and, if necessary, through consultation of a third investigator.

We collected the following data: (1) study characteristics (i.e., author(s), year, country of origin, sample size), (2) types and characteristics of interventions (i.e., intervention strategy, target groups of intervention), (3) characteristics of the practice of interest (i.e., target intervention, medical content area, medical settings), (4) outcomes of the study, (5) intervention efficacy, (6) costs of de-implementation (i.e., total costs, costs per unit), and (7) effect on health care costs (target group, size and direction of effect, and what was measured or estimated). The costs of de-implementation had to be reported in monetary form, and total or per unit cost were specified by the study authors. Data regarding costs of de-implementation and effect on health care costs are reported in this article; other outcomes are reported elsewhere [21].

Data synthesis and analysis

We summarized the characteristics and details of de-implementation strategies and target population(s) and provided an overview on de-implementation costs. We extracted the costs in the reported currency and converted it into USD in 2021 value to facilitate comparability across all included studies. We changed the currency from EUR to USD, because more studies have used USD. We used a modified Effective Practice and Organisation of Care (EPOC) taxonomy [21] to categorize interventions and to analyze possible cost differences between different de-implementation strategies.

Finally, we provided an overview of the impact of de-implementation on health care costs. We reported the direction of the effects and cost allocations. We relied on the authors’ conclusion on the significance of the effect. We analyzed possible between-study differences in influence on health care costs. This was reported in various ways (monetary and qualitative).

We planned to do subgroup analyses based on (i) health care settings, (ii) target of intervention, (iii) health care financing, and (iv) country income groups. We assumed beforehand that the studies would be heterogeneous so a meta-analysis would not provide any added value.

We used summary statistics (i.e., frequencies and proportions) to describe study characteristics. We used nonparametric tests to analyze differences between outcomes of our interest. For statistical analyses, we used IBM® SPSS® version 28.0.1 (IBM Corp., Armonk, NY, USA), and all reported P-values less than 0.05 were considered statistically significant.

Results

Of the 12,815 articles identified in our search, we evaluated 1022 full-text articles. We included 227 RCTs, of which only 50 (22%) reported any costs or impact on health care costs. Figure 1 presents the PRISMA flow diagram, and a list all of included studies is found in Additional file 4.

Fig. 1
figure 1

PRISMA flow diagram

Full size image

The publication dates of the included articles ranged from 1982 to 2021, where half the articles dated after 2010. Most articles were from North America (n = 18, 36%) and Europe (n = 16, 32%). The majority of the studies (n = 41, 82%) were targeted to one type of professionals, and nine studies (18%) reported several target groups. Around two-thirds of the studies (n = 35, 70%) were conducted in primary care. The trials were aimed at reducing the use of drug treatments (n = 37, 74%), laboratory test (n = 8, 16%), or diagnostic imaging (n = 6, 12%). The studies used 16 different de-implementation strategies. Twenty-six used only one strategy, and twenty-four were multifaceted including two or more strategies. In all studies, the goal was to reduce use of a specific health care intervention. In 14 studies, an additional goal was replacing. The description of the characteristics of the included studies is shown in Table 1, and full characteristics are found in Additional file 5.

Table 1 Characteristics of the de-implementation interventions
Full size table

Risk of bias

Randomization was adequately generated in all studies. However, allocation concealment was not adequate in 10 studies (20%), 22 studies (44%) had missing data, and 20 (40%) had imbalance in baseline characteristics. Data collectors were blinded in 41 studies (82%) but data analysts in only two studies (4%) (Table 2).

Table 2 Risk-of-bias assessment
Full size table

De-implementation costs

The total costs of de-implementation intervention were reported in 13 studies (6%). These total costs varied considerably, the median being US $32,300 (range: US $616 to 747,000). Table 3 shows total costs converted to US dollars in 2021 value.

Table 3 De-implementation total costs and costs per unit, converted by the authors to USD 2021 value (on October 25, 2022)
Full size table

The 13 studies (26%) that reported total costs used ten different de-implementation strategies. The most common strategies were educational materials (n = 9), audit and feedback (n = 7), educational meetings for individuals (n = 4), treatment algorithm (n = 3), educational meetings for groups (n = 3), and developing clinical practice guidelines (n = 2). The strategies used in one study included alerts, local consensus process, educational material for patients, and public intervention. Strategy combinations were diverse; many combined different educational strategies together. When using educational material in de-implementation, the median for total costs median was US $118,000 (range: US $6845 to 747,000). The total costs seemed higher in studies using educational materials than in studies not using such materials when not using it (Mann–Whitney test, p = 0.05). For other strategies, the total costs did not significantly differ between studies using vs. from not using each strategy (Mann–Whitney test, all p > 0.05).

In studies that used only one de-implementation strategy (n = 4, 31%), the median for total costs was US $8090 (range: US $616 to 32,300). In studies using two de-implementation strategies (n = 2, 15%), the median for total costs was US $224,000 (range: US $118,000 to 330,000), and with three or more strategies (n = 7, 54%), the median for total costs was US $43,600 (range US $6845 to 747,000).

Costs per unit were reported in 12 studies (5%). The most common unit was cost per physician, but also costs per health care provider, health care unit, day, and patient were reported (Table 3). In these studies, various de-implementation strategies were used. The most common were educational materials (n = 8), educational meetings for individuals (n = 6), and for groups (n = 5), audit and feedback (n = 5), and educational materials for patients (n = 2). Alerts, treatment algorithms, public intervention, and developing clinical practice guidelines were each used once. There were no differences in costs between using a given strategy and not using it (Mann–Whitney test, all p > 0.05).

Of the articles that reported total costs or costs per unit, 10 out of 18 (56%) offered at least some detailed information on the costs, but only four (22%) reported the exact costs. The most frequently reported types of costs were material costs, payment for trainers, and travel costs. In addition, postage, rent of premises, and loss of working hours were occasionally reported. Cost related to de-implementation intervention planning was rarely brought out. Information on costing methods was not mentioned in the articles. None of the articles separated costs related to the phases of de-implementation.

A meta-analysis was not possible due to the heterogeneity of the studies (e.g., the type and number of de-implementation strategies used). There were few studies in the pre-specified subgroups, so subgroup analyses were not appropriate.

Impact on health care costs

The impact on health care costs was reported in 43 studies (19%). In most cases, the reports did not specify to whom the impact was targeted (n = 25, 58%). In four studies (9%), the impact was on patients’ own costs, whereas in 14 studies (33%), it was on health care provider’s costs. In 27 (63%) studies, health care costs decreased, whereas in 14 (33%), there was no change, and in two (5%) studies, the costs increased. The impact was targeted to medicine costs (n = 29, 67%), laboratory test costs (n = 8, 19%), total health care utilization costs (n = 3, 7%), diagnostic testing costs (n = 2, 5%), and radiography costs (n = 2, 5%).

Most of the articles (n = 32, 74%) have based their assessments on calculations on differences in costs between intervention and control group. In eight studies (19%), the authors have expanded the intervention group costs changes to large groups or for longer time. In one study [29], the authors have performed cost-effectiveness analyses, and in two studies, costs that were reported were costs changes during intervention period.

The two studies [30, 31] with increased costs had minor increases in costs allocated to patients. When the impact was allocated to the health care unit, the de-implementation either decreased costs (n = 12, 86%) or had no effect on costs (n = 2, 14%). In six studies, the authors estimated the impact on health care costs. De-implementation influenced laboratory test costs (n = 6), medicine costs (n = 5), diagnostic testing costs (n = 2), radiology costs (n = 2), and total health care expenditure per visit (n = 1). Table 4 shows the direction and size of the impact. The size of the impact was reported in different ways (Table 4).

Table 4 De-implementation impact on health care costs per allocated health care unit
Full size table

Of the 25 studies, which did not detail allocation of the impact, 14 (56%) reported a decrease in costs, whereas 11 (44%) reported no change (Table 5). The impact was calculated in twelve studies (48%) and estimated in five studies (20%). In the rest of studies, it was not possible to assess from the report whether the impact was calculated or estimated. In most of the studies (n = 20, 80%), the de-implementation mainly influenced the costs of medicine and laboratory tests. The change in reported health care cost varied between US $12.6 per patient to US $80.4 million per country. Of these 25 studies, 15 reported the impact in a monetary measure using different currencies (Table 5).

Table 5 De-implementation impact on health care costs in studies, not specifying to whom change was allocated
Full size table

The 43 studies that reported impact on health care costs used 15 different de-implementation strategies. The most common strategies were educational meetings for groups (n = 14, 33%), educational materials (n = 13, 30%), audit and feedback (n = 8, 19%), educational meetings for individuals (n = 6, 14%), treatment algorithm (n = 5, 12%), educational materials for patients (n = 5, 12%), and developing clinical practice guidelines (n = 3, 7%). Two studies used public release of performance data and patient-mediated interventions. The strategies used in one study included financial incentives for health care workers, local consensus process, local opinion leaders, managerial supervision, and routine patients-reported outcome measures.

Total costs of de-implementation and the impact on health care costs were reported in seven articles (14%), while unit costs and impact on health care costs were reported in five (10%) articles (Table 6). The articles by Zwar et al. [27] and Butler et al. [26] reported both total and unit costs, and the unit costs were in the same unit as the impact was reported. In two studies [27, 28], the intervention unit costs were less than their impact on health care costs. In the study by Butler et al. [26], the authors commented that their study decreased health care costs, but the intervention costs exceeded the savings.

Table 6 De-implementation costs and impact on health care costs in USD (converted by authors in October 2022)
Full size table

Discussion

Even though de-implementation is often justified by emphasizing control of health care costs [5, 7, 32], our findings indicate that intervention costs or impact on health care costs was rarely reported in randomized trials of de-implementation. Even when costs were reported, the information on intervention costs or impact on health care costs was minimal. Costs related to data collection and analysis or de-implementation interventions planning were rarely brought out. We also found that methods for reporting intervention costs and impact on health care costs were heterogeneous, which obscures the relationship between costs and impact. Our results are similar to a previous systematic review in implementation research [13] that found aspects that were not adequately covered, such as project management costs, time costs for clinical time, and monitoring costs. A systematic review [18] on economic evaluations and cost analysis in guideline implementation found similar limitations in trial reporting. In all of the studies, the quality of cost information was limited, and only 27% of 235 studies reported any information on costs [18].

For economic evaluation, information on resource use, costs, time horizons, health outcomes, or the consequences of interventions are necessary [33]. Incomplete cost information on de-implementation interventions does not allow economic evaluation or, at worst, may lead to distorted conclusions. The lack of cost information has been identified as a barrier to implementation [12, 14]. De-implementation requires sufficient financial, technical, and human resources [34]. The lack of cost information makes it impossible to evaluate the costs in a systematic way or to basing decision-making on this information. Knowledge-based decisions become possible only when intervention costs and impact on health care costs are both known.

To improve the utilization of de-implementation research, economic evaluation should be planned along with the research. Subsequently, the studies should report precise monetary costs of de-implementation strategies and their impact on health care costs. When reporting costs, general considerations should be taken into account: (i) give detailed and reasoned values, (ii) separate included costs, (iii) provide the time horizon when the costs are applicable, and (iv) break down the planning and acting phase costs of the de-implementation process.

Strengths and limitations

Our highly sensitive literature search used a wide variety of de-implementation terms. However, due to heterogeneous indexing of de-implementation studies, it is possible that we may have missed relevant articles.

A strength of our article is that we searched for cost information and de-implementation impact information from the full text of articles, which noticeably increased the number of included articles — as the impact on health care costs tended to be reported in the full text, not in the abstract.

We restricted our study to RCTs, which may be seen as a limitation. Since many de-implementation projects have likely not included randomized control groups, we missed economic information from these studies. However, the efficacy of interventions should be studied in RCT settings, and thus, we believe that our decision to exclude other study designs is justified.

This review was made alongside with another review, which may have restricted the number of included studies. We excluded studies where one medical practice was used to de-implement another, because these often focus on implementation not on de-implementation. We focused on de-implementation of low-value care, and therefore, excluded articles were the reason for de-implementation which was cutting resource use. Both these restrictions may have excluded some articles where cost information could have been given. However, we believe that our careful selection of articles in the full-text phase has prevented this. Our perspective was to find out how costs are brought out in de-implementation studies, so the search was made on that view. It could be a limitation, and some studies with costs may have been missed. To avoid this, we searched also the references of included studies to find other articles on same studies. Using the approach that Vale has used, it may lead to a different result.

Conclusion

A vast majority of de-implementation trials have failed to report any intervention costs or impacts on health care costs. In studies that do include cost information, typically only nonnumerical information on economics impacts is reported, and direct costs of de-implementation strategies are excluded. To advance the field, researchers should consider economic aspects and include health economists when planning research. De-implementation interventions are often complex and resource intensive, and cost information is essential for effective health policymaking.

Availability of data and materials

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

PRISMA-ScR:

Prisma extension for scoping review

RCT:

Randomized controlled trial

RoB:

Risk of bias

USD:

US dollars

EPOC:

Effective practice and organization of care taxonomy

FRF:

French franc

CNY:

Renminbi

AUD:

Australian dollars

GBP:

Great Britain pounds

AB:

Antibiotics

QHES:

Quality of health economics studies

References

  1. Rooshenas L, Owen-Smith A, Hollingworth W, Badrinath P, Beynon C, Donovan Jl. “I won’t call it rationing…”: an ethnograpic study of healtcare disinvestment in theory and practice. Soc Sci Med. 2015;128:273–281. https://doi.org/10.1016/j.socimed.2015.01.020.

  2. Calabrò GE, La Torre G, de Waure C, Villari P, Federici A, Ricciardi W, et al. Disinvestment in healthcare: an overview of HTA agencies and organizations activities at European level. BMC Health Serv Res. 2018;18(1):148. https://doi.org/10.1186/s12913-0185-2941-0.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Chambers JD, Salem MN, D’Cruz BN, Subedi P, Kanal-Bahl SJ, Neumann PJ. A review of empirical analyses of disinvestment initiatives. Value in Health. 2017;20:909–18. https://doi.org/10.1016/jval.2017.03.015.

    Article  PubMed  Google Scholar 

  4. Norton WE, Chambers DA. Unpacking the complexities of de-implementation inappropriate health interventions. Implement Sci. 2020;15:2. https://doi.org/10.1186/s13012-019-0960-9.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Verkerk EW, Tanke MAC, Kool RB, van Dulmen SA, Westert GP. Limit, lean or listen? A typology of low-value care that gives direction in de-implementation. Int J Qual Health Care. 2018;30(9):736–9. https://doi.org/10.1093/intqhc/mzy100.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Haas M, Hall J, Viney R, Gallero G. Breaking up is hard to do: why disinvestment in medical technology is harder than investment. Aust Health Rev. 2012;26:148–52. https://doi.org/10.1071/AH11032.

    Article  Google Scholar 

  7. Niven DJ, Mrklas KJ, Holodinsky JK, Straus SE, Hemmelgarn BR, Jeffs LP, et al. Towards understanding the de-adoption of low-value clinical practices: a scoping review. BMC Med. 2015;13:255–75. https://doi.org/10.1186/s12916-015-0488-z.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Walsh-Bailey C, Tsai E, Tabak RG, Morshed AB, Norton WE, McKay VR, et al. A scoping review of de-implementation frameworks and models. Implement Sci. 2012;16:100. https://doi.org/10.1186/s13012-021-01173-5.

    Article  Google Scholar 

  9. Chassin MR, Galvin RW, The National Roundtable on Health Care Quality. The urgent need to improve health care quality. Institute of medicine national roundtable on health care quality. JAMA. 1998;280(11):1000–5.

    Article  CAS  PubMed  Google Scholar 

  10. Health Technology Assessment, WHO. Health technology assessment - Global (who.int) Accessed 10. June 2023.

  11. Hoomans T, Severens JL. Economic evaluation of implementation strategies in health care. Implement Sci. 2014;9:168. https://doi.org/10.1186/s13012-014-0168-y.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Gold HT, McDermott C, Hoomans T, Wagner TH. Cost data in implementation science: categories and approaches to costing. Implement Sci. 2022;17:11. https://doi.org/10.1186/s13012-021-01172-6.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Roberts SEL, Healey A, Sevdalis N. Use of health economic evaluation in the implementation and improvement science fields - a systematic literature review. Implement Sci. 2019;14:72. https://doi.org/10.1086/s13012-019-0901-7.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Eisman AB, Kilbourne AM, Dopp AR, Saldana L, Eisenberg D. Economic evaluation in implementation science: making the business case for implementation strategies. Psychiatry Res. 2020;283:112433. https://doi.org/10.1016/j.psychres.2019.06.008.

    Article  PubMed  Google Scholar 

  15. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7:3. https://doi.org/10.3389/fpubh.2019.00003.

    Article  PubMed  PubMed Central  Google Scholar 

  16. O’Leary MC, Hassmiller Lich K, Frerichs L, Leeman J, Reuland DS, Wheeler SB. Extending analytic methods for economic evaluation in implementation science. Implement Sci. 2022;17:27. https://doi.org/10.1186/s13012-022-01192-w.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65–76. https://doi.org/10.1007/s10488-010-0319-7.

    Article  PubMed  Google Scholar 

  18. Vale L, Thomas R, MacLennan G, Grimshaw J. Systematic review of economic evaluations and cost analyses of guideline implementation strategies. Eur J Health Econ. 2007;8:111–21. https://doi.org/10.1007/s10198-007-0043-8.

    Article  PubMed  Google Scholar 

  19. Prusaczyk B, Swindle T, Curran G. Defining and conseptualizing outcomes for de-implementation: key distinctions from implementation outcomes. Implement Sci Commun. 2020;1:43. https://doi.org/10.1186/s43058-020-00035-3.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MD, Horsley T, Weeks L, Hempel S, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. https://doi.org/10.7326/M18-0850.

  21. Raudasoja AJ, Falkenbach P, Vermooij RWM, Mustonen JMJ, Agarwak A, Aoki Y, et al. Randomized controlled trials in de-implementation research: a systematic scoping review. Implement Sci. 2022;17:65. https://doi.org/10.1186/s13012-022-01238-z.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Colla CH, Mainor AJ, Hargreaves C, Sequist T, Morden N. Interventions aimed at reducing use of low-value health services. a systematic review. Med Care Res Rev. 2017;74(5):507–50. https://doi.org/10.1177/107755871665670.

  23. Köchling A, Löffner C, Reinsch S, et al. Reduction of antibiotic prescriptions for acute respiratory tract infection in primary care: a systematic review. Implement Sci. 2018;13(1):47. https://doi.org/10.1186/s13012-018-0732-y.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Steinman MA, Boyd CM, Spar MJ, Norton JD, Tannenbaum C. Deprescribing and deimplementation: time for transformative change. J am Geriatr Soc. 2021;69:3693–5. https://doi.org/10.1111/jgs.17441.

    Article  PubMed  Google Scholar 

  25. Eldridge S, Campbell M, Campbell M, Drahota A, Giraudeau B, Higgins J, et al. Revised Cochrane risk of bias tool for randomized trials (RoB 2.0) Additional considerantions for cluster-randomized trials. https://www.riskofbias.info/welcome/rob-2-0-tool/archive-rob-2-0-cluster-randomized-trials-2016 (Accessed 9 May 2020).

  26. Butler CC, Simpson SA, Dunstan F, Pollnick S, Cohen D, Gillespie D, et al. Effectiveness of multifaceted educational programme to reduce antibiotic dispensing in primary care: practice based randomized controlled trial. BMJ. 2012;344:d8173. https://doi.org/10.1136/bmj.d8173.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Zwar N, Wolk J, Gordon J, Sanson-Fisher R, Kehoe L. Influencing antibiotic prescribing in general practice: a trial of prescriber feedback and management guidelines. Fam Pract. 1999;16:495–500.

    Article  CAS  PubMed  Google Scholar 

  28. Soumerai SB, Salem-Schatz S, Avorn J, Casteris CS, Ross-Degnan D, Propovsky MA. A controlled trial of educational outreach to improve blood transfusion practice. JAMA. 1993;270(8):961. https://doi.org/10.1001/jama.1993.03510080065033.

    Article  CAS  PubMed  Google Scholar 

  29. Naughton C, Feely J, Bennett K. A RCT evaluating the effectiveness and cost-effectiveness of academic detailing versus postal prescribing feedback in changing GP antibiotic prescribing. J Eval Clin Pract. 2009;15:807–12. https://doi.org/10.1111/j.1365-2753.2008.01099.x.

  30. Ngasala B, Mubi M, Warsame M, Petzold MG, Massele AY, Gustafsson LL, et al. Impact of training in clinical and microscopy diagnosis of childhood malaria on antimalarial drug prescription and health outcome at primary health care level in Tanzania: a randomized controlled trial. Malar J. 2008;7:199. https://doi.org/10.1186/1475-2875-7-199.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Phuong HL, Nga TTT, Giao PT, Hung LQ, Bihn TQ, Nam NV, et al. Randomised primary health center based interventions to improve the diagnosis and treatmentof undifferentiated fever and dengue in Vietnam. BMC Health Serv Res. 2010;10:275 biomedcentral.com/1472-6963/10/275.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Berwick DM, Hackbart AD. Eliminating waste in US Health Care. JAMA. 2012;307(14):1513–6. https://doi.org/10.1001/jama2012.362.

    Article  CAS  PubMed  Google Scholar 

  33. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value in Health. 2013;16:e1–5. https://doi.org/10.1016/j.val.2013.02.010.

    Article  PubMed  Google Scholar 

  34. Kroon D, van Dulmen SA, Westert GP, Jeurissen PPT, Kool RB. Development of the SPREAD framework to support the scaling of de-implementation strategies: a mixed-methods study. BMJ Open. 2022;12:e062902. https://doi.org/10.1136/bmjopen-2022-062902.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Alexander E, Weingarten S, Mohsenifar Z. Clinical Strategies to Reduce Utilization of Chest Physioterapy Without Compromising Patient Care. Chest 1996;110:430–32.

  36. Bexell A, Lwando E, von Hofsten B, Tembo S, Eriksson B, Diwan VK. Improving Drug Use through Continuing Education: A Randomized Controlled Trial in Zambia. H Clin Epidemiol 1996;49(3):355–7.

  37. Dormuth CR, Carney G, Taylor S, Bassett K, Maclure MA. Rondomized Trial Assessing the Impact of a Personal Printed Feedback Portrait on Statin Prescribing in Primary Care. J of Continuing Education in the Health Professions. 2012;32(3):153–62.

  38. Gulliford MC, Juszczyk D, Prevost AT, Soames J, McDermott L, Ashworth m, et al. Electronically-delivered, multicomponent intervention fro antimicrobial stewardship in primary care. Cluster randomized controlled trial (REDUCE trial) and cohort study of safety outcomes. BMJ (Clinical research ed.) 2019;364:1236.

  39. Hemkens LG, Saccilotto R, Reyes SL, Glinz D, Zumbrunn T, Grolimund O, et al. Personalized Prescription Feedback Using Routinely Collected Data to Reduce Antibiotic Use in Primary Care. JAMA Internal Medicine 177(2);176. https://doi.org/10.1001/jamainternmed.2016.8040.

  40. Köpke S. Mühlhauser I. Gerlach A. Haut A. Haastert B. Möhler R et al. Effect of a Guideline-Based Multicomponent Intervention on Use of Physical Restraints in Nursing Homes. A Randomized Controlled Trial. JAMA. 2012;307(20):2177–84.

  41. Ashworth N, Kain N, Wiebe D, Hernandez-Ceron N, Jess E, Mazurek K. Reducing prescribing of benzodiazepines in older adults: a comparison of four physician-focused interventions by a medical regulatory authority. Abstract BMC Family Practice. 2021;22(1). https://doi.org/10.1186/s12875-021-01415-x.

  42. Optimizing antibiotic prescribing for acute cough in general practice: a cluster-randomized controlled trial. J Antimicrob Chemother. 2004;54(3):661–72. https://doi.org/10.1093/jac/dkh374.

  43. Electronic Health Records for Intervention Research: A Cluster Randomized Trial to Reduce Antibiotic Prescribing in Primary Care (eCRT Study). Ann Fam Med. 2014;12(4):344–51. https://doi.org/10.1370/afm.1659.

  44. Nejad AS, Farrokhi Noori MR, Haghdoost AA, Bahaadinbeigy K, Abu-Hanna A, Eslami S. The effect of registry-based performance feedback via short text messages and traditional postal letters on prescribing parenteral steroids by general practitioners—A randomized controlled trial. Int J Med Inform. 2016;8736–43. https://doi.org/10.1016/j.ijmedinf.2015.12.008.

  45. Solomon DH, van Houten L, Glynn RJ, Baden L, Curtis K, Schrager H, et al. Academic Detailing to Improve Use of Broad-Spectrum Antibiotics at an Academic Medical Center. Arch intern Med. 2001;161:1897–902.

  46. Avorn J, Soumerai SB. Improving Drug-Therapy Decisions through Educational Outreach, a Randomized Controlled Trial of Academically Based “Detailing”. N Engl J Med. 1983;308:1457–63.

  47. Cals JWL, Ament AJHA, Hood K, Butler CC, Hopstaken RM, Wassink GF, Geert-Jan D. C-reactive protein point of care testing and physician communication skills training for lower respiratory tract infections in general practice: economic evaluation of a cluster randomized trial. J Eval Clin Pract. 2011;17(6):1059–69. 10.1111/jep.2011.17.issue-6. https://doi.org/10.1111/j.1365-2753.2010.01472.x.

  48. Das J, Chowdhury A, Reshmaan H, Banerjee AV. The impact of training informal health care providers in India: A randomized controlled trial Delivering health care to mystery patients Science. 2016;354(6308). https://doi.org/10.1126/science.aaf7384.

  49. Bates DW, Kuperman GJ, Jha A, Teich JM, Orav EJ, Ma’luf N, et al. Does the Computerized Display of Changes Affect Inpatient Ancillary Test Utilization? Arch Intern Med. 1997;157:2501–8.

  50. Bates DW, Kupermann GJ, Rittenberg E, Teich JM, Fiskio J, Ma’luf N, et al. A randomized Trial of a Computer-based Intervention to Reduce Utilization of Redundant Laboratory Tests. Am J Med. 1999;106:144–50.

  51. Daley P, Garcia D, Inayatullah R, Penney C, Boyd S. Modified Reporting of Positive Urine Cultures to Reduce Inappropriate Treatment of Asymptomatic Bacteriuria Among Nonpregnant, Noncatheterized Inpatients: A Randomized Controlled Trial. Infect Control Hosp Epidemiol. 2018;39:814–9. https://doi.org/10.1017/ice.2018.100.

  52. Impact of Providing Fee Data on Laboratory Test Ordering. JAMA Inter Med. 2013;173(10):903. https://doi.org/10.1001/jamainternmed.2013.232.

  53. Shojania KG, Yokoe D, Platt R, Fiskio J, Ma’luf N, Bates DW. Reducing Vancomycin Use Utilizing a Computer Guideline: Results of a Randomized Controlled Trial. J AM Med Inform Assoc. 1998;5:554–62.

  54. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer Predictions of Abnormal Test Results. Effects on Outpatient Testing. JAMA. 1988;259:1194–8.

  55. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing Physicians of the charges for outpatient diagnostic tests. N Engl J Med. 1990;322:1499–504.

  56. Effect of a Social Norm Email Feedback Program on the Unnecessary Prescription of Nimodipine in Ambulatory Care of Older Adults. JAMA Network Open. 2020;3(12):e2027082. https://doi.org/10.1001/jamanetworkopen.2020.27082.

  57. Auleley G-R, Ravaud P, Giraudeau B, Kerboull L, Nizard R, Masssin P, et al. Implementation of the Ottawa Ankle Rules in France. JAMA. 1997;277:1935–9.

  58. Cohen R, Allart FA, Callens A, Menn S, Urbinelli R, Roden A. Evaluation médico-économique d’une intervention éducative pour l’optimisation du treatment des rhinopharyngites aiguës non compliquées de l’enfant en pratique ville. Méd Mal Infect. 2000;30:691–8.

  59. Capitation Combined With Pay-For-Performance Improves Antibiotic Prescribing Practices In Rural China Health Affairs 2014;33(3):502–10. https://doi.org/10.1377/hlthaff.2013.0702.

  60. Tan WJ, Acharyya S, Chew MH, Foo FJ, Chan WH, Wong WK, Ooi LL, Ng JCF, Ong HS. Randomized control trial comparing an Alvarado Score-based management algorithm and current best practice in the evaluation of suspected appendicitis. Abstract World Journal of Emergency Surgery. 2020;15(1). https://doi.org/10.1186/s13017-020-00309-0

  61. Effect of a Price Transparency Intervention in the Electronic Health Record on Clinician Ordering of Inpatient Laboratory Tests. JAMA Intern Med. 2017;177(7):939. https://doi.org/10.1001/jamainternmed.2017.1144.

  62. Chazan B, Ben Zur Turjeman R, Frost Y, Besharat B, Tabenkin H, Stainberg A, et al. Antibiotic Consumption Succesfully Reduced by a Community Intervention Program. IMAJ. 2007;9:16–20.

  63. Nudging Guideline-Concordant Antibiotic Prescribing. JAMA Intern Med. 2014;174(3):425. https://doi.org/10.1001/jamainternmed.2013.14191.

  64. Ray WA, Stein CM, Byrd V, Shorr R, Pichert JW, Gideon P, et al. Educational Program for Physiciand to Reduce Use of Noon-Steroidal Anti-Inflammatory Drugs Among Community-Dwelling Elderly Persons. A Randomized Controllod Trial. Madical Care. 2000;39(5):425–35.

  65. Ilett KF, Johnson S, Greenhill G, Mullen L, Brockis J, Golledge CL, et al. Modification of general practitioner prescribing of antibioitics be use of a therapeutics adviser 8academic detailer). J Clin Pharmacol. 1999;49:168–73.

  66. Danaher PJ, Milazzo NA, Kerr KJ, Lagasse CA, Lane JW. The Antibiotic Support Team. A Successful Educational Approach to Antibiotic Stewardship. Mil Med. 2009;174(2):201.

  67. Le Corvoisier P, Renard V, Roudot-Thoraval F, Cazalens T, Veerabudun K, Canoui-Poitrine F, Montagne O, Attali C. Long-term effects of an educational seminar on antibiotic prescribing by GPs: a randomised controlled trial. Br J Gen Pract. 2013;63(612);e455–64. https://doi.org/10.3399/bjgp13X669176.

  68. Masia M, Matoses C, Padilla S, Murcia A, Sánchez V, Romero I, et al. Limited efficacy of a nonresticted intervention on antimicrobial prescription of commonly used antibiotics in the hospital setting: results of a randomized controlled trail. Eur J Clin Microbiol Infect Dis. 2008;27:597–605. https://doi.org/10.1007/s10096-008-0482-x.

  69. Bernall-Delgado E, Galeote-Mayor M, Pradas-Arnal F, Moreno-Peiró S. Evidence based educational outreach visits: effects on prescriptions of non-steroidal anti-inflammatory drugs. J Epidemiol Community Health 2002;56:653–8.

  70. Cummings KM, Frisof KB, Long MJ, Hrynkiewich G. The Efects of Price Information on Physicians’ Test-Ordering Behavior. Medical Care. 1982;XX(3):293–301.

  71. The effect of patient self-completion agenda forms on prescribing and adherence in general practice: a randomized controlled trial. Fam Pract. 24(1):77–83. https://doi.org/10.1093/fampra/cml057.

  72. Pagaiya N, Garner P. Primary care nurses using guidelines in Thailand: a randomized controlled trial. Tropical Medicine and International Helath. 2005;10(5):471–7.

  73. Pinto D, Heleno B, Rodrigues DS, Papoila AL, Santos I, Caetano PA. Effectiveness of educational outreach visits compared with usual guideline dissemination to improve family physician prescribing—an 18-month open cluster-randomized trial. Implement Sci. 2018;13(1). https://doi.org/10.1186/s13012-018-0810-1.

  74. Ruangkanchanasetr S. Laboratory Investigation Utilization in Pediatric Out-Patient Department Ramathibodi Hospital. Journal of the Medical Association of Thailand = Chotmaihet thangpjaet. 1993;76(Suppl 2):194–208.

  75. Recommending Oral Probiotics to Reduce Winter Antibiotic Prescriptions in People With Asthma: A Pragmatic Randomized Controlled Trial. Ann Fam Med. 2016;14(5):422–30. https://doi.org/10.1370/afm.1970.

  76. Tang Y, Liu C, Zhang X. Public reporting as a prescriptions quality improvement measure in primary care settings in China: variations in effects associated with diagnoses Abstract Scientific Reports. 2016;6(1). https://doi.org/10.1038/srep39361.

  77. Wei X, Zhang Z, Hicks JP, Walley JD, King R, Newell JN, et al. Long-term outcomes of an educational intervention to reduce antibiotic prescribing for childhood upper respiratory tract infections in rural China: Follow-up of a cluster-randomized controlled trial. PLos Med 2019;16(2):e1002733. https://doi.org/10.1371/journal.pmed.1002733.

  78. Public reporting improves antibiotic prescribing for upper respiratory tract infections in primary care: a matched-pair cluster-randomized trial in China. Health Research Policy and Systems. 2014;12(1). https://doi.org/10.1186/1478-4505-12-61.

Download references

Acknowledgements

The authors would like to thank biostatistician Paula Pesonen for advice regarding statistical analysis.

Funding

Open Access funding provided by University of Oulu including Oulu University Hospital. This research is funded by the Strategic Research Council (SRC) (numbers 335288, 336278, 336281). The funding source was not involved in the study design; collection, analysis, and interpretation of data; writing the report; nor in the decision to submit the article for publication.

Author information

Authors and Affiliations

  1. Finnish Coordinating Center for Health Technology Assessment, Oulu University Hospital, University of Oulu, Oulu, Finland

    Petra Falkenbach

  2. Finnish Medical Society Duodecim, Faculty of Medicine, University of Helsinki, Helsinki, Finland

    Aleksi J. Raudasoja

  3. Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht, The Netherlands

    Robin W. M. Vernooij

  4. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands

    Robin W. M. Vernooij

  5. Mehiläinen Oy, Helsinki, Finland

    Jussi M. J. Mustonen

  6. Department of Medicine, Division of General Internal Medicine, McMaster University, Hamilton, ON, Canada

    Arnav Agarwal

  7. Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada

    Arnav Agarwal

  8. Department of Urology, University of Fukui Faculty of Medical Sciences, Fukui, Japan

    Yoshitaka Aoki

  9. Department of General Practice and Elderly Care Medicine, University Medical Centre Groningen, University of Groningen, Groningen, The Netherlands

    Marco H. Blanker

  10. Department of Gynaecology, Chelsea and Westminster NHS Foundation Trust, London, UK

    Rufus Cartwright

  11. Department of Surgery, Division of Urology/Uro-Oncology, School of Medicine, Universidad del Valle, Cali, Colombia

    Herney A. Garcia-Perdomo

  12. Department of Urology, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

    Tuomas P. Kilpeläinen & Kari A. O. Tikkinen

  13. Department of Radiology, Faculty of Medicine and Health Technologies, Imaging Centre, Tampere University Hospital, Tampere University, Tampere, Finland

    Olli Lainiala

  14. Finnish Medical Society Duodecim, Helsinki, Finland

    Tiina Lamberg, Raija Sipilä & Jorma Komulainen

  15. Wellbeing Services County of Pirkanmaa, Unit of Health Sciences, Faculty of Social Sciences, Hatanpää Health Center, Tampere University, Tampere, Finland

    Olli P. O. Nevalainen

  16. Department of Dentistry and Oral Health, Oral Health Care, Institute of Dentistry, Aarhus University, University of Eastern, Kuopio, Finland

    Eero Raittio

  17. Division of Urology, Centre Hospitalier Universitaire de Sherbrooke, Sherbrooke, Canada

    Patrick O. Richard

  18. Departments of Surgery and Health Research Methods Evidence and Impact, McMaster University, Hamilton, Canada

    Philippe D. Violette

  19. Department of Surgery, South Karelian Central Hospital, Lappeenranta, Finland

    Kari A. O. Tikkinen

  20. Oulu University Hospital, University of Oulu, Oulu, Finland

    Miia Turpeinen

Authors
  1. Petra Falkenbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aleksi J. Raudasoja
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robin W. M. Vernooij
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jussi M. J. Mustonen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arnav Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yoshitaka Aoki
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Marco H. Blanker
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rufus Cartwright
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Herney A. Garcia-Perdomo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Tuomas P. Kilpeläinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Olli Lainiala
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Tiina Lamberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Olli P. O. Nevalainen
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Eero Raittio
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Patrick O. Richard
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Philippe D. Violette
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Kari A. O. Tikkinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Raija Sipilä
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Miia Turpeinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Jorma Komulainen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PF, JK, and MT conceived the study. PF, AJR, MT, RS, KAOT, and JK initiated and designed the study plan. AJR, TL, JK, RS, and KAOT performed the search. PF, AJR, RWMV, JMJM, AA, YA, MHB, RC, HAG, TPK, OL, OPON, ER, POR, and PDV independently screened search results and extracted data from eligible studies. PF, AJR, RWMV, JMJM, AA, YA, MHB, RC, HAG, TPK, OL, OPON, ER, POR, and PDV exchanged data extraction results and resolved disagreement though discussion. PF performed the statistical analysis. PF, RS, MT, and JK drafted the manuscript. All authors contributed to the revision. The authors read and approved the final manuscript. MT and JK supervised the study.

Corresponding author

Correspondence to Petra Falkenbach.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

A. J. R. is editor in Finnish Choosing Wisely recommendations. P. O. R. has received honorariums from Janssen, Knight Canada, Bayer, and Tolmar. P. D. V. is the Chair of CUA guideline on thromboprophylaxis and Panel Member of ESAIC guideline on thromboprophylaxis. R. S. is the managing editor, and J. K. is the editor in chief of The Finnish National Current Care Guidelines (Duodecim). The other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

PRISMA-ScR checklist.

Additional file 2.

Search strategy.

Additional file 3.

RoB tool.

Additional file 4.

List of included studies.

Additional file 5.

Characteristics of included studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Falkenbach, P., Raudasoja, A.J., Vernooij, R.W.M. et al. Reporting of costs and economic impacts in randomized trials of de-implementation interventions for low-value care: a systematic scoping review. Implementation Sci 18, 36 (2023). https://doi.org/10.1186/s13012-023-01290-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-023-01290-3

Keywords

Implementation Science

ISSN: 1748-5908