Table 1 Potential considerations in prioritizing the testing of unproven medical practices

Priority should be given to practices where the present evidence base is weakest.

For instance, a tiered system may be utilized: Level 1 (Weak) Randomized trials of interventions claiming subjective benefits, that are unblinded or fail to use proper controls. 2 (Weaker) Historically controlled studies of interventions that purport survival benefits, case series documenting improvements in subjective endpoints and quasi-experimental studies. 3 (Weakest) Practices based on pathophysiology and expert opinion alone. In many cases, professional conflicts may also prove problematic; thus, it may be reasonable to pursue this technique using content-specific experts in strictly an advisory capacity

Priority should be given to interventions with significant net financial burden on health payers.

For instance, orthopedic procedures for chronic back and joint pain, including knee and hip replacement surgeries are widely utilized in the United States, incur large financial burden on payers, but have little evidence of sustained long term benefits.

Priority should be given to practices for which there are several alternative options, particularly if alternatives are of completely different mechanisms (thus unlikely to also be overturned), or of low cost or bolstered by stronger evidence.

For instance, consider the market for anti-rheumatologic agents. Maintenance treatment of rheumatoid arthritis (RA) with disease modifying agents (DMARDS) has historically relied upon oral anti-immunologic agents such as methotrexate, azathiaprine, cyclosporin, and hydroxychloroquine. Recent years have witnessed a boom in novel drugs, typically expensive monoclonal antibodies against circulating cytokines or cell surface receptors. To date, this market has been limited by paucity of head to head trials, and, of trials that have been conducted, the majority are industry-sponsored studies. Collectively, there remains clinical uncertainty about how best to use these agents [20].

Priority should be given to test practices where the harms are well documented and confer substantial morbidity.

For instance, there is growing awareness of strut fracturization, embolism, and migration of IVC filters. At the same time, the IVC filter has never shown to improve any patient-centered outcome for any patient population in a prospective trial, and traces its approval through the FDAs 510 k mechanism [21].

Priority should be given to test practices where the cost to test is far less than ongoing expenditures of the practice.

In some respects, trialists should think like CEOs, weighing the costs of conducting a study, which may find a practice ineffective versus the ongoing expenditures for that practice. At times, such calculations may favor costly trials where the existing evidence base is weak, observational studies suggest inefficiencies, and the ongoing costs are large [22]. At other times, small trials that eliminate boutique practices may be employed [23]. Whose financial bottom line is being affected is important to consider. For that reason, nonconflicted bodies should make these determinations, utilizing investigators without financial conflicts of interest.

Priority should be given to test practices where negative results may truly gain traction.

Some specialties (primary care providers) may be more ready to abandon contradicted medical practices, and it is reasonable to test practices when there is genuine belief that contradiction can gain traction. Furthermore, some practices may be cumbersome (tight glycemic control in the ICU), time-consuming (routine gown and glove precautions) or unpleasant, and their contradiction may also be palatable. Finally, as payment structures shift from fee for service towards bundles [24], costly components may lose faithful disciples. Other fields, those with numerous and hyperbolic third party advocates, have been notoriously unwilling to trust results that undermine their worldview, no matter how robust the science.

Priority should be based on the expected value of funding a specific study that may inform de-implementation, at the size and cost proposed.

Value of information (VOI) offers a decision-making framework that tries to capture several of the above issues, at least the ones that can be best quantified [19]. VOI can be used to prioritize and power clinical trials taking into account the costs of increasing study sample size, the potential number of persons affected by changes in that practice, the costs of the practice, including downstream costs, and the increased knowledge of marginal changes in health outcomes that may result from testing — converting all to the final common denominator of cost per favorable outcome gained.