|Design element||Operational considerations||Researcher considerations|
|Sites and population|
|The BHIP operational initiative has already begun.||
Need for results to inform continuing process.|
Can capitalize on momentum of the system to engage and motivate sites, promulgate best-practice models.
Helps to sell the project to facilities.|
Increases likelihood of incorporation into sustainable practice.
The stepped-wedge design can assess secular trends.
|Identifying the population of facilities to target||Slower-to-adopt facilities are of concern.||Working with this population avoids ceiling effects (high performers) and insufficient commitment to change (laggards).|
|Site recruiting via operational structures||Hierarchical communications and reporting structure enhance facility identification and endorsement of program.||Provides access beyond “usual suspect” volunteer facilities and “friends of friends” facilities to enhance external validity.|
|Intervention and design|
|Need for all participating sites to receive implementation support||
Harder to justify the project on policy level if not all sites receive support.|
Can be a site recruiting tool.
Stepped wedge can accommodate this, though analysis is more complicated than traditional parallel-groups design|
Additionally, stepped wedge can enhance formative evaluation and evaluate secular trends.
|Balance in randomization||Experience-based expertise contributes identifying characteristics relevant to success.||Sophisticated statistical expertise provides site alancing techniques.|
|Control condition||Sites seek as much active support as possible, as soon as possible.||Researchers develop a credible contrast condition by which to evaluate the impact of the implementation strategy.|
|Length of implementation support||Experience-based expertise suggests one year of support needed.||Pilot data agree, but the need for timely data provision requires steps in wedge of 4 rather than 12 months.|
|Need to work with existing VAMC staff without external research-funded support besides external facilitators||Resource limitations preclude deploying additional clinical or administrative staff (limitation of both OMHO and QUERI funding).||
Makes sustainability more likely.|
Provides distinct scientific contribution enhancing effectiveness data beyond that from more traditional CCM clinical trials to date.
|Delineating the interface between quality improvement program evaluation and research||
The BHIP initiative is nationwide in scope and facility participation is not optional.|
However, a facility’s participation in this implementation project is optional.
Medical center participation in the project is the decision of the medical center director and mental health leadership, not individual provider.|
However, providers can choose not to participate in qualitative interviews.
Patients can choose not to participate in health status and perception of care assessments.
|Use of videoconference and telephone as main modalities for external facilitation||Budget (OMHO or QUERI) will not support frequent site visits by external facilitators.||Provides greater likelihood of spread of intervention strategy if successful.|
|Outcome assessment and analysis|
|Identification of outcome domains and appropriate instruments||Program fidelity measures must be streamlined and targeted, and wherever possible benchmarked against national data.||Patient-level measures must be psychometrically valid and feasible in a heterogeneous patient population.|
|Both quality and health status impacts are important||Operational priority issues are (a) whether CCM can be implemented into BHIP teams and (b) whether CCM-guided BHIP teams have impact on the target population.||Hybrid type III designs can accommodate implementation outcomes and health status outcomes.|
|Data must both be scientifically valid and reported in a time frame useful to operational partners.||Three-year outcomes can help plan strategy for next initiatives, but are too late to make tactical improvements to this phase of BHIP roll-out.||Design and analysis accommodate “early looks” at the data on semi-annual basis, using adjustment of significance testing parameters.|
|Ethical and regulatory issues||A non-voluntary national initiative receives expert support from researchers in order to optimize their roll-out based on valid empirical data.||Researchers gather a broader range of data to answer relevant research questions from voluntary subjects.|