Table 2 Summary of included studies
| Type of Innovation | Implementation Theories, Models and Frameworks | Types of Strategies Evaluated | Design | Sample | Factors Evaluated | Effectiveness of Implementation | |
|---|---|---|---|---|---|---|---|
| Baer et al. 2009 [39] | Motivational Interviewing (MI) |
"Context Tailored Training” (CTT) Characteristics of Clinicians: tailoring the intervention to the specific context. An adaptation of Rollnick et al.’s [40] “context-bound” training. |
CONTEXT Tailoring the intervention to the specific work context vs. 2-day workshop | Randomised trial |
Participants: Gender: female (68%), Ethnicity: Caucasian(81%), Age: 42 years, Education: Bachelor’s degrees or more (68%), Experience: 4.8 years Treatment Setting: United States of America (USA), community-based, National Institute on Drug Abuse (NIDA) |
Primary Outcomes: Fidelity to intervention Adherence to training Predictors of implementation: Clinician characteristics: demographics, perspectives on current work, beliefs about the origin and treatment of addictive behaviours Clinician Evaluation: satisfaction with training Acceptability and appropriateness: Organisational Readiness for Change (ORCA [41];) and Perception of Agency Support |
Primary Outcome: CTT did not improve training outcomes, but mitigating factors found. Predictors of implementation: Clinician Characteristics: Higher education and lower endorsement of disease model beliefs Clinician Evaluation: Modest differences between conditions in satisfaction. Acceptability: Encouraging staff to do new things, higher self-efficacy and greater openness to new techniques |
| Carpenter et al. (2012) [42] | MI | Nil |
TECHNOLOGY SUPERVISION Workshop plus tele-conferencing supervision vs. workshop plus standard tape-based supervision vs. workshop alone | Randomised trial |
Participants: Education: Bachelor’s degree or more (69%), Therapeutic Orientation: Cognitive Behavioural Therapy (CBT) (79%), harm reduction (45%), Alcoholics Anonymous/Narcotics Anonymous (AA/NA) principles (32%), MI (10%), Treatment Setting: USA, community-based, NIDA |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician Characteristics: age, gender, ethnicity, counselling style, verbal and abstract reasoning skills |
Primary Outcome: Clinician characteristics moderated the effect. Predictors of implementation: Clinician Characteristics: Less education, strong vocabulary and low average verbal abstract reasoning |
| Carroll et al. (2006) | MI | Nil |
MULTIPLE Workshop and supervision (randomised to either MI training group or standard intake/ evaluation group) | Randomised trial |
Participants: Gender: female (68%), Ethnicity: Caucasian (81%), Age: 42 years, Education: Bachelor’s degree or more (68%), Experience: 7 years Treatment Setting: USA, community-based, NIDA |
Primary Outcome: Fidelity to the intervention Predictors of implementation: Clinician Characteristics: demographics, experience, counselling orientation, and clinical techniques Clinical Outcomes: Retention Substance use timeline follow back (TLFB) Predictors of clinical outcomes: Characteristics of Patients: demographics, legal system involvement |
Primary Outcomes: Community-based clinicians achieve fidelity when provided training and supervision. Predictors of implementation: No significant findings Clinical Outcomes: MI training group had significantly better retention through the 28-day follow-up than those assigned to the standard intervention. |
| Decker and Martino (2013) [43] | MI | Rogers et al. [44]: individuals are more likely to adopt an intervention after they have an increased knowledge about it and then develop a more favourable attitude towards it. |
MULTIPLE/ LOCAL EXPERT Self-study vs. workshop and supervision, vs. workshop and supervision from program-based trainers | Randomised trial |
Participants: No information of whole sample at baseline Treatment Setting: USA, community-based, NIDA |
Primary Outcome: Fidelity to the intervention Clinician Predictors of implementation: Clinician Characteristics: demographics, experience, treatment allegiance, recovery status, interest, confidence and commitment in using intervention. |
Primary Outcome: No significant differences found. Predictors of implementation: Confidence was associated with increased competence in the use of advanced MI strategies. |
| Garner et al. (2012) [45] | The Adolescent Community Reinforcement Approach (A-CRA) | Nil | FINANCIAL INCENTIVE “Pay for Performance” (P4P) vs. controls | Cluster randomised trial |
Participants: Gender: female (74%), Ethnicity: Caucasian (55%), Age: 36.5 years, Education: Master's Degree or higher (55%), Experience: 6.5 years Treatment Setting USA, community-based, funded by Substance Abuse and Mental Health Services (SAMHSA) |
Primary Outcome: Fidelity to intervention Clinical Outcomes: Remission status Substance use |
Primary Outcome: P4P therapists were significantly more likely to demonstrate A-CRA competence. Clinical Outcomes: Patients in the P4P condition were significantly more likely to receive target A-CRA. No significant differences between conditions with regard to patients' end-of-treatment remission status. |
| Gaume et al. (2014) [46] | Brief motivational intervention (BMI) | Nil | WORKSHOP ONLY vs. controls | Randomised Controlled Trial (RCT) |
Participants: Gender: 'equally distributed', Experience: 8.3 years Treatment Setting: Switzerland, outpatient service, University Hospital |
Predictors of implementation: Fidelity to intervention Clinician Characteristics: demographics, experience, experience in intervention, views of the intervention Self-report of effectiveness in implementing BMI Clinical Outcomes: Substance Use: a drinking composite score, usual drinks per drinking day, and frequency of binge drinking Predictors of Clinical Outcomes: Patient Characteristics: demographics |
Predictors of implementation: Clinician Characteristics: Age and experience - young men with more experienced counsellors had significantly better outcomes than young men having had no intervention. Beliefs - Counsellors viewing themselves as more effective in delivering BMI and having higher belief in BMI efficacy also had clients with better outcomes. Clinical Outcomes: Significant decrease in alcohol use among the BMI group on all three drinking variables. |
| Helseth et al. (2018) [47] | Contingency Management (CM) |
Consolidated Framework for Implementation Research [11] Rogers’ [48]: Diffusion of Innovations theory |
MULTIPLE/ LOCAL EXPERT Treatment as usual (TAU) vs. TAU plus access to a technology transfer specialist plus innovation champion plus role-specific training in the change process ["Science to Service Laboratory" (SSL)] | Controlled before-and-after study |
Participants: Gender: female (68%), Ethnicity: ‘minority’ (23%), Caucasian (77%), Experience: 60% had 3+ years, Education: Bachelor’s degree or more (23%), Treatment Setting: USA, community-based settings |
Primary Outcome: Adoption of intervention Predictors of implementation: Clinician Characteristics: demographics, experience, caseload Clinician Evaluation: Provider Attribute Scale (PAS [49];) Acceptability and appropriateness: ORCA [41] |
Primary Outcome: SSL significantly increased CM adoption. Predictors of implementation: Acceptability and appropriateness: Intervention Characteristic - Compatibility had a negative effect on CM adoption that was attenuated among SSL-providers. |
| Johnson et al. (2002) [50] | Therapeutic community (TC) drug treatment - drug abuse treatment (DAT) services |
"Therapeutic community treatment theory" [51]: devised for the Drug Abuse Treatment Training Experiment. "Program Theory" [52]: Johnson et al. [53] demonstrated how a pro-gram theory can be tested in the substance abuse field. |
BOOSTER TRAINING SESSIONS 6 weeks basic training vs. 8 weeks basic training plus booster sessions - theoretically grounded Managing Organisational Change (MOC) course. | A subject-by-trial split-plot design with repeated measures. Randomised trial |
Participants: No information of whole sample at baseline Treatment Setting: Peru, Drug Abuse Treatment organisations, USA Department of State contract |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician Characteristics: demographics, experience, prior training and exposure to intervention, level of stress, cognitive and affective learning Clinician evaluation: training appraisals, trainer competency, curriculum content, classroom environment, and cultural sensitivity Appropriateness, Penetration: organisational characteristics including TC certification status, description of service Clinical Outcomes: Retention Service System Outcomes: Location, entry criteria, types of services offered, client to staff ratio, staff turnover, record data quality |
Primary Outcomes: The basic training in combination with the MOC increased the magnitude of effects. Predictors of implementation: Clinician Characteristics: some aspects of ‘affective learning’ established and maintained. Clinician Evaluation: nearly all participants gave positive appraisals of the trainers, the training content and methods, the training environment, and the cultural sensitivity. Penetration: DAT training influenced organisational decisions to implement TC methods with fidelity in the booster training session group. Clinical and Service System Outcomes: no significant findings |
| Larson et al. (2013) [54] | Web based CBT course for addiction counsellors named TEACH-CBT (Technology to Enhance Addiction Counselor Helping) | Nil |
TECHNOLOGY Online CBT course vs. training with treatment manual | Randomised trial |
Participants: No information of whole sample at baseline Treatment Setting: USA, Outpatient and residential facilities, NIDA |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician Characteristics: demographics, prior training, exposure to the adoption of new techniques, attitudes towards evidence-based treatments (EBTs), intervention strategies, barriers, and knowledge Feasibility: unit size |
Primary Outcome: Web-course participation did not increase fidelity relative to training with treatment manual Predictors of implementation: Feasibility: Unit size – web course training achieved higher fidelity in larger addiction units and training with a treatment manual achieved higher fidelity in the smaller agencies. |
| Liddle et al. (2010) [55] | Multi-dimensional family therapy (MDFT) | Simpson [56]: systemically-oriented dissemination models, and the evaluation of these efforts in multiple domains, including organisational, clinician and client outcomes. |
CONTEXT Collaboration with staff, administration and patient outcomes (design implies that they were their own controls) | Interrupted time series design |
Participants: Gender: female (80%), Ethnicity: Hispanic (50%), African American (20%), White (20%), Haitian (10%), Education: Bachelor’s and above (70%) Treatment Setting: Florida USA, Adolescent Day Treatment Program, University of Miami Medical School/Jackson Memorial Hospital |
Primary Outcomes: Fidelity to intervention Adherence to intervention approach Predictors of implementation: Penetration: program level changes Community-Oriented Programs Environment Scale [57] Clinical Outcomes: Substance use (TLFB and urine screens) Emotional and Behavioural symptoms (Child Behaviour Checklist and Youth Self Report [58]) |
Primary Outcome: Fidelity to the intervention was obtained following the intervention, and changes were sustained over time. Predictors of implementation: Penetration: Program environment more controlled, more practical and useful approach, clearer expectations, greater autonomy. Clinical Outcomes: Increased abstinence. Reduction in internalising and externalising behaviour. |
| Martino et al. (2008) [59] | Motivational Enhancement Therapy (MET) | Nil |
MULTIPLE/ LOCAL EXPERT Workshop, supervision, local experts vs. counselling as usual | RCT |
Participants: Gender: female (60%), Age: 39 years, Ethnicity: Caucasian (77%), Education: Masters’ degree (43%), Experience: 8.1 years, Treatment Setting: USA, Outpatient (non-methadone), NIDA |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician characteristics: experience, education, and commitment to empirically supported therapies Clinical Outcomes: Change in motivation Substance Use (self-reports TLFB and urine samples) |
Primary Outcome: Community program clinicians can be trained to administer MET with fidelity. Predictors of implementation: No significant findings. Clinical Outcome: Greater fidelity was associated with increases in client motivation and some positive client treatment outcomes. |
| Martino et al. (2011) [60] | MI | Nil |
CONTEXT Train-the-trainer vs. self-study | Randomised trial |
Participants: Gender: female (65%), Ethnicity: Caucasian (83%), Education: Master’s degree (50%) Treatment Setting: USA, Outpatient programs |
Primary Outcome: Fidelity to intervention |
Primary Outcomes: The train-the-trainer group increased fidelity to the intervention at different assessment points comparted to the self-study group. Predictors of implementation: Gains required a substantial amount of training and implementation resources. Clinicians may need more supervision over time. |
| Martino et al. (2016) [61] | MI | Nil |
SUPERVISION A more cost-effective supervision approach – Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency (MIA:STEP) vs. supervision as usual | RCT (hybrid type 2) |
Participants: Gender: female (79%) Age: 41 years, Ethnicity: Caucasian (65%), Hispanic, (20%), African American, (14%), other (1%), Education: Bachelor’s Degrees or more (72%), Experience: 8 years Treatment Setting: USA, Outpatient Programs, non-for-profit |
Primary Outcomes: Fidelity to intervention Supervision integrity Supervision Adherence and Competence Scale Implementation Outcome: Cost of the intervention Clinical Outcomes: Treatment Retention Substance Use (TLFB, breathalysers and urine screening) Treatment utilisation (of alternate services) |
Primary Outcomes: MIA: STEP increased fidelity significantly more than supervision as usual. Supervision delivery and integrity - significantly better MIA: STEP. Implementation Outcome: Cost - MIA: STEP substantially more expensive compared to usual supervisory practices. Clinical Outcomes: similar rates of attendance, program retention, abstinence between groups. |
| Meier et al. (2015) [62] | Integrated Cognitive Behavioural Therapy (ICBT) or Individual Addiction Counselling (IAC). | Nil |
MULTIPLE Manual, workshop, supervision vs. control | RCT |
Participants: Gender: female (82%), Age: 44 years, Ethnicity: Caucasian (100%), Education: Bachelor’s Degree or more (100%), Experience: 7 years Treatment Setting: USA, community outpatient, not-for-profit |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician Characteristics: demographics Clinical Outcomes: Posttraumatic Stress Disorder (PTSD) symptoms (Clinician Administered PTSD Scale [63]) Substance Use (Addiction Severity Index [64]) |
Primary Outcome: Clinicians were able to deliver both therapies with at least adequate fidelity. Predictors of implementation: Clinician Characteristics: Gender - predictive of higher adherence and competence ratings for both ICBT and IAC therapies. Education level - predictive of higher fidelity as session 1 but not session 4. Clinical Outcomes: Fidelity to ICBT at session 4 predicted reductions in alcohol problem severity. Fidelity to IAC at session 4 predicted greater drug severity reductions. |
| Miller et al. (2004) [65] | MI | Nil |
MULTIPLE 2-day Workshop/2-day workshop plus feedback/2-day workshop plus up to 6 individual coaching sessions/2-day workshop, ongoing feedback and up to 6 individual coaching sessions/self-guided | RCT |
Participants: Gender: female (50%), Age: 48 years, Education: Master’s Degree or more (85%), Experience: 11 years, Therapeutic Orientation: CBT (48%), 12-step, (26%), humanistic (22%) Treatment Setting: USA |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician Characteristics: substance use history, self-esteem, attitudes associated with drinking outcomes, temperament |
Primary Outcome: The four trained groups had significantly greater gains in fidelity compared to controls. Predictors of implementation: Sustainability - only feedback and coaching) conditions achieved fidelity at follow-up. |
| Morgenstern et al. (2001) [66] | CBT | Nil |
MULTIPLE Didactic, clinical case training workshops, supervision vs. controls | RCT |
Participants: Gender: female (65%), Age: 42 years, Ethnicity: Caucasian (72%), African American (21%), Hispanic (7%); Education: Master’s Degree or more (45%) Experience: ‘extensive’ Treatment Setting: USA, Outpatient programs |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician Characteristics: demographics, beliefs about the nature of alcoholism and substance abuse treatment Clinician evaluation: satisfaction with training and methods, perceived clinical utility, appraised self-efficacy, ideological conflict |
Primary Outcome: Positive response to the CBT content and format of the training. Predictors of implementation: Clinician evaluation: Satisfaction with the training as a whole, satisfaction with manualised training method, high perceived clinical utility of CBT. Ideological conflict - little evidence of dogmatism or closed-mindedness. |
| Rawson et al. (2013) [67] | CBT | Nil | TECHNOLOGY Distance learning through teleconferencing vs. training and coaching in person vs. controls (manual and - hour orientation) | RCT |
Participants: Gender: female (75%), Age: 38.1 years, Ethnicity: ‘White’ (36%), ‘Black’ (31%), ‘Coloured’ (19%), other (14%), Education: Bachelor’s degree or more (62.3%) Experience: 7 years Treatment Setting: South Africa, outpatient addiction treatment centres |
Primary Outcomes: Fidelity to intervention Knowledge Predictors of implementation: Clinician Characteristics: demographics, training, experience, therapeutic orientation, knowledge, skills in intervention Cost |
Primary Outcome: Significant differences found between groups in knowledge and fidelity. Predictors of implementation: Clinician Characteristics: CBT Knowledge - training and coaching in person brought about a significantly greater gain in CBT knowledge. CBT Fidelity - the distance learning and training and coaching in person groups had significantly better skills. Training and coaching in person achieved a higher level of fidelity overall. Cost Comparison: The training and coaching in person condition was most expensive followed by the distance learning and control conditions. |
| Smith et al. (2012) [68] | MI | Nil |
TECHNOLOGY Tele-conferencing supervision (TCS) plus workshop vs. standard tape-based supervision plus workshop vs. workshop alone | RCT |
Participants: Gender: female (65%), Age: 44 years, Ethnicity: African American (40%), Caucasian (29%), Latino (26%), other (5%), Education: Bachelor’s degree or more (71%), Treatment Setting: USA, community-based, NIDA |
Primary Outcome: Fidelity to intervention Predictors of implementation: Clinician Characteristics: demographics, treatment clinic, years in the field, years in current position |
Primary Outcome: TCS plus workshop training increased fidelity, but supervision methods need improvement. Predictors of implementation: Overall, the findings support the importance of providing feedback and supervision after workshop training to improve fidelity, which could potentially be achieved through a TCS format. |
| Weingardt et al. (2006) [69] | CBT | Nil |
TECHNOLOGY Web-based training vs. face-to-face training workshop with identical content vs. delayed training controls | RCT |
Participants: Gender: female (55%), Age: 44 years, Ethnicity: Caucasian (56%), African American (21%), Latino (12%), other (10%), Education: Bachelor’s or more (81%), Experience: 7 years Treatment Setting: USA, counsellor outpatient |
Primary Outcome: Knowledge Predictors of implementation: Clinician Characteristics: experience, education, familiarity with intervention at baseline |
Primary Outcome: Clinicians in both the web-based technology (WBT) and face-to-face workshop conditions showed significant improvement in knowledge compared to clinicians in the delayed training control condition. Predictors of implementation: No significant findings. |
| Weingardt et al. (2009) [70] | CBT | Nil |
TECHNOLOGY Use of web conferencing. Online modules on CBT and group supervision sessions via web conferencing | Randomised trial (randomised to either strong or weak adherence expectations) |
Participants: Gender: female (62%), Age: 47 years, Ethnicity: Caucasian (64%), Education: Bachelor’s degree or more (68%), Treatment Setting: USA, counsellor outpatient |
Primary Outcome: Knowledge Self-Efficacy Predictors of implementation: Clinician Characteristics: demographics, SUD recovery, familiarity with intervention, work setting, job Burnout |
Primary Outcome: Statistically and clinically significant differences in knowledge and self-efficacy were obtained for the web-conferencing group. Predictors of implementation: No significant findings. |