Skip to main content

Table 2 Strategies to improve mental health of children and adolescents: summary table

From: Quality improvement, implementation, and dissemination strategies to improve mental health care for children and adolescents: a systematic review

Strategy, study designs, N Target condition and ages of youth Comparisons Component of the strategy Major findings Strength of evidence from results Reasons for strength of evidence
Training therapists to implement an EBP
Beidas et al. [50]
Cluster RCT, 115 therapists
Anxiety
Ages 8–17 years
Augmented active learning vs. routine professional training workshop Educational meetings or materials No differences between arms for practitioner satisfaction with approach, protocol adherence, or practitioner skill Low for no benefit for practitioner satisfaction, adherence, and skill Low risk of bias, small sample size, imprecise results
Computerized routine training vs. routine professional training workshop Educational meetings or materials No differences between arms for practitioner protocol adherence or program model fidelity, or skill; computerized training group practitioners less satisfied than routine training group practitioners Low for no benefit for practitioner satisfaction, adherence, and skill Low risk of bias, small sample size, imprecise results
Feedback of patient symptoms to practitioners
Bickman et al. [13]
Cluster RCT, N of clinicians unclear,
340 youth, 144 clinicians, 383 caregivers
General mental health problem (children who receive home-based mental health treatment)
Mean age = 15 years
Weekly and cumulative 90-day feedback vs. cumulative 90-day feedback only on patient symptoms and functioning to practitioners Audit and feedback Two thirds of practitioners did not view Web module Insufficient for practitioner adherence High study limitations, unknown precision for adherence
Membership in the weekly feedback group increased the rate of decline in functional severity scale by 0.01 (range: 1 to 5, higher scores indicate greater severity) Low for benefit for functional severity High study limitations, precise results for symptoms
Feedback of patient treatment progress (symptoms and functioning) and process (e.g., therapeutic alliance) to practitioners
Bickman et al. [61]
Randomized block RCT, N of clinicians unclear,
257 youth, 2 clinics (one urban, one rural) at a single agency
21 clinicians, 255 caregivers
General mental health problem (children who receive mental health treatment from a community mental health clinic)
New patients aged 11–18
Session-by-session feedback vs. cumulative 6-month feedback to clinicians Audit and feedback No significant differences in percentage of sessions held or percentage of clinicians, youth, or caregivers who completed the questionnaire required at each visit Insufficient for patient engagement, for practitioner adherence/program model fidelity, and system uptake High study limitations, unknown precision for each intermediate outcome.
No patient-reported, caregiver-reported, or clinician-reported differences in symptoms or functioning of youth associated with intervention group in either clinic except feedback effects only seen in clinician ratings from one clinic (beta feedback*slope = −0.01, p = 0.045) Low for no benefit for symptom severity High study limitations, precise results for symptoms
Computer decision support for guidelines
Carroll et al. [46]
Cluster RCT, 84 patients
General mental health problem (children who receive home-based mental health treatment)
Mean age = 15 years
Computer decision support plus electronic health record (EHR) that included diagnosis and treatment guidelines vs. computer decision support plus EHR only Educational meetings or materials
Patient-reported data
Reminders
Quality monitoring
Practitioner adherence improved through uptake of guidelines for diagnostic assessment (aOR, 8.0; 95% CI, 1.6 to 40.6); more reporting of 3 of 4 symptom domains at diagnosis Low for benefit for practitioner adherence and program model fidelity Medium study limitations, imprecise results with small number of events, large magnitude of effect
No statistically significant differences on practitioner adherence through reassessment of symptoms at 3 months, adjustment of medications, and mental health referral Insufficient for practitioner adherence (reassessment of symptoms) at 3 months, adjustment of medications, and referral Medium study limitations, imprecise results (CIs cross the line of no difference)
Visit to a mental health specialist calculated OR 2.195; 95% CI, 0.909 to 5.303; p = 0.081; reported p value in study = 0.054 Insufficient for service utilization Medium study limitations, imprecise results (CIs cross the line of no difference)
Internet portal to provide access to practice guidelines
Epstein et al. [56]
Cluster RCT, 746 patients
Attention deficit hyperactivity disorder (ADHD)
Ages 6 to 12 years
Internet portal providing practitioner access to practice guidelines vs. wait-list control Educational meetings or materials
Patient-reported data
Audit and feedback
Reminders
Quality monitoring
Strategy appeared to improve 4 of 5 examined outcomes that measured practitioner protocol adherence and program model fidelity outcomes (mean change in proportion of patients who received targeted, evidence-based ADHD care outcomes between groups ranged from 16.6 to −50), but estimates were very imprecise, with large CIs Low for benefit for practitioner protocol adherence and program model fidelity Medium study limitations, imprecise (wide CIs)
Collaborative consultation treatment service to implement quality measures
Epstein et al. [47]
Cluster RCT, 38 practitioners, 144 patients
ADHD
Mean age = 7 years
Collaborative consultation treatment service to promote the use of titration trials and periodic monitoring during medication management vs. control Audit and feedback
Multidisciplinary team
Practitioner adherence/ fidelity as measured by use of titration trials β = −0.283; SE, 0.09; p < 0.01 and by use of medication monitoring trials:
p = NS, details NR
Insufficient for practitioner adherence and fidelity High study limitations, imprecise results (small sample size)
Lower odds with overlapping confidence intervals of practitioner citing obstacles to implementation of EBP in 6 of 8 measures (2 reached statistical significance) Insufficient for practitioner competence/ skills High study limitations, imprecise results (small sample size)
F score for decrease in combined parent and teacher ratings of ADHD symptoms for group x time interaction: F 2, 144 = 0.44, p = 0.65 Insufficient for patient change in mental health symptoms High study limitations, imprecise results (small sample size)
Paying practitioners to implement an EBP
Garner et al. [53]
Cluster RCT, 105 therapists, 986 patients
Substance use disorders
Mean age = 16 years
Paying practitioners for performance in successfully delivering an EBP intervention vs. implementation as usual Provider incentives Therapists in the P4P group were over twice as likely to demonstrate implementation competence compared with IAU therapists (Event Rate Ratio, 2.24; 95% CI, 1.12 to 4.48) Moderate for benefit for practitioner competence Medium study limitations, precise results
Patients in the P4P condition were more than 5 times as likely to meet target implementation standards (i.e., to receive specific numbers of treatment procedures and sessions) than IAU patients (OR, 5.19; 95% CI, 1.53 to 17.62) Low for benefit for practitioner adherence and program fidelity Medium study limitations, imprecise results (wide CIs)
No statistically significant differences between groups OR, 0.68; 95% CI, 0.35 to 1.33 Low for no benefit for patient change in mental health symptoms Medium study limitations, precise results
Program to improve organizational climate and culture
Glisson et al. [14]a
Two-stage RCT,
596 youth, 257 therapists
Externalizing behaviors (youth referred to juvenile court with behavioral or psychiatric symptoms that require intervention)
Ages 9–17 years
Program to improve organizational climate and culture vs. usual care Educational meetings or materials
Educational outreach visits
Provider satisfaction initiative
Audit and feedback
Details NR but does not demonstrate improvements in any measure of adherence by strategy group for any ARC vs. no ARC comparison Low for no benefit for practitioner adherence Medium study limitations, precise results
Difference in out-of-home placements and child behavior problem scores at 18 months between ARC-only and usual-care groups did not meet statistical significance (p = 0.05). Low for no benefit for patient change in mental health symptoms at 18 months Medium study limitations, precise results (small sample size), CIs likely overlap
Program to improve organizational climate and culture
Glisson et al. [51, 60]
Cluster RCT
352 caregivers of youth ages 5–18 in 18 programs
General mental health problems
Ages 8–24 years
Program to improve organizational climate and culture vs. usual care Educational meetings or materials
Educational outreach visits
Provider satisfaction initiative
Audit and feedback
Trends toward improvement in all domains; nonoverlapping CI for some domains showing significant improvements (p < 0.05) for ARC group vs. usual care Low for benefit for practitioner satisfaction Medium study limitations, imprecise results (small study sample)
Lower problem behavior scores for youth in the ARC group compared with those in the control group during first 6 months of follow-up (following 18-month organizational implementation), effect size = 0.29 Low for benefit for patient change in mental health symptoms Medium study limitations, imprecise results (small study sample)
Nurse training to implement an EBP
Gully et al. [55]
Interrupted time series in Study 1,
172 parents or caregivers; RCT in Study 2,
51 parents or caregivers
General mental health symptoms (children suspected of abuse during forensic medical examinations)
Ages 2–17 years
Protocol to train nurses to educate parents about EBPs vs. typical services Educational meetings or materials
Educational outreach visits
Patient-reported data
Strategy improved parent ratings of access to care (mean difference between groups ranged from 0.08 to 2.1 points in Study 1 and 0.6 to 1.9 in Study 2) (scale = 1–5) Low for benefit for patient access to care High risk of bias, consistent, direct, precise results
     Improved parent ratings of satisfaction of care by a mean of 0.4 in Study 1 and 0.9 in Study 2 (scale = 1–5) Low for benefit for patient satisfaction High risk of bias, consistent, direct, precise results
     Improved parent ratings of treatment engagement by a mean of 0.9 in Study 1 and 2.5 in Study 2
(scale = 1–5)
Low for benefit for treatment engagement High risk of bias, consistent, direct, precise results
     Improved parent ratings of therapeutic alliance by a mean of 0.4 in Study 1 and 0.9 in Study 2
(scale = 1–5)
Low for benefit for therapeutic alliance High risk of bias, consistent, direct, precise results
Intensive quality assurance to implement an EBP
Henggeler et al. [54]
Controlled clinical trial, 30 practitioners, N of caregiver and patient reports and monthly data points NR
Substance use disorders (adolescents with marijuana abuse)
Ages 12–17 years
Intensive Quality Assurance (IQA) system vs. workshop only to implement an EBP intervention Quality monitoring Study does not provide sufficient detail to judge magnitude of effect on practitioner adherence to cognitive behavioral therapy and monitoring techniques Insufficient for practitioner adherence and fidelity High study limitations, imprecise results
Training through workshop and resources to implement an EBP
Henggeler et al. [59]
Cluster RCT; 161 therapists
Substance use disorders
Ages 12–17 years
Workshop and resources (WSR) vs. WSR and computer-assisted training (WSR + CAT) to implement an EBP intervention Educational meetings or materials No statistically significant difference between groups for use, knowledge, and adherence Insufficient for additional benefit of WSR + CAT vs. WSR comparison group for practitioner use, knowledge, and adherence Medium study limitations, imprecise, small sample sizes, cannot determine whether CIs cross line of no difference
   WSR vs. WSR + CAT
and supervisory support (WSR + CAT + SS) to implement an EBP intervention
Educational meetings or materials
Educational outreach visits
No statistically significant difference between groups for use, knowledge, and adherence Insufficient for additional benefit of WSR + CAT + SS vs. WSR comparison group on practitioner use, knowledge, and adherence competence/skills Medium study limitations, imprecise, small sample sizes, cannot determine if CIs cross line of no difference
Professional training to identify and refer cases
Lester et al. [48]
Cluster RCT;
110 practices, 179 patients
Psychosis (adolescents and adults with first-episode psychosis)
Ages 14–30 years
Professional training to identify and refer cases vs. usual care Educational meetings or materials
Educational outreach visits
Relative risk (RR) of referral to early intervention after first contact: 1.20, 95% CI, 0.74 to 1.95, p = 0.48 Insufficient for patient access to care High study limitations, imprecise results
     No statistically significant differences between groups in changes in patient mental health status Insufficient for patient change in mental health symptoms High study limitations, imprecise results
     Patients in the professional training group averaged 223.8 fewer days for time from the first decision to seek care to the point of referral to an early intervention service than patients in the control group Low for benefit for service utilization High study limitations, imprecise results
     No adverse events were reported, no significant between-group differences for false-positive referral rates from primary care Insufficient for patient harms High study limitations, unknown precision
Professional training plus feedback
Lochman et al. [57]
Cluster RCT, 511 patients
Externalizing behaviors (children at risk for aggressive behaviors)
Ages: third-grade students
Professional training plus feedback (CP-TF) to implement an EBP intervention vs. control Educational meetings or materials
Audit and feedback
Students in CP-TF group had fewer behavioral problems as rated by teachers (beta = −0.41, SE = 0.16, p = 0.01) than controls but no significant difference in teacher ratings or parent ratings Low for no benefit for changes in mental health status Medium study limitations, precise results
     Students in CP-TF group had fewer minor assaults (e.g., hitting or threatening to hit a parent, school staff, or student) as reported by the child (beta = −0.25, SE = 0.12, p = 0.03) and social/academic competence as reported by the teacher (beta = 0.35, SE = 0.13, p = 0.01) compared with controls Low for benefit for change in socialization skills and behaviors Medium study limitations, precise results
   Professional training only to implement an EBP intervention (CF-BT) vs. control Educational meetings or materials No significant difference in behavioral problems as rated by teachers or parents or student-reported assaults between CP-BT and control groups Low for no benefit for changes in mental health status Medium study limitations, precise results
     No significant differences in social/ academic competence as reported by the teacher, nor were any significant differences found between groups on social skills as rated by parents. Low for no benefit for change in socialization skills and behaviors Medium study limitations, precise results
Medication monitoring therapy
Ronsley et al., 2012 [49]
Interrupted time series
Health care practitioners for 2376 patients
Psychosis
Ages <19 years (mean age = 11)
Patient medication monitoring training program for practitioners vs. usual care Educational meetings or materials
Educational outreach visits
Reminders
38.3% of patients had a metabolic monitoring and documentation tool (MMT) in the charts after program implementation; drop in the prevalence of second-generation antipsychotic prescribing from 15.4% in the pre-metabolic monitoring training program (MMTP) period to 6.4% in the post-MMTP period (p < 0.001) Low for benefit for practitioner adherence High study limitations, precise outcomes
     Increased metabolic monitoring over time (level of change varied by type of monitoring) Low for benefit for patient service utilization High study limitations, precise outcomes
Staffing models to implement an EBP to screen, conduct a brief intervention, and refer adolescents with substance use to treatment from primary care settings
Sterling et al. [58]
Cluster RCT, 47 pediatricians with 1871 eligible patients
Varied conditions among children attending a pediatric primary care office
Ages 12–18
Pediatrician only vs. embedded behavioral health care practitioner (BHCP) implementation of an EBP Multidisciplinary teams No significant differences in substance use assessment between study arms (aOR, 0.93; 95% CI, 0.72 to 1.21); patients in the embedded BHCP group more likely than those in the pediatrician-only group to receive brief intervention (aOR = 1.74, 95% CI, 1.31 to 2.31); patients in the BHCP group less likely to receive a referral to a specialist than patients in the primary-careb only group (aOR = 0.58, 95% CI, 0.43 to 0.78) Low for no benefit for practitioner adherence (2 of 3 adherence outcomes were statistically significant) Medium study limitations, unable to assess precision
Co-location of a behavioral health EBP parenting program in primary care to help children with externalizing behavioral problems
Wildman et al. [52]
Controlled clinical trial,
4 pediatric practices, 20,917 children with primary care visit
Externalizing behavior problems
Ages 2–12 years
Colocation of a behavioral health EBP parenting program in primary care vs. enhanced referral to a behavioral health EBP parenting program in a location external to the practice. Changing the scope of benefits OR for attending first EBP visit, 3.10; 95% CI, 1.63 to 5.89 Low for benefit for patient access to care High study limitations, precise results
     No improvement in mean number of sessions attended (calculated mean difference: −1.01; 95% CI, −2.60 to 0.58) Insufficient for patient service utilization High study limitations, precise results
Implementation of a school-based cognitive-behavioral group EBP
Warner et al. [62]
Stratified RCT
138 youth, 7 master’s level school counselors, 5 doctoral-level psychologists
Social anxiety disorder.
Adolescents in grades 9–11 from three suburban public high schools identified via school-wide screening, parent telephone screening, and clinical diagnostic evaluation with no other mental disorders of equal or greater severity.
Implementation by a school counselor vs. by a psychologist Changing provider No significant differences in implementation adherence or competence. Insufficient for practitioner adherence or competence High study limitations, unknown precision for each intermediate outcome
     No significant differences between groups for any of the severity or functioning scales at post-treatment or follow-up with the exception of 3 posttreatment outcomes (treatment response, treatment remission and social anxiety severity as rated by parents) where youth in the school counselors group did not do as well as those in the psychologist group when noninferiority was tested Insufficient for patient change in mental health status High study limitations, unknown precision for each intermediate outcome
  1. aFour study groups were examined: ARC + MST, ARC only, MST only, and usual care. Comparisons were ARC only vs. usual care or any ARC (combined ARC + MST and ARC only) vs. no ARC (combined MST and usual care), as noted
  2. bFewer referrals seen as improvement because this outcome indicates that the practitioner was able to give brief intervention without referral to behavioral health specialists
  3. ADHD attention deficit hyperactivity disorder, aOR adjusted odds ratio, ARC Availability, Responsiveness, and Continuity, CBT cognitive behavioral therapy, CI confidence interval, CP-TF Coping Power training plus feedback, EBP evidence-based practice, EHR electronic health record, IAU implementation as usual, IQA Intensive Quality Assurance, MMT metabolic monitoring program, MMTP metabolic monitoring training program, MST multisystemic therapy, N number, NR not reported, NS not significant, OR odds ratio, p probability, P4P pay for performance, RCT randomized controlled trial, RR relative risk, SE standard error, WSR workshop plus resources, WSR + CAT workshop plus resources plus computer-assisted training, WSR + CAT + SS workshop plus resources plus computer-assisted training plus supervisory support