Table 2 Assessment measures across timepoints and participants
Construct | Measure | Brief description | Timepoints | Participant(s) | Method |
|---|---|---|---|---|---|
Training outputs | |||||
Knowledge | |||||
The PCIT Coaches Quiz [96] | 22-items to assess clinicians’ knowledge of PCIT concepts and coaching scenarios; mixed question format of multiple choice and short answer | Baseline | Supervisors | Self-report | |
12 months | Clinicians | ||||
24 months | |||||
Skill | |||||
PCIT Therapist Competency Checklist | 17-item checklist of PCIT competency criteria based on established PCIT Training Guidelines | Ongoing | Completed by the trainer on clinicians | Live Behavior observation | |
FIRST Coach Coding System [97] | Assesses the quality and style of a therapist’s coaching | Ongoing | Completed by the trainer on clinicians | Video review behavior observation | |
Supervisor Impressions of Competence in PCIT [98] | Supervision structure, practice, content, and skill | 6 months | Completed by the supervisors on clinicians | Supervisor Report of clinician behavior | |
12 months | |||||
24 months | |||||
Attitude | |||||
Training Satisfaction [99] | Satisfaction with on-site training materials, content, and trainer | During each on-site training | Supervisors | Self-report | |
clinicians | |||||
Treatment Satisfaction | Satisfaction with the treatment, concerns or barriers to using PCIT, and suggestion for improving its relevance | 6 months | Supervisors | Self-report | |
12 months | Clinicians | ||||
24 months | |||||
Usage Rating Profile—Intervention [100] | 35-item measure of intervention usage, including acceptability, understanding, feasibility, and system support | 6 months | Supervisors | Self-report | |
12 months | Clinicians | ||||
24 months | |||||
PCIT Learning Experience Feedback Form | 8-item survey to assess satisfaction with PCIT training | 6 months | Administrators | Self-report | |
12 months | |||||
24 months | |||||
Training inputs | |||||
Trainee characteristics | |||||
Demographic Information Form | Basic demographic and contact information | Baseline | Administrators | Self-report | |
Supervisors Clinicians | |||||
Treatment Provider and Practices Survey [101] | Demographic information and variables hypothesized to affect adoption of EBTs | Baseline | Supervisors Clinicians | Self-report | |
Training design | |||||
Training Fidelity Checklist | Checklist to ensure that trainers implement the LC and CM conditions in the manner intended (accuracy, consistency) | During each on-site training | Research staff member | Behavior observation/ video review | |
Training dose | |||||
Consultation Records | Standardized form completed by the trainer for contact with trainees to document key topics discussed | During each consult call | Completed by the trainer on clinicians | Trainer report | |
PCIT Feedback Form | Clinical open-ended feedback provided to clinicians on clinical competency | Ongoing | Completed by the trainer on clinicians | Behavior observation/ video review | |
Organizational/work environment | |||||
Infrastructure Survey of Children’s Mental Health [102] | Structured, 1-hour, interview survey to obtain information on the governance, financing, staffing, services, and implementation practices of organizations as well as their perspectives on factors important to the implementation of new treatment and services | Baseline | Administrators | Interview (qualitative and quantitative) | |
12 months | |||||
24 months | |||||
Dimensions of Organizational Readiness—Revised [103] | 21-item scale that requires respondents to rate the importance of specific intra- and extra- organizational variables (e.g., leadership support, fiscal benefits, match, admin burden) likely to affect implementation | Baseline | Administrators | Self-report | |
12 months | |||||
24 months | |||||
TCU Organizational Readiness for Change Survey [104] | Assessment of organizational functioning including program needs, training needs, and pressures for change, program resources, and organizational dynamics | Baseline | Administrators | Self-report | |
12 months | Supervisors | ||||
24 months | |||||
TCU Survey of Organizational Functioning [104] | Includes the TCU Organizational Readiness for Change plus nine additional scales assessing job attitudes and workplace practices | Baseline | Administrators | Self-report | |
12 months | Supervisors | ||||
24 months | |||||
Implementation outcomes | |||||
Treatment process and acceptability | |||||
Treatment Implementation Feedback Form [105] | 13-item questionnaire to understand clinicians’ experiences using an EBT including the extent of use, relevance, and helpfulness of the treatment | 6 months | Supervisors | Self-report | |
12 months | Clinicians | ||||
24 months | |||||
Treatment Summary Report [106] | 15-item questionnaire to assess nature and outcome of treatment services for a particular family (e.g., service length, barriers, disposition) to provide acceptability data for a specific family | Ongoing as families discharge from treatment | Supervisors | Clinician report and chart review | |
Clinicians report on child and family behavior | |||||
Fidelity | |||||
PCIT Session Fidelity Checklist [107] | Detailed checklists of components of each PCIT session to measure fidelity | Ongoing through video review | Completed by the trainer on clinicians | Behavior observation/ video review | |
Use of Treatment Practices [108] | 15-item clinician self-report of completion of specific treatment tasks with an individual family | Ongoing as families discharge from treatment | Supervisors | Self-report | |
Clinicians | |||||
Cost | |||||
Chief Financial Officer Survey (updated version of a survey created by Olmstead et al. [109]) | Provides information necessary to complete the cost analysis (e.g., program overhead and fringe rates, annual expenditures | Baseline | Administrators | Self-report | |
Adoption | Time-tracking phone app | Provides time estimates of trainer activity by condition | Ongoing | Trainers | Self-report |
Evidence-Based Practice Survey [110] | List of evidence-based practices respondent has implemented in the last year | Baseline | Administrators | Self-report | |
12 months | Supervisors | ||||
24 months | Clinicians | ||||
Penetration | |||||
PCIT Learning Experience Feedback Form | Report of consumer levels (e.g., the number of consumers who receive PCIT divided by the number of consumers who were eligible to receive PCIT and did not receive it | 6 months | Administrators | Self-report | |
12 months | |||||
24 months | |||||
Sustainability | |||||
Evidence-based Practice Sustainability Telephone Interview | Survey of current PCIT delivery, penetration, practice adaption, and barriers to sustaining PCIT practice within the organization | 24 months | Administrators | Self-report | |
Family and child outcomes | |||||
Family characteristics | |||||
Demographic Information | Measure to gather information including age, race, gender, family constellation, living arrangement, school placement, and socio-economic status. | Baseline | Families | Self-report | |
Child focused | s | ||||
Eyberg Child Behavior Inventory (ECBI) [111] | 36-item parent report of child conduct problems | Baseline | Families | Self-report | |
3 months | |||||
6 months | |||||
12 months | |||||
Vanderbilt Assessment Scale Parent Version [112] | 47-item assessment of core symptoms of DSM-IV diagnostic criteria (related to inattention, hyperactivity, oppositional behaviors, conduct problems, and anxiety depression) and impairment in performance | Baseline | Families | Self-report | |
3 months | |||||
6 months | |||||
12 months | |||||
Dadds CU Scale [113] | 20-item rating scale to assess psychopathic traits (narcissism, impulsivity, callousness-unemotionality) in children ages 6-13 | Baseline | Families | Self-report | |
3 months | |||||
6 months | |||||
12 months | |||||
Parent focused | |||||
Public Health Questionnaire—9 [114] | 10-item self-report to assess and monitor symptoms consistent with Major Depressive Disorder | Baseline | Families | Self-report | |
3 months | |||||
6 months | |||||
12 months | |||||
Generalized Anxiety Disorder Scale—7 [115] | 7-item self-report scale to screen and monitor symptoms consistent with Generalized Anxiety Disorder | Baseline | Families | Self-report | |
3 months | |||||
6 months | |||||
12 months | |||||
Alabama Parenting Questionnaire [116] | Assessment of parenting practices yielding three subscales (Positive Parenting, Inconsistent Discipline, and Poor Supervision) | Baseline | Families | Self-report | |
3 months | |||||
6 months | |||||
12 months | |||||
Treatment participation | |||||
Barriers to Treatment Participation Scale [117] | 44-item parent report of potential barriers to treatment participation | 3 months | Families | Self-report | |
6 months | |||||
12 months | |||||
Treatment use and satisfaction | Therapy Attitude Inventory [118] | 10-item, 5-point Likert-type parent report of satisfaction with process and outcome of therapy | 3 months | Families | Self-report |
6 months | |||||
12 months | |||||
Multi-Sector Service Contacts—Revised Caregiver Form [119] | Report of additional services received by family/child in past 3 months and satisfaction with service(s) | Baseline | Families | Self-report | |
3 months | |||||
6 months | |||||
12 months | |||||