The effects of a prolonged exposure workshop with and without consultation on provider and patient outcomes: a randomized implementation trial

Background Prolonged exposure therapy (PE) is an evidence-based treatment for posttraumatic stress disorder (PTSD) that is underutilized in the military health system. Standard workshop training in PE may not be sufficient to alter provider behavior, but post-workshop consultation requires significant resources. Therefore, it is important to determine the incremental utility of post-workshop consultation. Methods This study used a hybrid type III randomized implementation trial at 3 US Army installations. Providers were randomized to receive a 4-day prolonged exposure workshop (Standard training condition, n = 60), or the prolonged exposure workshop followed by 6–8 months of post-workshop expert case consultation (Extended training condition, n = 43). The effects training condition were examined on provider attitudes (self-efficacy in delivering PE, expectations for patient improvement, and beliefs about PE), use of PE and PE components, and clinical outcomes of patients with PTSD (using the Clinician-Administered PTSD Scale (CAPS-5)). Results Extended condition providers reported greater improvements in self-efficacy, b = .83, 95% CI [.38, 1.27], t(79) = 3.71, p = .001, and d = .63. A greater proportion of patients in the Extended condition (44%) than in the Standard condition (27%) received at least 1 PE session, b = .76, t(233) = 2.53, p = .012, and OR = 2.13. Extended condition providers used more PE components (M = .9/session) than did Standard condition providers (M = .5/session), b = .54, 95% CI [.15, .93], t(68) = 2.70, p = .007, and d = .68. Finally, decrease in patients’ PTSD symptoms was faster for patients of Extended condition providers than for patients of Standard condition providers, b = − 1.81, 95% CI [− 3.57, − .04], t(263) = − 2.02, p = .045, and d = .66, and their symptoms were lower at the second assessment, b = − 5.47, 95% CI [− 9.30, − 1.63], t(210) = − 2.81, p = .005, and d = .66. Conclusions Post-workshop consultation improved self-efficacy for delivering PE, greater use of PE, faster PTSD reduction, and lower PTSD severity at the second assessment. To our knowledge, this is the first demonstration that post-workshop case consultation for PE improves patient outcomes. Trial registration Clinicaltrials.gov, NCT02982538. Registered December 5, 2016; retrospectively registered


Background
Despite the robust evidence supporting the efficacy of prolonged exposure (PE) for posttraumatic stress disorder (PTSD) [1], PE is underutilized [2][3][4]. This underutilization is consistent with the low use of exposure therapy [5] and evidence-based psychotherapies (EBPs) in general [6,7], highlighting the need for research on strategies to improve EBP implementation. One barrier that impedes EBP use is lack of therapist skill in delivering the treatment [8]. Indeed, Becker et al. [9] found that few providers who treat PTSD were trained in PE, and inadequate training was the most common reason for not using PE.
The US Departments of Defense (DoD) and Veterans Affairs (VA) have each pursued large-scale disseminationimplementation initiatives to increase the use of PE for PTSD, with differential success. In the VA PE roll-out, training included 4-day workshops followed by individual weekly consultation with 2 patients. An uncontrolled evaluation of this model found that consultation was associated with improved provider attitudes toward PE [10], which is independently associated with subsequent therapy adoption [11]. Eighteen months after PE consultation, 71% of VA providers reported using PE [12]. In contrast, the DoD adopted a workshop-only PE training model. Only 3% of DoD behavioral health providers listed PE as one of their primary treatments for patients with PTSD [13]. Consistent with broader implementation research findings, this pattern suggests that therapists require support after initial training in order to adopt the treatment (see [14] for a review). Consultation may also impact patient outcomes. For example, Monson et al. [15] found that consultation in cognitive processing therapy (CPT), including discussion of cases and conceptualization without audio review, resulted in significantly greater reductions in patients' PTSD outcomes relative to no consultation in a VA setting. This study also found that audio review was not a necessary component for consultation to improve patient outcomes. Several other uncontrolled studies have found similar effects [16][17][18]. However, to our knowledge, prior PE implementation studies evaluating post-workshop consultation have not examined provider behavior and patient outcomes. This is an important gap in the literature, as therapists sometimes express reluctance to utilize exposure therapy, particularly in the presence of comorbid conditions [19]. Furthermore, whereas Monson et al.'s [15] study was conducted in the VA, there may be important organizational factors within the military that alter the impact of consultation on provider and patient outcomes in this setting. We are not aware of any controlled trials of consultation as an implementation strategy for improving provider and patient outcomes within military clinics.
Relative to workshops alone, case consultation requires significant time and financial resources for both the providers and consultants. Therefore, it is important to determine the incremental utility of this implementation strategy, both on PE use and on patient outcomes in large systems such as the military. This study used a hybrid type III effectivenessimplementation design, in which the primary goal was to determine the utility of an implementation strategy and the secondary goal was to assess clinical outcomes associated with the strategy [20]. The current study evaluated the impact of post-workshop consultation over 6-8 months on provider self-efficacy, expectations for patient improvement, beliefs related to PE, use of PE, and patients' outcomes. We hypothesized that, compared to workshop alone ("Standard" training), the addition of post-workshop consultation ("Extended" training) would (1) increase provider selfefficacy for delivering PE, expectations for patient improvement following PE, and positive beliefs about PE; (2) increase the use of PE to treat patients with PTSD; and (3) significantly reduce PTSD symptoms.

Contributions to the literature
Adding expert case consultation to workshop training in prolonged exposure therapy improved provider self-efficacy, increased provider use of prolonged exposure, and reduced patients' PTSD symptom severity.
Post-workshop consultation appeared to be important not only for shifting provider attitudes and behavior, but also for improving patient outcomes, which is the ultimate goal of provider training.
The promising impact of augmenting a provider workshop with expert case consultation may justify this relatively resource intensive effort.

Methods
The Madigan Army Medical Center Institutional Review Board (IRB) served as the IRB for the three military sites. The IRBs at the University of Pennsylvania and the University of Texas Health Science Center at San Antonio as well as the US Army Human Research Protections Office also approved the protocol. Providers and patients who participated provided written informed consent.

Patients
Patients were active military personnel with PTSD symptoms (N = 242) who were receiving or seeking individual psychotherapy from a participating provider. The inclusion criterion was that they have clinically significant PTSD symptoms (score of ≥ 25 [21] on the Clinician-Administered PTSD Scale for DSM-5 [CAPS-5] [22]). Exclusion criteria were current bipolar disorder I or psychotic disorder, evidence of a moderate or severe traumatic brain injury, or current suicidal ideation severe enough to warrant immediate attention.

Procedures
The study design was a 2-armed randomized implementation trial comparing two PE training models: Standard training (workshop only) and Extended training (workshop plus consultation). Behavioral health providers were invited to participate in a 4-day PE workshop conducted on post. Initial PE workshops at each site were followed by additional on-site workshops as new providers enrolled in the study. Interested providers then completed informed consent with research staff. Local leadership agreed to support changes to participating providers' schedule templates to allow providers to have 2 timeslots of 90 min each week during the study. Providers at each site were randomized 1:1 by the study coordinators using an online random-number generator. Unfortunately, the random-number method for treatment assignment resulted in more providers being assigned to the Standard condition (n = 60) than to the Extended condition (n = 43). Unbalanced randomization can occur naturally (though infrequently) when using pure randomization methods. Fortunately, as reported later in the "Results" section, this uneven assignment did not result in treatment condition differences between providers on any of the baseline demographic or study variables. Data were collected from January 2014 to January 2018, when target enrollment was met, and the final data point was collected. After the workshop, those randomized to the Extended condition received weekly individual telephone consultation from a PE expert, including the review of session videotapes, with a goal of 2 PE training cases over 6-8 months. Providers obtained consent from patients who were PE training cases to video-record the sessions for the purpose of consultation. PE training cases were not invited to participate in the study. Sessions of patients who were study participants were not video-recorded.
Providers were asked to complete online surveys prior to the workshop, immediately after the workshop, and at 3-month intervals for up to 18 months after finishing training. However, since one purpose of this study was to train as many providers as possible, providers were trained up until about 9 months before completion of the study. So, some providers who enrolled in the study only had time to provide a few (as little as 2) assessments of attitudes before the study ended. During the training phase, providers in the Extended condition were encouraged to implement PE with at least two training cases. During the data collection phase, providers were encouraged to use their clinical judgment when deciding which clinical issue to focus on and which treatment approach to use, allowing for an evaluation of the impact of the implementation strategy (Standard versus Extended) on providers' selection of treatment approaches. These instructions aimed at eliminating the bias of demand characteristics that might influence greater use of PE while providers were in the study. Providers completed a checklist of the treatment procedures used with patients after each psychotherapy session.
Patients with significant PTSD symptoms (PTSD Checklist for DSM-5 [PCL-5] ≥ 25 [23] [completed as part of routine care]) receiving treatment from participant-providers were invited by their provider to provide data for the study. Interested patients met with research staff to provide informed consent. They then met with an independent evaluator (IE), blind to the type of therapy the patient received and to the type of training received by the provider, who assessed their PTSD symptoms on two occasions: at study enrollment and after 8-15 sessions or 5 months, whichever came first. The second assessment occurred from sessions 3-15 (M = 9.4 for Standard patients, M = 9.1 for Extended), with variability due to factors such as patient transfer to another military installation, deployment, and inability to continue treatment. IEs provided a summary of the assessment findings to the patient's provider. The study protocol and the statistical analysis plan are included in the online supplement.

Provider measures Provider demographics
This form assessed demographics and professional information (see Table 1).

Provider measure of attitude
This 52-item measure inquired about treatment practices, attitudes, expectancies, and beliefs. It included 8  items assessing self-efficacy in delivering PE (e.g., I can effectively guide patients through imaginal exposure), 1 item assessing expectations for patient improvement following PE (How effective do you think PE, if delivered competently, would be at improving your patients' PTSD symptoms and functioning?), 21 items assessing "beliefs about PE" (e.g., Asking patients to discuss traumatic memories in PE may retraumatize them) using the Therapist Beliefs about Exposure Scale [24] modified to refer specifically to PE, 7 items assessing the proportion of PTSD patients with whom various psychotherapy practices are used (e.g., Provide a detailed rationale of how a treatment works), and 15 items assessing the degree to which various patient factors (e.g., Stabilized comorbid bipolar disorder; Anger; Dissociation) would deter providers from using PE. Self-efficacy items were scored 1 = not at all confident to 7 = completely confident. Expectations for patient improvement following PE were scored 1 = not at all effective to 7 = very effective. Beliefs about PE were scored 1 = disagree strongly to 5 = agree strongly. The internal consistencies of the self-efficacy measure and the beliefs measures in the present sample were α = .918 and α = .924, respectively.

Procedures used in treatment checklist
This form assessed the use of 14 different procedures in the previous treatment session, including an "other" option. Providers could check multiple procedures, including PE-related procedures (imaginal exposure, in vivo exposure, processing of traumatic memory, and providing rationale for PE) and non-PE procedures (e.g., supportive therapy, eye-movement desensitization and reprocessing, psychodynamic therapy).

Patient measures Patient demographics
This form assessed standard demographics (see Table 2).

Clinician-administered PTSD scale for DSM-5 (CAPS-5)
The CAPS-5 [22] is a structured interview that was used to assess PTSD symptom severity based on the Diagnostic and Statistical Manual of Mental Disorders (DSM-5).
The CAPS-5 was administered by an IE blind to treatment condition at both patient assessment points (Cronbach's α = .915).

Four-day PE workshop
Four-day PE workshops were conducted at each military installation throughout the duration of the study in order to continue to recruit providers throughout the course of the study.

Data analytic plan
First, treatment condition differences on both provider and patient baseline characteristics were examined. Since there were 26 different characteristics compared, inflation of type I error was controlled using the Benjamini-Hochberg test. Multilevel models (MLMs) and generalized linear mixed models (GLMMs) were used to analyze the data. These intent-to-treat models included all providers and participants who provided at least 1 assessment. Per protocol, baseline level of outcome was used as a covariate when available. P was set at .05, two-sided. For hypothesis 1, we expected greater improvement in PE attitudes (self-efficacy, expectancy, and beliefs about PE) over time in Extended vs. Standard providers. Since these variables improved initially and then leveled out, we used a quadratic growth curve to model change over time. The quadratic model fit the data better than other curvilinear models (e.g., log, hyperbolic, etc.) according to Akaike Information Criterion (AIC) and the Bayes Information Criterion (BIC).
For hypothesis 2 (provider use of PE components), we first examined the total number of PE components used by each provider across all their patients and sessions using a Poisson regression. Since some providers had more patients enrolled in the study than others, the total number of treatment sessions differed between providers, so number of treatment sessions was included as an "offset" in the analysis, converting the result to number of PE components used per session. In addition, we also compared the proportion of patients in each condition who were treated with any PE (defined conservatively as at least one session that included both imaginal and in vivo exposure). Patients were coded 1 (treated with PE) or 0 (not treated with PE) in a GLMM analysis (patients nested within providers). Since there were two outcomes for this hypothesis, P values were corrected for inflation of Type 1 error using the Benjamini-Hochberg correction. The analysis for hypothesis 3 had three levels (the two repeated measurements of PTSD severity [baseline and final assessment] were nested within patients, who were nested within providers). Per our protocol, the primary outcome was PTSD symptoms at the second assessment. Since symptoms were expected to be lower for patients whose second assessment was after more sessions, session number of the final assessment was included as a moderator of treatment condition differences. And because the effect of sessions leveled out over time (i.e., the decrease in PTSD severity leveled out over sessions), "sessions" was coded as the square root of session number (this coding provided the best fit [lowest AIC and BIC] for the data). Follow-up GLMM analyses examined the secondary outcomes (change in PTSD diagnosis and clinically significant change, both dichotomous outcomes) using similar models. P values for the secondary outcomes were corrected for inflation of type 1 error using the Benjamini-Hochberg correction.
Initial analyses controlled for site as a covariate and moderator. However, site was not a significant factor in any of the analyses. Therefore, site was dropped and the analyses recomputed.
A priori power analyses, using RMASS2, indicated > .85 power to detect a medium effect size (d = .5) for hypothesis 1 (treatment condition differences in self-efficacy, expectancy, and beliefs toward PE) assuming 100 providers with at least 3 assessments each, thus setting sample size at N = 100. Post hoc power analyses, using actual N in each analysis, indicated that for hypothesis 1, power was > .90 power to detect a medium effect size. For hypotheses 2 and 3, the power to detect medium effects was > .80. Figure 1 displays participant flow through the study. Provider and patient demographics and assessment information are reported in Tables 1 and 2, respectively. Out of the 26 variables reported in these tables, no treatment condition differences emerged after Benjamini-Hochberg correction. Substantive (but not significant) differences between conditions did exist on patient gender and "any comorbid anxiety disorder" (Extended condition females = 19.2%, comorbid anxiety disorders = 56.4%, Standard condition females = 7.1%, anxiety disorders = 28.6%). However, neither patient gender nor patient comorbid anxiety disorders was a significant moderator or covariate in the analyses of the patients, nor did including them in these analyses change any of the reported results.

Use of PE therapy components
Due to the fact that new therapists entered the study throughout the study, not all therapists had patients with PTSD enter the study; many had no qualifying patients before the conclusion of the study. There were 49 Standard condition providers (81.7%) and 23 Extended condition providers (53.5%) who had patients in the study and were included in the analyses for hypotheses 2 and 3. The proportion of Extended condition providers without

Patient outcomes
For hypothesis 3, Standard condition providers had 172 patients (3.5 patients per provider) in the study. Extended condition providers completed consultation in time to have 70 patients (3.0 patients per provider). The small difference in patients per provider was primarily due to the consultation delay before Extended condition providers could take patients into the study. Thirty-six percent of patients did not complete the second PTSD assessment, in part due to transfers/deployment. These patients did not differ from those who completed both assessments on demographics or baseline PTSD severity.
Three-level MLM analyses included random effects for patients and for providers (which were non-significant) and baseline severity coded above/below median (this allowed inclusion of baseline CAPS in the MLM, providing intent-to-treat analyses). Improvement in PTSD symptoms per session was greater for patients of Extended condition providers than for patients of Standard providers, training × sessions interaction: b = − 1.81, 95% CI [− 3.57, − .04], t(263) = − 2.02, p = .045, and d = .62 (Fig. 3). Further analyses examined whether treatment condition differences were also apparent for the secondary outcomes: the loss of PTSD diagnosis and for clinically significant change (CSC) [25,26]. These dichotomous outcomes were analyzed using GLMM with a logistic linking function. The independent variables in these models included training, sessions, and the training × sessions interaction as above. Using data from a large, multisite PTSD study [15] and using the Jacobson and Truax definition of CSC [25], CSC was defined as reliable change (1.96 × SD = 13) [26] plus having the outcome within the normal range (CAPS-5 < 20.89) [15]. Analyses indicated that the proportion of patients with loss of PTSD diagnosis was higher in patients of Ex-  Post hoc sensitivity analyses were conducted to control for the fact that Extended providers saw two more PTSD patients than Standard providers before treating PTSD patients for the study. To test whether this accounted for treatment condition differences, two sets of analyses controlled for number of PTSD patients treated by each provider during the study, one including, and one not including, the two consultation patients in the count of patients. Both sets of analyses showed that (1) number of PTSD patients treated was not related to patient outcomes either as a moderator or as a covariate and (2) patients of Extended providers improved more than patients of Standard providers even when analyses controlled for number of PTSD patients treated (either as a covariate or as a moderator). These results suggest that simply treating more PTSD patients does not seem to account for the superior outcomes for patients of Extended providers.

Discussion
This hybrid type III effectiveness-implementation trial is the first study to demonstrate that consultation as an implementation strategy improved military providers' selfefficacy for using EBP, use of EBP, and patients' outcomes. These findings suggest that consultation is an effective implementation strategy for EBPs in the military health system and is also likely to be effective in other health systems.
Consistent with prior studies of post-workshop training [10], consultation increased provider self-efficacy in delivering PE, indicating that expert consultation improves confidence in providers' ability to use PE effectively. Inconsistent with hypotheses, consultation did not increase expectations for improvement in PTSD following PE, or positive beliefs about PE, more than workshop training alone. A lack of effect of consultation on expectations for improvement is particularly interesting since it suggests that our results were not merely a result of expectations/self-fulfilling prophecies [27].
Consistent with hypothesis, consultation was associated with greater use of PE. The rate of use of PE components by Extended providers was almost 1 component (exactly .9 components) per treatment session, which was 80% greater than the use of PE components by Standard providers (.5 per treatment session). Further, Extended condition providers implemented PE (defined as at least 1 session with both in vivo and imaginal exposure) with a greater proportion of their PTSD patients. The rate of PE use was 50% higher among providers who received consultation compared to those who did not. This occurred despite the fact that providers were not directly incentivized to use PE in this study.
Also consistent with our hypothesis, consultation resulted in faster improvement and lower PTSD symptoms among patients. This study also found that patients of Extended providers were much more likely to lose their PTSD diagnosis than patients of Standard providers (77% compared to 44%). In addition, over twice as many patients of Extended providers demonstrated clinically significant change than did patients of Standard providers (50% vs. 24%). To our knowledge, this is the first study to demonstrate that PE consultation impacts not only providers but also the clinical outcomes of their patients. This finding is consistent with prior research in the context of veterans receiving CPT for PTSD [15] and with several uncontrolled studies [16][17][18]. This finding is important because improving patient outcomes is the ultimate purpose of provider EBP training, and hastening recovery is critical to minimizing patient suffering, cost of treatment, and achievement of military readiness efficiently. These findings are consistent with a large body of evidence showing that workshop attendance alone is not sufficient to alter provider behavior, whereas follow-up consultation after the workshop can improve implementation [14]. This is the first study to our knowledge to demonstrate the benefit of consultation for improving implementation and patient outcomes.
Several limitations should be noted. First, to preserve external validity, we asked providers to self-report on the treatment procedures used in session, rather than using more intrusive data collection methods such as recording of sessions. It is possible that demand characteristics biased provider reporting, although providers were encouraged to use any treatment approach they felt was indicated for their patients. Second, many providers did not enroll any PTSD patients. Although providers reported having a caseload with least 20% having traumarelated difficulties, those who did not enroll patients to the study indicated that they either did not have patients with PTSD who were seeking psychotherapy (e.g., they may have presented for an evaluation or for crisis management) or did not have PTSD patients who met study criteria and agreed to participate. Importantly, there were no significant differences between providers who had study patients and those who did not on any of the assessed measures (on attitudes or on the 16 demographics). Third, relying on providers to invite patients may have biased the patient-participant pool. Study staff informed the participating providers frequently about the patients on their case load who met the screening criteria and encouraged providers to recruit all their eligible patients. However, ultimately, the provider made the decision of who to recruit. This is a limitation inherent to a type III hybrid implementation design, which required minimal influence on providers' selection of patients in order to obtain more generalizable observations about the effect of the implementation strategy on providers' behavior. Finally, our initial random number method for randomizing providers to treatment condition resulted in more providers being assigned to the Standard condition than to the Extended condition. Although this was unfortunate, unbalanced randomization can occur naturally (though infrequently) when using pure randomization methods rather than block randomization. Since there were no significant differences between providers assigned to the two treatment conditions, there is no indication that this uneven treatment assignment reflected a bias in treatment assignment. Thus, it is unlikely to be responsible for our obtained pattern of results. Finally, the findings may not generalize outside the military health system, as systems vary with regard to their organization, training and supervision practices, quality assurance practices, and the patient groups they serve.

Conclusions
The results of this study demonstrate the importance of post-workshop consultation. Compared to Standard training, Extended training was associated with: (1) greater self-efficacy delivering PE, (2) greater use of PE and PE components, (3) faster improvement and lower PTSD symptoms among patients, (4) greater probability of loss of PTSD diagnosis among patients, and (5) more than twice the probability of experiencing clinically significant gains among patients. To our knowledge, this is the first study to demonstrate that PE consultation impacts not only providers but also the clinical outcomes of their patients. However, consultation remains costly. For example, in this study, sessions were video-recorded and mailed by providers to expert consultants who reviewed sessions and met with providers individually for 1 hour by phone weekly over several months. Recent evidence suggests that review of session recordings may not be necessary in effective consultation [15]. Future research could focus on critical components of consultation in order to provide benefits with lower cost.

Supplementary information
Supplementary information accompanies this paper at https://doi.org/10. 1186/s13012-020-01014-x. prevent, diagnose, and treat combat-related PTSD and co-occurring conditions in active duty and recently discharged veterans. The consortium's portfolio of more than 65 affiliated projects has different foci, principal investigators, and research teams and are funded by a variety of federal and private sources. However, they benefit from the consortium's established infrastructure, specialized research cores, and synergistic collaboration. The STRONG STAR Consortium is directed by Alan Peterson, PhD, professor of psychiatry and behavioral sciences in the Long School of Medicine at The University of Texas Health Science Center at San Antonio, petersona3 @uthscsa.edu. For more information, visit www.strongstar.org.

Disclaimer
The views expressed in this article are solely those of the authors and do not reflect an endorsement by or the official policy of the US Army, the Department of Defense, the Department of Veterans Affairs, or the US Government. The investigators have adhered to the policies for protection of human subjects as prescribed in 45 CFR 46.
Authors' contributions Dr. Foa had full access to the study data and takes responsibility for the integrity of the data and the accuracy of the data analysis. Foa)]. The funding source had no involvement in the study design, the collection, analysis and interpretation of data, the writing of this report, or the decision to submit this article for publication.

Ethics approval and consent to participate
The University of Pennsylvania Institutional Review Board (IRB) served as the coordinating center for this study and the Madigan Army Medical Center (MAMC) IRB provided review and oversight for the coordinating center and each of the three study performance sites, in compliance with all applicable federal regulations governing the protection of human subjects. The MAMC IRB reviewed and approved the University of Pennsylvania protocol (MAMC protocol # 214055) on April 9, 2014; the Evans Army Community Hospital protocol (MAMC protocol # 214056) on April 9, 2014; the William Beaumont Medical Center protocol (MAMC protocol # 215025) on January 27, 2015; and the Blanchfield Army Community Hospital protocol (MAMC protocol # 215045) on April 16, 2015. All study staff, including investigators at military installations, have completed the requisite human subjects training preparatory to research. All participants provided informed consent.

Consent for publication
Not applicable.