A randomized controlled dismantling trial of post-workshop consultation strategies to increase effectiveness and fidelity to an evidence-based psychotherapy for Posttraumatic stress disorder
© Wiltsey Stirman et al.; licensee BioMed Central Ltd. 2013
Received: 12 April 2013
Accepted: 26 July 2013
Published: 1 August 2013
Posttraumatic Stress Disorder (PTSD) is a serious mental health condition with substantial costs to individuals and society. Among military veterans, the lifetime prevalence of PTSD has been estimated to be as high as 20%. Numerous research studies have demonstrated that short-term cognitive-behavioral psychotherapies, such as Cognitive Processing Therapy (CPT), lead to substantial and sustained improvements in PTSD symptoms. Despite known benefits, only a minority of clinicians provide these therapies. Transferring this research knowledge into clinical settings remains one of the largest hurdles to improving the health of veterans with PTSD. Attending a workshop alone is insufficient to promote adequate knowledge transfer and sustained skill; however, relatively little research has been conducted to identify effective post-training support strategies.
The current study investigates whether clinicians receiving post-workshop support (six-month duration) will deliver CPT with greater fidelity (i.e., psychotherapy adherence and competence) and have improved patient outcomes compared with clinicians receiving no formal post-workshop support. The study conditions are: technology-enhanced group tele-consultation; standard group tele-consultation; and fidelity assessment with no consultation. The primary outcome is independent assessment (via audio-recordings) of the clinicians’ adherence and competence in delivering CPT. The secondary outcome is observed changes in patient symptoms during and following treatment as a function of clinician fidelity. Post-consultation interviews with clinicians will help identify facilitators and barriers to psychotherapy skill acquisition. The study results will inform how best to implement and transfer evidence-based psychotherapy (e.g., CPT) to clinical settings to attain comparable outcomes to those observed in research settings.
Findings will deepen our understanding of how much and what type of support is needed following a workshop to help clinicians become proficient in delivering a new protocol. Several influences on clinician learning and patient outcomes will be discussed. An evidence-based model of clinical consultation will be developed, with the ultimate goal of informing policy and influencing best practice in clinical consultation.
KeywordsTraining Implementation Consultation Psychotherapy PTSD Cognitive processing therapy
The lifetime prevalence of PTSD is approximately 9% in Canada’s general population , 6.8% in the United States (U.S.) , and 15% to 19% among veterans of the recent conflicts in Iraq and Afghanistan [3, 4]. PTSD is associated with substantial functional impairment, suicide risk , numerous physical diseases [6–9], higher rates of mortality, high healthcare utilization and costs, and annual productivity loss of over $3 billion in the U.S. [10–15]. Studies of chronic PTSD indicate little change over time without active intervention .
Cognitive-behavioral therapies (CBT) for PTSD have been shown to be efficacious and are recommended as first-line therapies in treatment guidelines [17, 18]. Cognitive Processing Therapy (CPT) is a 12-session, trauma-focused cognitive therapy that targets symptoms of PTSD and its comorbidities  that results in sustained, significant reductions in symptoms and increases in quality of life for diverse populations [20–34]. Despite substantial efficacy data, clinicians do not routinely deliver CPT or other evidence-based psychotherapies (EBPs) .
In Canada, the U.S., and the United Kingdom, policymakers have prioritized investigating the implementation of EBPs to address this critical gap between research and practice . In response to the shortage of adequately trained clinicians , policymakers have issued mandates, provided incentives, and invested billions of dollars in EBP training of clinicians in public mental health settings [35, 38]. However, many of the utilized training strategies are not empirically supported, and much remains to be learned about how best to disseminate and implement EBPs in routine clinical practice. Without proven strategies to increase clinician use, there is little assurance that the investment that individuals and organizations make to develop EBPs will ultimately benefit patients. Both the Canadian Institute of Health Research (CIHR)’s Health Research Roadmap (2009 to 2014; ) and the U.S. National Institute of Mental Health  note the growing need for implementation science to better understand the determinants and uptake of knowledge. The use of evidence-based implementation strategies will ensure that money spent on treatment research has maximum impact.
Didactic workshops are a cornerstone of existing large-scale EBP implementation efforts [35, 38, 41]. However, workshops alone are insufficient to promote high-quality implementation and sustainment of EBPs in clinical settings. After a single workshop, most clinicians are unable to deliver EBPs with fidelity (adherence to the protocol and competence in delivery of the interventions [42, 43]). Fidelity to EBPs has been shown to predict patient outcomes for a number of mental health disorders [44–49], and has been associated with PTSD symptom outcomes in our preliminary research on CPT. To adequately prepare clinicians to deliver EBPs with fidelity, training packages that include post-workshop consultation and feedback have emerged as best practices, but little is understood about the relative benefits of different consultation strategies [41, 42].
Previous comparisons of post-workshop consultation strategies have lacked external validity (e.g., have been conducted under controlled conditions with highly motivated therapists in training clinics), rarely assessed fidelity objectively and in real-world practice conditions , had poor follow-up rates, rarely addressed barriers to implementation, and have demonstrated less than optimal results [41–43]. Thus, there is little empirical evidence to identify optimal, cost-efficient consultation and feedback strategies. Feedback based on observation is considered the ‘gold standard’ for training and is assumed to be necessary for optimal fidelity . Without more direct observation of therapist behavior, there is limited opportunity for the expert to provide the type of feedback required to promote fidelity and high-quality service delivery . The one randomized study that tested this assumption found that coaching based on review of session material resulted in greater change talk among clients . This study evaluated individual-level consultation under controlled conditions with limited external validity, and had no comparison to group consultation strategies. Individual feedback and consultation is too costly and resource-intensive to be feasible in large healthcare delivery systems . One strategy used in several training initiatives is expert-delivered group tele-consultation, in which clinicians receive feedback from an expert on their treatment provision based solely on their verbal reports of their sessions . However, the association between clinician report of their behavior and objective assessment of therapy fidelity is unknown.
To address this limitation of the standard consultation strategy, the present study also tests a technology-enhanced group tele-consultation. Our approach involves an expert and group members listening to, and providing feedback on, portions of group members’ audio-recorded therapy sessions on a weekly basis. This strategy provides exposure to a broad sample of case material for vicarious learning, and also allows the expert to provide more specific and accurate feedback than in standard consultation. However, while it provides the opportunity for more in-depth discussion of work samples for the clinicians who present in a given week, the time spent on the audio review may result in less time for other clinicians in the group to discuss their own cases when not presenting audio. Thus, each consultation strategy may have unique advantages over the other. The two consultation conditions will be compared to a no consultation (fidelity monitoring only) condition in order to assess the incremental benefit of post-workshop consultation. The study is funded by the Canadian Institutes of Health Research and represents the collaborative efforts of investigators at Ryerson University, Veterans Affairs Canada’s Operational Stress Injury National Network (OSINN), the U.S. National Center for PTSD, and Boston University. This article describes the study’s aims, method and proposed analytic strategy.
Research questions and hypotheses
Question 1: The principal research question to be addressed is whether different types of post-workshop support for mental health clinicians have differential effects on clinician fidelity to the CPT protocol.
Hypothesis 1: We hypothesize that the technology-enhanced group tele-consultation condition will evidence the highest levels of fidelity, the standard group tele-consultation condition will evidence intermediate levels of fidelity, and the no consultation condition will evidence the lowest fidelity.
Question 2: Is clinician fidelity to the CPT protocol is related to client outcomes?
Hypothesis 2: We hypothesize that fidelity to the CPT protocol, irrespective of consultation condition, will be positively associated with improved client outcomes.
Question 3: What are contextual barriers or facilitators to the skilled use of CPT in their routine practice?
Hypothesis 3: We hypothesize that organizational contextual variables such as local leadership support, organizational structure, and organizational climate, as well as clinician attitudes toward EBPs will influence the uptake of CPT.
This randomized controlled dismantling trial has independent fidelity rating as the primary outcome and patient outcomes as the secondary outcomes. Clinicians are randomly assigned to six months of post-workshop support following an intensive two-day CPT workshop delivered by the last author. The conditions are: standard group tele-consultation, technology-enhanced group tele-consultation, or no consultation/fidelity monitoring only. The post-workshop consultation conditions vary as to whether audio-recorded therapy sessions and other work samples, such as CPT worksheets, are reviewed in the group. This design permits the evaluation of the additive value of each of the elements (i.e., expert consultation, technology-assisted expert consultation). Follow-up assessments of fidelity will be conducted at six months post-workshop for clinicians, and follow-up symptom assessments will be conducted at three months post-treatment for patients.
Participation in the study requires that all clinicians consent at least two clients to have all of their CPT sessions audio-recorded for use in consultation and/or fidelity rating. Four therapy sessions will be randomly selected across the six-month period for fidelity rating, and fidelity to treatment will be assessed by trained graduate students and post-doctoral fidelity assessors. Consistent with recommended clinical practice and research trials, patient outcomes will be assessed pre-treatment, at alternate therapy sessions, and at three months post-treatment. A secure web-based application is used to manage the continuous uploading of encrypted audio-recorded therapy sessions to the study server and to protect the privacy and confidentiality of patients and clinicians. Study-provided digital audio-recorders (USB-equipped) are used to record, store and upload CPT sessions to the secure server for fidelity ratings. Study staff retrieve the randomly-selected audio-recordings and assign them to the CPT fidelity raters, who are blinded to study condition. To check if rater blinding has been achieved, fidelity raters will be asked to guess the study condition for each session they rated. Patient self-report data is collected by the clinicians, who then upload the data directly to the server and clinician CPT activity outside of study-related procedures is also recorded.
Participants in all conditions attend the workshop, receive the CPT manual and related materials, and have access to resources available through the free CPT-web online training (https://cpt.musc.edu/index). All participants who upload the required number of session recordings and patient symptom measures will be eligible to receive ‘CPT Provider’ status. This status is not tied to any change in remuneration for services. Provider status will be awarded to clinicians who demonstrate an acceptable level of fidelity to CPT based on session recordings randomly selected from the last two weeks of the six-month post-workshop period. Workshop participants who provide informed consent are randomized to one of the three post-workshop consultation conditions after completing the workshop. Participants in the two tele-consultation conditions receive six months of weekly one-hour group tele-consultation with a CPT expert. Each call includes discussions of the CPT protocol, challenging cases, treatment obstacles, and specific issues raised by participants within each group. Both tele-consultation conditions include a maximum of six to eight clinicians per call. CPT experts provide consultation across both consultation conditions to control for any expert-related effects. The prescribed and proscribed activities for each consultation condition are described in manuals, and study team members (SWS, CMM, ML) review consultant calls on a monthly basis to ensure that fidelity to the consultation condition is maintained. Consultants are trained by study team members prior to beginning consultation calls and receive consultation and feedback every four to six weeks or as needed for the first two waves of consultation, and as needed during the third consultation wave.
Inclusion and exclusion criteria
Mental health clinicians from the Operational Stress Injury clinics, Canadian Forces mental health services, and the broader Canadian community are eligible to participate in the study if they: attend a CPT workshop; are licensed as a mental health clinician and are able to practice independently; currently provide psychotherapy to patients with PTSD; consent to be randomized to one of the study conditions; and are willing to solicit patient participation. Eligible patient participants are: diagnosed with PTSD; and consent to have their sessions audio-recorded and listened to by a fidelity rater and potentially other clinicians teaching and learning CPT. Patients are permitted to continue other psychotherapeutic interventions if they are not specifically focused on treating PTSD symptoms. Ineligible patients include those not eligible for CPT based on the state of research evidence, including those with: current uncontrolled psychotic or bipolar disorder; substance dependence; imminent suicidality or homicidality that requires acute care; and significant cognitive impairment (although mild to moderate traumatic brain injury is permitted).
Outcome measures and assessment procedures
Fidelity ratings and assessment of CPT modification from audio-recorded CPT sessions
This study uses a modified version of the CPT fidelity measure  that has been used in previous clinical trials . The CPT fidelity measure examines clinicians’ adherence to, and competence in, delivering specific CPT interventions prescribed in each session. Clinicians are rated on their adherence and competence in delivering these elements (rated on a 7-point, Likert-type scale). Two studies have assessed the reliability of the measure and found 97% agreement between two raters across all items for adherence in one study (agreement was not reported for competence scores; ) and 100% agreement for adherence and overall competence ratings in another . Pilot research has been conducted to assess the predictive validity of the unique and essential scales for a small sample of veterans. CPT competence was highly correlated across all sessions with competence in cognitive therapy as measured by a validated cognitive therapy fidelity-assessment instrument, r = 0.82, p = 0.001 [44, 57]. Therapist competence in the first session of CPT was positively correlated with self-reported symptom improvement among patients from baseline to post-treatment, r = 0.79, p = 0.02. Additionally, we use a framework of modifications to identify any deviations from or adaptations to CPT that occur in the sessions . At least 10% of the sessions will be rated by all raters to independently assess inter-rater agreement.
Patient outcome measures
Patients are asked to complete measures related to their PTSD symptoms, functional impairment and overall quality of life at baseline (pre-treatment), at every alternate CPT treatment session, and at post-treatment follow-up (three months post-treatment). The assessment measures include: ‘The Posttraumatic Stress Disorder Checklist’ (PCL-S; ), a 17-item questionnaire that reliably measures the severity of distress related to PTSD symptoms (Chronbach’s alpha = 0.94). Each item is measured on a 5-point Likert scale, which maps on to the DSM-IV criteria for PTSD; ‘The Outcome Questionnaire – 45’ (OQ- 45; ), a 45-item self-report measure of three dimensions of client functioning in the past week: symptom distress (e.g., anxiety, depression), interpersonal problems, and social role adjustment (Chronbach’s alpha = 0.71 to 0.93). The OQ-45 has established clinical cut-offs and a reliable change indices to signal meaningful improvements or deterioration; and ‘The Short-Form Quality of Life Health Survey’ (SF-12; ), a 12-item survey that measures mental components (vitality, social functioning, role-emotional, and mental health) and physical components (physical functioning, role-physical, bodily pain, and general health) of health related to quality of life. All questionnaires are available in English and French, and take clients around 15 to 20 minutes to complete. Therapists upload summary scores to a secure website at the same time that they upload their audio-recordings.
Clinician demographic characteristics and experience
A pre-workshop questionnaire is administered to assess relevant clinician demographic information (i.e., age, gender, education), clinician experience (years of licensed practice, hours of formal training in CBT, hours of supervised post-graduate CBT training), prior CPT training experience (workshop hours and supervision hours), experience treating clients with PTSD (number of clients), and volume of current clinical activity (caseload and hours devoted to psychotherapy).
Clinician monthly CPT activity reporting
During the six-month active study period, all clinicians complete a monthly report via a web-based survey on CPT activity. Questions include: the number of CPT sessions provided over the past month (irrespective of patients’ enrollment in the study); cumulative number of new CPT clients seen; frequency, duration, and satisfaction with consultation over the past month (as applicable); and a rating of clinician confidence in their ability to adhere to adherence to the CPT protocol.
Quantitative and qualitative measurement of potential barriers and facilitators to CPT skills uptake and implementation outcomes
The Organizational Readiness to Change Questionnaire  is a pre-workshop self-report questionnaire that assesses various elements of a clinician’s perceived support for training implementation. A total of 18 scales (Chronbach alphas = 0.56 to 0.92) are grouped within four dimensions of readiness (i.e., motivation for change, adequacy of resources, staff attributes, organizational climate). The Dimensions of Organizational Readiness-Revised (DOOR-R; ) scale assesses perspectives on intra- and extra-organizational implementation readiness. The Evidence-Based Practice Attitudes-50 (EBPAS-50; ) is a 50-item, self-report measure of clinician attitudes toward EBPs, consisting of 12 factors, with moderate to large factor loadings and fair to excellent internal consistency reliabilities.
We also interview clinicians at baseline, post-consultation, and three months post-consultation to further assess for barriers and facilitators to the use of CPT in their practice. The interviews query effective tactics and methods that may have been used outside of the consultation methods (i.e., positive deviance; ) that the clinicians deemed helpful in acquiring and maintaining CPT skills. We based our interview questions on those used to assess influences on training and implementation in previous research , which are based in part on the Consolidated Framework for Implementation Research (CFIR ).
Sample size and justification for power calculations
In conducting power calculations, we considered assessment attrition, within-clinician correlation, and within-setting effects. Based on a power curve calculation with the following assumptions: 30 clinicians per condition; 4 sessions rated for fidelity per clinician, randomly selected over time; with an alpha-level of 0.05, within provider correlation estimated at 0.38, 10% proportion of variance accounted for by the random effect associated with agency, based on a two-sided test, the detectable effect size for at least 80% power will be medium to large, per Cohen’s  classifications, ranging from d = 0.41 to 0.55.
Recruitment and randomization
We recruited OSI clinicians directly. Recruitment took place at CPT workshops in Canada from 2012 to 2013. Three months before the first workshop, the OSINN invited clinicians and clinic coordinators from the OSI clinic network, Canadian Forces health services, and approximately 800 community-based clinicians who provide services to veterans with PTSD. The clinicians include psychologists, social workers, psychiatric nurses, and psychiatrists. This invitation was accompanied by a letter soliciting participation in the study. Prior to enrolling clinicians, ethics approval for the study was obtained from the clinic or institution where the clinician worked. Community clinicians were informed that the CHIR-supported workshop was free, and 24 clinicians received up to $500 reimbursing travel and accommodation. Partial reimbursement for the community clinicians was intended to offset the barrier of cost associated with a two-day workshop. Following the initial CIHR-funded workshop, two provincial health authorities and one hospital hosted workshops. One of the authors (CMM) was the primary CPT trainer. Interested clinicians consented prior to, or within the week following the workshop. Consent forms outlined the purpose and requirements of the study, as well as potential risks. A random allotment of participants was generated by the study statistician for each condition. To avoid any biases in condition assignment, condition assignment was communicated by participant number to the study coordinator after participant numbers had been assigned to the consented participants. Clinicians received a notification e-mail from the study coordinator within one week after the workshop regarding their randomized condition. The e-mail also included consultant contact information and a schedule of consultation sessions (if applicable to their condition). As in other studies of this nature (e.g., ), the clinicians were asked to provide an information sheet about the study and obtain verbal consent from patients who agreed to participate in the study, then document patient consent in his/her medical records. Upon returning to their respective work places, participants began audio-recording CPT sessions with consented patients.
Proposed analytic strategy
Trial reporting will follow the CONSORT 2010 statement for reporting randomized controlled trials . Analyses will be preceded by careful examination of the descriptive statistics and distributional form of all variables. The effectiveness of randomization will be evaluated by comparing the different conditions on baseline measures, demographics, and key outcome variables using Chi-squared tests and t-tests. The primary analyses will be conducted using random effects regression modeling. These models provide more flexibility in modeling missing data than general linear models and are appropriate for nested data. ‘Intention-to-train’ principles will be used in the analyses of clinicians who are assigned to a consultation modality but either fail to engage or are unable to fulfill the post-workshop participation requirements (e.g., do not attend consultation sessions). Analyses of both the intention-to-train and completer samples will be conducted.
To explore the contribution of organization and clinician-level factors to the outcomes of interest, in addition to random effects regression modeling, we will conduct qualitative analyses on interview data. Qualitative and quantitative data will be integrated to facilitate a fine-grained understanding of processes and characteristics that influence training outcomes. Interviews will be analyzed in line with goals, objectives and key research questions. Using content analysis, we will identify analytical categories to describe and explain our observations. Our work will occur in five stages outlined in Pope, Ziebland, and Mays’ framework approach to qualitative analysis : Familiarization; Identifying a Thematic Framework; Indexing; Charting; and Mapping and Interpretation. In the second stage, codes will be derived deductively by identifying categories at the beginning of the research (e.g., elements of the CFIR), and inductively by identifying those that emerge gradually from the data. In subsequent phases, survey data on contextual factors will be examined in conjunction with fidelity scores for the purposes of validation, hypothesis generation, and expansion [71, 72] to understand ways in which contextual factors influence fidelity to CPT.
To date, six workshops have been completed, and, through efforts to enroll clinicians over three waves of recruitment, the recruitment goal of 90 clinicians was exceeded. The first and second waves of consultation have been completed, and the third wave is underway. Pre- and post-training interviews with providers from waves one and two are complete. Thus far, our retention rate and compliance with study procedures (e.g., uploading of sessions) has been high, and a high proportion of clinicians uploaded recordings at baseline and over the course of the consultation period. We anticipate that data collection will be complete in early 2014.
There are four significant innovations that the current project offers the field of Implementation Science. First, this will be the first study to examine psychotherapy training and consultation within a broader implementation context. Previous randomized training studies have not examined the organizational context. Our use of mixed-methods research strategies in the context of a randomized controlled trial  will allow us to examine a broad range of potential influences on the outcomes that we are assessing. This approach will extend our understanding of interactions between these factors and training outcomes.
A second major advancement is the empirical investigation of the optimal (efficacious, scalable and replicable) methods of post-workshop consultation. To adequately test the incremental value of consultation, we are conducting a randomized, dismantling research design, which includes manipulation of the type of post-workshop consultation while holding other variables constant. Results will help inform policy on post-workshop support. The inclusion of the fidelity monitoring only condition allows for testing an as yet unevaluated and potentially efficient strategy for increasing fidelity to EBPs: recording sessions with the knowledge that they will be reviewed and evaluated for fidelity. Our control condition allows for the formal testing of this possibility while allowing us to examine the incremental value of different types of expert consultation.
Although fidelity is a key implementation outcome in the model by Proctor and colleagues  and in this study, fidelity to EBPs has not been adequately assessed by independent evaluators in prior implementation research. It has either not been assessed at all, or has been assessed through self-report, standardized patient protocols , or a single session selected by providers (c.f., [52, 75]). Our third innovation is to improve upon this methodological limitation of prior studies by randomly selecting audio-recorded sessions for fidelity monitoring. We will also conduct the only investigation of the effect of clinician fidelity to an EBP in routine care settings on PTSD outcomes to date. Because this investigation will examine sessions from novice clinicians who received varying intensities of consultation, we expect to see a greater range of fidelity than that seen in previous investigations of the impact of fidelity on symptom outcomes, as most studies to date have used clinical trial data with sessions from closely supervised clinicians. Finally, no evidence-based guidance exists regarding how to facilitate fidelity to the CPT protocol and ultimately patient outcomes. Using transcripts from consultation sessions, members of our research team will develop and test a theory of evidence-based consultation.
The study has been approved by the Ryerson University Research Ethics Board, and the Boston VA and Boston University Institutional Review Boards.
Posttraumatic stress disorder
Cognitive processing therapy
Cognitive behavioral therapy
Evidence based psychotherapy.
- Van Ameringen M: Post-traumatic stress disorder in Canada. CNS Neurosci Ther. 2008, 14 (3): 171-181. 10.1111/j.1755-5949.2008.00049.x.View ArticlePubMedGoogle Scholar
- Kessler RC: Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the national comorbidity survey replication. Arch Gen Psychiatry. 2005, 62 (6): 617-10.1001/archpsyc.62.6.617.View ArticlePubMedPubMed CentralGoogle Scholar
- Tanielian T, Jaycox L: Invisible wounds of war : psychological and cognitive injuries, their consequences, and services to assist recovery. 2008, Santa Monica: RAND CorporationGoogle Scholar
- Schell TL, Marshall GN: Survey of individuals previously deployed for OEF/OIF, in invisible wounds of war: psychological and cognitive injuries, their consequences, and services to assist recovery. Edited by: Tanielian T, Jaycox LH. 2008, Santa Monica: RAND Center for Military Health Policy Research, 87-115.Google Scholar
- Sareen J: Physical and mental comorbidity, disability, and suicidal behavior associated with posttraumatic stress disorder in a large community sample. Psychosom Med. 2007, 69 (3): 242-248. 10.1097/PSY.0b013e31803146d8.View ArticlePubMedGoogle Scholar
- Sareen J: The relationship between anxiety disorders and physical disorders in the U.S. National Comorbidity Survey. Depress Anxiety. 2005, 21 (4): 193-202. 10.1002/da.20072.View ArticlePubMedGoogle Scholar
- Schnurr PP: Predictors and outcomes of PTSD in World War II veterans exposed to mustard gas. J Consult Clin Psychol. 2000, 68 (2): 258-268.View ArticlePubMedGoogle Scholar
- Sibai AM, Fletcher A, Armenian HK: Variations in the impact of long-term wartime stressors on mortality among the middle-aged and older population in Beirut, Lebanon, 1983–1993. Am J Epidemiol. 2001, 154 (3): 128-137.View ArticlePubMedGoogle Scholar
- Schnurr P, Green BL: Understanding relationships among trauma, post-traumatic stress disorder, and health outcomes. Adv Mind Body Med. 2004, 20 (1): 18-29.PubMedGoogle Scholar
- Kessler RC: Posttraumatic Stress Disorder: The burden to the individual and to society. J Clin Psychiatry. 2000, 61 (suppl 5): 4-12.PubMedGoogle Scholar
- Hoge CW, Auchterlonie JL, Milliken CS: Mental health problems, use of mental health services, and attrition from military service after returning from deployment to Iraq or Afghanistan. J Am Med Assoc. 2006, 295: 1023-1032. 10.1001/jama.295.9.1023.View ArticleGoogle Scholar
- Wolfe J: Posttraumatic stress disorder and war-zone exposure as correlates of perceived health in female Vietnam War veterans. J Consult Clin Psychol. 1994, 62 (6): 1235-1240.View ArticlePubMedGoogle Scholar
- Schnurr PP: PTSD and utilization of medical treatment services among male Vietnam veterans. J Nerv Ment Dis. 2000, 188 (8): 496-504. 10.1097/00005053-200008000-00004.View ArticlePubMedGoogle Scholar
- Walker EA, Newman E, Koss M: Costs and health care utilization associated with traumatic experiences, in Trauma and health: Physical health consequences of exposure to extreme stress. Edited by: Schnurr P, Green BL. 2004, Washington, DC: American Psychological Association, 43-69.View ArticleGoogle Scholar
- Dobie JD: Posttraumatic stress disorder in female veterans: association with self-reported health problems and functional impairment. Arch Intern Med. 2004, 164 (4): 394-400. 10.1001/archinte.164.4.394.View ArticlePubMedGoogle Scholar
- Resick PA, Monson CM, Gutner C: Psychosocial treatments for PTSD., in handbook of PTSD: science and practice. Edited by: Friedman MJ, Keane TM, Resick PA. 2007, NY: Guilford, 330-358.Google Scholar
- Foa EB: Effective treatments for PTSD: Practice guidelines from the International Society for Traumatic Stress Studies. Effective treatments for PTSD: Practice guidelines from the International Society for Traumatic Stress Studies (2nd ed.). 2009, New York, NY US: Guilford Press, 2Google Scholar
- Group, V.A.D.o.D.T.M.o.P.-T.S.W: VA/DoD Clinical Practice Guideline for Management of Post-Traumatic Stress. 2010Google Scholar
- Resick MC, Chard KM: Cognitive processing therapy veteran/military version: Therapist and patient materials manual. 2008Google Scholar
- Resick PA: A randomized clinical trial to dismantle components of cognitive processing therapy for posttraumatic stress disorder in female victims of interpersonal violence. J Consult Clin Psychol. 2008, 76 (2): 243-258.View ArticlePubMedPubMed CentralGoogle Scholar
- Resick PA: Long term outcomes of cognitive behavioral treatments for posttraumatic stress disorder among female rape survivors. J Consult Clin Psychol. 2012, 80 (2): 210-212.View ArticleGoogle Scholar
- Schulz PM: The effectiveness of cognitive processing therapy for PTSD with refugees in a community setting. Cognitive and Behavioral Practice. 2006, 13: 322-331. 10.1016/j.cbpra.2006.04.011.View ArticleGoogle Scholar
- Resick PA: A comparison of cognitive processing therapy, prolonged exposure and a waiting condition for the treatment of posttraumatic stress disorder in female rape victims. J Consult Clin Psychol. 2002, 70 (4): 867-879.View ArticlePubMedPubMed CentralGoogle Scholar
- Monson CM: Cognitive processing therapy for veterans with military-related posttraumatic stress disorder. J Consult Clin Psychol. 2006, 74 (5): 898-View ArticlePubMedGoogle Scholar
- Thomas TL, Kang HK, Dalager NA: Mortality among women Vietnam veterans, 1973–1987. Am J Epidemiol. 1991, 134: 973-980.PubMedGoogle Scholar
- Chard KM: Cognitive processing therapy for sexual abuse: a treatment outcome study. J Consult Clin Psychol. 2005, 73: 965-971.View ArticlePubMedGoogle Scholar
- Galovski TE: Does cognitive-behavioral therapy for PTSD improve perceived health and sleep impairment?. J Trauma Stress. 2009, 22 (3): 197-204. 10.1002/jts.20418.View ArticlePubMedPubMed CentralGoogle Scholar
- Alvarez J: The comparative effectiveness of cognitive processing therapy for male veterans treated in a VHA posttraumatic stress disorder residential rehabilitation program. J Consult Clin Psychol. 2011, 79 (5): 590-599.View ArticlePubMedGoogle Scholar
- Resick PA, Schnicke MK: Cognitive processing therapy for sexual assault victims. J Consult Clin Psychol. 1992, 60 (5): 748-756.View ArticlePubMedGoogle Scholar
- Chard KM: A comparison of OEF and OIF veterans and vietnam veterans receiving cognitive processing therapy. J Trauma Stress. 2010, 23 (1): 25-32.PubMedGoogle Scholar
- Chard KM: Dissemination and experience with cognitive processing therapy. JARD. 2012, 49 (5): 667-678.Google Scholar
- Ahrens J, Rexford L: Cognitive processing therapy for incarcerated adolescents with PTSD. Journal of Aggression, Maltreatment & Trauma. 2002, 6 (1): 201-216. 10.1300/J146v06n01_10.View ArticleGoogle Scholar
- Schulz PM: The effectiveness of cognitive processing therapy for PTSD with refugees in a community setting. Cognitive and Behavioral Practice. 2006, 13 (4): 322-10.1016/j.cbpra.2006.04.011.View ArticleGoogle Scholar
- Chard KM: An evaluation of cognitive processing therapy for the treatment of posttraumatic stress disorder related to childhood sexual abuse. J Consult Clin Psychol. 2005, 73 (5): 965-971.View ArticlePubMedGoogle Scholar
- McHugh RK, Barlow DH: The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. American Psychology. 2010, 65 (2): 73-84.View ArticleGoogle Scholar
- Insel TR: Devising prevention and treatment strategies for the nation’s diverse populations with mental illness. Psychiatr Serv. 2007, 58 (3): 395-10.1176/appi.ps.58.3.395.View ArticlePubMedPubMed CentralGoogle Scholar
- Weissman MM: National survey of psychotherapy training in psychiatry, psychology, and social work. Arch Gen Psychiatry. 2006, 63 (8): 925-934. 10.1001/archpsyc.63.8.925.View ArticlePubMedGoogle Scholar
- Karlin BE: Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the veterans health administration. J Trauma Stress. 2010, 23 (6): 663-673. 10.1002/jts.20588.View ArticlePubMedGoogle Scholar
- Canadian Institutes of Health Research: Health Research Roadmap: Creating innovative research for better health and health care. 2013,http://www.cihr-irsc.gc.ca/e/documents/strat_plan_2009_e.pdf,Google Scholar
- National Institute of Mental Health: Dissemination and Implementation Research Workshop: Harnessing Science to Maximize Health. Available from: http://www.nimh.nih.gov/research-priorities/scientific-meetings/2007/dissemination-and-implementation-research-workshop-harnessing-science-to-maximize-health.shtml
- Rakovshik SG, McManus F: Establishing evidence-based training in cognitive behavioral therapy: a review of current empirical findings and theoretical guidance. Clin Psychol Rev. 2010, 30 (5): 496-516. 10.1016/j.cpr.2010.03.004.View ArticlePubMedGoogle Scholar
- Beidas RS, Kendall PC: Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010, 17 (1): 1-10.1111/j.1468-2850.2009.01187.x.View ArticleGoogle Scholar
- Herschell AD: The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010, 30 (4): 448-466. 10.1016/j.cpr.2010.02.005.View ArticlePubMedPubMed CentralGoogle Scholar
- Shaw BF: Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. J Consult Clin Psychol. 1999, 67 (6): 837-846.View ArticlePubMedGoogle Scholar
- Hogue A: Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. J Consult Clin Psychol. 2008, 76 (4): 544-555.View ArticlePubMedPubMed CentralGoogle Scholar
- DeRubeis RJ: Medications versus cognitive behavior therapy for severely depressed outpatients: mega-analysis of four randomized comparisons. Am J Psychiatry. 1999, 156 (7): 1007-1013.PubMedGoogle Scholar
- Strunk DR, Brotman MA, DeRubeis RJ: The process of change in cognitive therapy for depression: predictors of early inter-session symptom gains. Behav Res Ther. 2010, 48 (7): 599-606. 10.1016/j.brat.2010.03.011.View ArticlePubMedPubMed CentralGoogle Scholar
- Strunk DR: Therapist competence in cognitive therapy for depression: predicting subsequent symptom change. J Consult Clin Psychol. 2010, 78 (3): 429-437.View ArticlePubMedPubMed CentralGoogle Scholar
- Schoenwald SK: Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Adm Policy Ment Health. 2008, 35 (5): 379-394. 10.1007/s10488-008-0181-z.View ArticlePubMedGoogle Scholar
- Decker SE, Jameson MT, Naugle AE: Therapist training in empirically supported treatments: a review of evaluation methods for short- and long-term outcomes. Adm Policy Ment Heath. 2011, 38: 254-286. 10.1007/s10488-011-0360-1.View ArticleGoogle Scholar
- Brosan L, Reynolds S, Moore RG: Self evaluation of cognitive therapy performance: Do therapists know how competent they are?. Behav Cogn Psychother. 2008, 36 (5): 581-587. 10.1017/S1352465808004438.View ArticleGoogle Scholar
- Miller WR: A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004, 72 (6): 1050-1062.View ArticlePubMedGoogle Scholar
- Ruzek JI, Rosen RC: Disseminating evidence-based treatments for PTSD in organizational settings: a high priority focus area. Behav Res Ther. 2009, 47 (11): 980-989. 10.1016/j.brat.2009.07.008.View ArticlePubMedGoogle Scholar
- Resick PA, Nishith, and M.C. Astin: Empirically based therapy for posttraumatic stress disorder. , in Effective Interventions in Mental Health: Cognitive Behavior Therapy Introduced. Edited by: Bilsen HV. 1999, Wellington, New Zealand: Central Institute of Technology PressGoogle Scholar
- Resick PA: A randomized clinical trial to dismantle components of cognitive processing therapy for posttraumatic stress disorder in female victims of interpersonal violence. J Consult Clin Psychol. 2008, 76 (2): 243-View ArticlePubMedPubMed CentralGoogle Scholar
- Resick PA: A comparison of cognitive-processing therapy with prolonged exposure and a waiting condition for the treatment of chronic posttraumatic stress disorder in female rape victims. J Consult Clin Psychol. 2002, 70 (4): 867-View ArticlePubMedPubMed CentralGoogle Scholar
- Vallis TM, Shaw BF, Dobson K: The cognitive therapy scale: psychometric properties. J Consult Clin Psychol. 1986, 54: 381-385.View ArticlePubMedGoogle Scholar
- Stirman S: Development of a framework and coding system for modifications made to evidence-based programs and interventions. Implement Sci. 2013, 8: 65-10.1186/1748-5908-8-65.View ArticlePubMedPubMed CentralGoogle Scholar
- Blanchard EB, Jones-Alexander J, Buckley TC, Forneris CA: Psychometric properties of the PTSD checklist (PCL). Behavioral Research & Therapy. 1996, 34: 669-673. 10.1016/0005-7967(96)00033-2.View ArticleGoogle Scholar
- Lambert MJ: The reliability and validity of the outcome questionnaire. Clin Psychol Psychother. 1996, 3: 249-258. 10.1002/(SICI)1099-0879(199612)3:4<249::AID-CPP106>3.0.CO;2-S.View ArticleGoogle Scholar
- Ware JE, Kosinski M, Keller SD: A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34 (3): 220-233. 10.1097/00005650-199603000-00003.View ArticlePubMedGoogle Scholar
- Lehman WE, Greener JM, Simpson DD: Assessing organizational readiness for change. J Subst Abuse Treat. 2002, 22 (4): 197-209. 10.1016/S0740-5472(02)00233-7.View ArticlePubMedGoogle Scholar
- Hoagwood K, Schoenwald SK, Chapman JE: Dimensions of Organizational Readiness-Revised (DOOR-R). 2003, Unpublished instrumentGoogle Scholar
- Aarons G: Administration and policy in mental health and mental health services research. Expanding the Domains of Attitudes Towards Evidence-Based Practice: The Evidence Based Practice Attitude Scale-50. 2010, 1:Google Scholar
- Marsh DR: The power of positive deviance. BMJ. 2004, 329 (7475): 1177-10.1136/bmj.329.7475.1177.View ArticlePubMedPubMed CentralGoogle Scholar
- Stirman SW: Perspectives on training and consultation in a cognitive therapy implementation program: implications for implementation. Depression Research and Treatment. 2012, 2012: 11-Google Scholar
- Damschroder LJ: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009, 4: 50-10.1186/1748-5908-4-50.View ArticlePubMedPubMed CentralGoogle Scholar
- Cohen J: Statistical Power Analysis for the Behavioral Sciences (Second ed.). 1988, Hillsdale, NJ: ErlbaumGoogle Scholar
- Schulz KF, Altman DG, Moher D: CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010, 152 (11): 726-W.293-View ArticleGoogle Scholar
- Pope C, Ziebland S, Mays N: Qualitative research in health care. Analysing qualitative data. BMJ Research. 2000, 320 (7227): 114-116. 10.1136/bmj.320.7227.114.View ArticleGoogle Scholar
- Palinkas L: Mixed Method Designs in Implementation Research. Administration and Policy in Mental Health and Mental Health Services Research. 2010, 38 (1): 44-53.View ArticlePubMed CentralGoogle Scholar
- Spillane J: Mixing methods in randomized controlled trials (RCTs): validation, contextualization, triangulation, and control. Educ Assess Eval Account. 2010, 22 (1): 5-28. 10.1007/s11092-009-9089-8.View ArticleGoogle Scholar
- Proctor EK: Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research. 2009, 36 (1): 24-35. 10.1007/s10488-008-0197-4.View ArticlePubMedGoogle Scholar
- Baer JS: An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug Alcohol Depend. 2004, 73 (1): 99-106. 10.1016/j.drugalcdep.2003.10.001.View ArticlePubMedGoogle Scholar
- Simons AD: Training and dissemination of cognitive behavior therapy for depression in adults: a preliminary examination of therapist competence and client outcomes. J Consult Clin Psychol. 2010, 78 (5): 751-756.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.