Table 3 Summary of data for studies included in the review
Author(s) | Study design | Implementation strategy | Implementation outcome | Outcome measurement | Related findings |
|---|---|---|---|---|---|
Bland et al. [56] | Retrospective cohort | 1) Assess for readiness and identify barriers and facilitators 2) Audit and provide feedback 3) Conduct educational meetings 4) Develop and implement tools for quality monitoring | Fidelity | Visual inspection of 17 months of time series for increased adherence > 5% | Median adherence ranged from .52 to .88 across all settings and professional disciplines; PT had the greatest adherence across disciplines (p < .004); IRF and acute higher had adherence than outpatient (p < .001). Specific events increased adherence 40% of the time, with those gains maintained for >1 month 60% of the time. |
Braun et al. [46] | Process evaluation | 1) Conduct educational outreach visits 2) Distribute educational materials 3) Identify and prepare champions 4) Make training dynamic | Appropriateness Feasibility | Pre structured patient files; patient logs; therapist and patient questionnaires | In 11 out of 16, 69%, of participants, the mental practice intervention was delivered according to the framework; patients received the minimum amount of mental practice recommended (13 out of 14, 93%), and they undertook unguided (12 out of 16, 75%) practice as recommended. Implementation was more challenging than expected. |
Clarke et al. [62] | Process evaluation | 1) Assess for readiness and identify barriers and facilitators 2) Conduct educational meetings 3) Use train-the-trainer strategies | Adoption | Ethnographic approach—observations, interviews & documentary analysis | Minimal adoption of LSCTC across units and professions; adoption varied due to staff skill and expertise, infrastructure, and local mgmt. factors. Contextual factors, including organizational history and team relationships, external policy, and service development initiatives, impinged on the implementation of the caregiver training program in unintended ways. |
Connell et al.1 [63] | Formative evaluation | 1) Capture and share local knowledge 2) Create a learning collaborative 3) Work with educational institutions | Adoption Fidelity | Semi-structure interviews | Fidelity to GRASP was lower than expected (not always used in a way shown to be effective) even though adoption was perceived to be strong. Almost all intervention components were adapted to some degree when used in clinical practice; coverage was wider, the content adapted, and the dose, when monitored, was less. |
Connell et al.2 [45] | Cross-sectional | 1) Identify early adopters 2) Visit other sites | Acceptability Adoption Appropriateness Feasibility | Self-administrated questionnaire | Sixty-one therapists (22.2%) reported that they had tried occasionally or regularly in practice (33 PTs & 28 OTs). GRASP was used most frequently by therapists in community settings (n = 27), followed by rehabilitation (n = 20) and acute (n = 14). Therapists working in rehabilitation were significantly more positive when asked about whether a manual would have positive outcomes for people with stroke (p = .003), the applicability of a manual in stroke rehabilitation (p = .009), and whether it could be easily incorporated into their work setting (p = .002). Therapists displayed positive opinions toward implementing a manual with graded progressions of structured upper limb exercises for people after stroke. Opinions were different between therapists who had used GRASP and those who had not. |
Doyle and Bennett [44] | Pre–post | 1) Conduct educational outreach visits 2) Develop educational materials 3) Distribute educational materials | Acceptability Adoption | Questionnaire; Patient Practitioner Orientation Scale (PPOS) | A theory-based workshop yielded significant changes in knowledge, attitudes, and perceived behavioral control and intended behaviors about sensory impairment management, research utilization, and shared decision making. Preworkshop: agreement for current behaviors ranged 5.3–52.6%, knowledge scores (M = 3.32, SD = 1.06), PPOS scores (M = 54.56, SD = 6.39). Postworkshop: agreement for intended behaviors ranged 84.2 to 100%, knowledge scores (M = 8.53, SD = 0.91) statistically significant (p = .00), PPOS scores (M = 49.22, SD = 5.80) statistically significant (p = .00)—indicating more patient-centeredness. |
Eriksson et al. [55] | Longitudinal | 1) Create a learning collaborative 2) Develop academic partnerships 3) Develop educational materials 4) Distribute educational materials 5) Facilitation 6) Make training dynamic 7) Obtain formal commitments 8) Organize clinician implementation team meetings 9) Promote adaptability 10) Purposefully reexamine the implementation | Adoption | Focus group interviews | Three subcategories were identified from the focus group interviews: (1) including in the scientific world, (2) involving as an actor of science, and (3) integrating into a partnership. A core category emerged: the implementation of client-centered practice enabled the fusion of practice and science. An increased experience of using CADL and support from the researchers changed the OTs' attitudes towards engaging in research from being an outsider to the scientific world to being included and then becoming a part of the research as an implementer of science. |
Frith et al. [54] | Pre–post | 1) Develop educational materials 2) Develop resource sharing agreements 3) Distribute educational materials 4) Tailor strategies | Adoption Acceptability | Knowledge test and survey | Two hundred four learners completed the module: 68% of learners scoring 100% in the post-module knowledge test. It was not possible to determine whether a behavior change had occurred as a result of completing the RTD module or whether this had translated to improved care in the management of RTD. Twelve learners completed the additional survey (8 OTs)—self-report outcomes indicate positive effects taking more responsibility (n = 8) in managing RTD, providing clearer information to patients (n = 4), and supplementing verbal information with written information. |
Kristensen and Hounsgaard [68] | Hermeneutic phenomenology | 1) Audit and provide feedback | Adoption Fidelity | OT medical record; daily self-reported recordings; focus group interviews | Audit and feedback methods proved useful for providing therapists with important information to evaluate and further the implementation process. Daily practice in both settings adapted to the clinical guidelines. Implementations of the standardized assessment tools (AMPS, A-ONE, COPM) seemed to be the most successful. |
Levac et al.1 [64] | Pre–post | 1) Develop educational materials 2) Identify and prepare champions 3) Make training dynamic 4) Provide ongoing consultation 5) Remind clinicians | Adoption Appropriateness | Focus group interviews; ADOPT-VR instrument; self-reported knowledge and skills survey | The therapist's intention to use VR did not change. Knowledge and skills improved significantly following e-learning completion (p = 0.001) and were sustained six months post-study. Below average perceived usability of the IREX (19th percentile) was reported. Lack of time was the most frequently reported barrier to VR use. A decrease in the frequency of perceived barriers to VR use was not significant (p = 0.159). Therapists reported that client motivation to engage with VR facilitated IREX use in practice but that environmental and IREX-specific barriers limited use. |
Levac et al.2 [53] | Pre–post | 1) Conduct educational outreach visits 2) Distribute educational materials 3) Remind clinicians | Adoption Fidelity | Self-report survey; focus group interviews | The KT intervention improved self-reported confidence about MLS use as measured by confidence ratings (p < 0.001). Therapists favored transferring skills from VR to real-life tasks over employing a more comprehensive MLS approach. Chart-Stimulated Recall indicated a moderate level of competency in therapists' clinical reasoning about MLSs following the intervention, with no changes following additional opportunities to use VR (p = .944). No behavior change for MLS use was noted (p = 0.092) on the Motor Learning Strategy Rating Instrument |
Luconi et al. [61] | Prospective cohort | 1) Assess for readiness and identify barriers and facilitators 2) Identify and prepare champions 3) Remind clinicians | Appropriateness Feasibility | My Guidelines Implementation Barometer (MGIB); questionnaire; comments; Information Assessment Method (IAM) | Satisfaction, relevance, and cognitive impact of delivered information varied across disciplines and recommendations. Agreement with the recommendations was high across disciplines. On average, three interdisciplinary recommendations (related to post-stroke depression, post-stroke fatigue, and patients' and caregivers’ learning needs) were rated the most relevant for at least one patient. Most clinicians would use the recommendations for a specific patient and expected health benefits by applying those recommendations. |
McCluskey and Middleton [59] | Pre–post | 1) Assess for readiness and identify barriers and facilitators 2) Audit and provide feedback 3) Conduct educational outreach visits | Adoption Feasibility | Administrative data (medical records) | Medical record audits found that teams delivered six or more outdoor journeys to 17% of people with stroke pre-intervention, rising to 32% by 12 months post-intervention. This change represents a modest increase in practice behavior (15%) across teams. The “Out-and-About Implementation Program” helped rehabilitation teams change their practice, implement evidence, and improve client outcomes. |
McCluskey et al.1 [60] | Pre–post | 1) Assess for readiness and identify barriers and facilitators 2) Build a coalition 3) Conduct educational outreach visits 4) Develop educational materials 5) Identify and prepare champions 6) Provide clinical supervision | Adoption Feasibility Fidelity | Administrative data; motor assessment scale; box and block test; nine hole peg test; motor activity log | Sixteen stroke participants were recruited (M = 15.3 months post-stroke, SD 11.9), and 6 CIMT programs were conducted over 24 months, compared with none pre-implementation. The behavior change program resulted in multiple CIMT programs being delivered safely and with fidelity. Capacity building, skill development, and preparation for the first CIMT program took many hours. |
McCluskey et al.2 [52] | Randomized controlled trial | 1) Assess for readiness and identify barriers and facilitators 2) Audit and provide feedback 3) Conduct ongoing training Distribute educational materials | Adoption Fidelity | Administrative data | At 12 months after implementing the behavior change program, 9% of audited experimental group stroke survivors received four or more outings during therapy compared with 5% in the control group (adjusted risk difference 4%, 95% CI [9 to 17], p = 0.54). The Out-and-About program did not change team or stroke survivor behavior. |
McEwen et al.1 [43] | Pre–post | 1) Conduct educational meetings 2) Conduct educational outreach visits 3) Develop educational materials 4) Facilitation 5) Provide technical assistance 6) Provide ongoing consultation 7) Recruit, designate, and train for leadership | Adoption Fidelity | Questionnaire; administrative data (medical records) | No charts showed evidence of CO-OP use at baseline, compared with 8/40 (20%) post-intervention. Post-intervention, there was a trend towards reduction in impairment goals, and significantly more component goals were set (z = 2.7, p = .007) |
McEwen et al.2 [51] | Pre–post | 1) Centralize technical assistance 2) Conduct educational outreach visits 3) Conduct ongoing training 4) Distribute educational materials 5) Identify and prepare champions 6) Promote network weaving 7) Remind clinicians | Adoption | Written tests; surveys | Participation in REPS was associated with an increase in stroke rehabilitation knowledge immediately following the program and at 6-month follow-up; participants reported positive practice changes following completion of the program and at the 6-month follow-up |
Moore et al. [50] | Time series | 1) Alter incentive allowance structures 2) Audit and provide feedback 3) Conduct local consensus discussions 4) Conduct ongoing training 5) Develop academic partnerships 6) Develop and organize quality monitoring systems 7) Develop educational materials 8) Distribute educational materials 9) Fund and contract for the clinical innovation 10) Identify and prepare champions 11) Involve executive boards 12) Mandate change 13) Promote adaptability 14) Provide clinical supervision 15) Provide ongoing consultation 16) Purposefully reexamine the implementation 17) Stage implementation scale-up 18) Tailor strategies 19) Use advisory boards and workgroups 20) Use data experts 21) Use train-the-trainer strategies | Adoption Penetration Sustainability | Surveys | Survey data indicate the BRAI resulted in a significant increase in the use of EBPs to make clinical decisions and justify care. Survey participants reported a substantial increase in the use of outcome measures in 2012 (74%) and 2015 (91%) and EBP in 2012 (62%) and 2015 (82%). In 2012, significant differences (p = .01) in the effect of the BRAI on practice were identified between therapists who were directly involved in the project and interventions compared with uninvolved therapists. In 2015, no significant differences existed between involved and uninvolved therapists. After 6 years of sustained implementation efforts, the BRAI expedited the adoption of EBPs throughout a large system of care in rehabilitation. |
Petzold et al [49] | Pre–post | 1) Assess for readiness and identify barriers and facilitators 2) Conduct educational outreach visits 3) Conduct ongoing training 4) Develop educational materials 5) Distribute educational materials 6) Make training dynamic 7) Remind clinicians 8) Tailor strategies | Adoption Feasibility Acceptability | Knowledge questionnaire; EBP self-efficacy scale; clinician/work environment variables measure; patient case vignettes | A significant improvement in knowledge of best practice unilateral spatial neglect management (p < 0.000) and evidence-based practice self-efficacy in carrying out evidence-based practice activities (p < 0.045) post-intervention. |
Salbach et al. [48] | Process evaluation | 1) Assess for readiness and identify barriers and facilitators 2) Capture and share local knowledge 3) Conduct educational outreach visits 4) Create a learning collaborative 5) Distribute educational materials 6) Facilitation 7) Identify and prepare champions 8) Remind clinicians | Adoption | Patient outcomes; self-report checklists | Facilitated KT intervention was associated with improved implementation of sit-to-stand (p = 0.028) and walking (p = 0.043) training. In contrast, the passive KT intervention was associated with improved implementation of standing balance training (p = 0.037) after adjusting for clustering at patient and provider levels and covariates. Facilitated KT intervention was unsuccessful in improving the integration of 18 treatments concurrently. The facilitated approach may not have adequately addressed barriers to integrating numerous treatments simultaneously and complex treatments that were unfamiliar to providers |
Schneider et al. [58] | Pre–post | 1) Assess for readiness and identify barriers and facilitators 2) Conduct educational meetings 3) Create a learning collaborative 4) Develop a formal implementation blueprint | Fidelity | Observations; recorded data | Outcomes were measured across n = 15 classes (n = 30 patients). Between baseline and 12 months, the mean proportion of practice time per class increased by 52% (95% CI 33–70; p < 0.001), and the mean no. of repetitions per practice time increased by 5.1 reps/min (95% CI 1.7–8.4; p < 0.01). Between baseline and 18 months, the mean proportion of practice time per class increased by 53% (95% CI 36–69; p < 0.001), and the mean no. of reps per practice time increased by 3.9 reps/min (95% CI 1.9–5.9; p < 0.001). Providing professional development was associated with increased intensity of practice in an inpatient, upper limb rehabilitation class. The increase was maintained 6 months later. |
Stewart et al. [57] | Pre–post | 1) Assess for readiness and identify barriers and facilitators 2) Audit and provide feedback 3) Conduct ongoing training 4) Create a learning collaborative 5) Provide ongoing consultation | Adoption Fidelity | Medical record audit; behavioral mapping; observations | Post-intervention, no. of participants with practice books increased from 1 to 6 (OR = 11, 95% CI = (0.9, 550.7)), but this change was not statistically significant (p = 0.069). There was no change in median repetitions recorded (r = 0.00, 95% CI = (− 0.4, 0.4), p = 1.000) or observed active practice (r = − 0.02, 95% CI = (− 0.4, 0.4), p = 0.933). The staff behavior change intervention led to increasing use of practice books but no statistically significant difference in adoption of practice books or intensity of active practice. |
Terio et al. [65] | Process evaluation | 1) Audit and provide feedback 2) Change physical structure and equipment 3) Conduct educational outreach visits 4) Conduct ongoing training 5) Develop a formal implementation blueprint 6) Facilitation 7) Involve patients/consumers and family members 8) Promote adaptability 9) Provide local technical assistance 10) Purposefully reexamine the implementation | Acceptability Fidelity | Logbooks; semi-structure interviews | In 11 out of 14 cases, the clients were compliant with the intervention. However, challenges such as technical problems were reported. The target of conducting 16 phone calls for each client was achieved to 74%. Mechanisms contributing to the implementation of the intervention included engaged facilitators and motivated participants. Challenges in client recruitment and poor information dissemination were some of the mechanisms impeding the implementation. Several mediators in the process drove the project forward, including strong facilitation and motivated participants. |
Tetteroo et al. [66] | Participatory action research | 1) Conduct educational meetings 2) Conduct ongoing training 3) Make training dynamic 4) Provide local technical assistance 5) Provide ongoing consultation 6) Purposefully reexamine the implementation | Adoption Acceptability | Semi-structure interviews; questionnaires; observation notes; usage logs | TagTrainer system was used in 34 therapy sessions, 20-group, 14-individual. In general, therapists reported moderate to high self-efficacy, except for their perceived ability to resolve technical problems with TagTrainer (M = 32.5, SD = 28.7). In addition, they reported significantly higher levels of self-efficacy (t(3)=4.899, p = 0.016) for using TagTrainer in individual therapy sessions (M = 80.0, SD = 21.6), compared with group therapy sessions (M = 60.0, SD = 28.3). The credibility (M = 19.5, SD = 3.11) and expectancy (M = 13.9, SD = 5.22) ratings that the therapists gave for the TagTrainer system show that they find it to be credible for arm–hand rehabilitation but are neutral in respect to the expected effectiveness of the system for the improvement of arm–hand performance. |
Vratsistas-Curto et al. [47] | Pre–post | 1) Assess for readiness and identify barriers and facilitators 2) Audit and provide feedback 3) Conduct cyclical small tests of change 4) Conduct educational meetings 5) Conduct ongoing training 6) Distribute educational materials 7) Provide ongoing consultation | Fidelity | Medical records; administrative data | Between the 1st & 4th audits (2009 & 2013), 20 of the 27 areas targeted (74%) met or exceeded the minimum target of 10% change. Practice areas that showed the most change included sensation screening (+ 75%) and rehabilitation (+ 100%), neglect screening (+ 92%), and assessment (100%). Some target behaviors showed a drop in compliance, such as anxiety and depression screening (− 27%) or little or no overall improvement, such as patient education about stroke (6% change). Audit feedback and education increased the proportion of inpatients with stroke receiving best practice rehabilitation in some but not all practice areas. |
Willems et al. [67] | Pre–post | 1) Inform local opinion leaders 2) Involve patients/consumers and family members 3) Obtain and use patients/ consumers and family feedback 4) Prepare patients/consumers to be active participants 5) Promote adaptability 6) Recruit, designate, and train for leadership 7) Remind clinicians 8) Stage implementation scale-up 9) Use train-the-trainer strategies | Adoption | Questionnaires | After the knowledge broker (KB) intervention, more patients (48%; n = 217) reported at least some encouragement by HPs to be physically active than before (26%; n = 243, p < 0.000). HPs (n = 288) on average reported encouraging patients more often after the intervention, but this difference was significant only for occupational therapists and KBs. Based on the patient’s reports of HP behavior, the KB intervention appears effective since more patients felt encouraged to be physically active after the intervention than before. |