Skip to main content

Testing an implementation strategy bundle on adoption and sustainability of evidence to optimize physical function in community-dwelling disabled and older adults in a Medicaid waiver: a multi-site pragmatic hybrid type III protocol



In partnership with a state Medicaid home and community-based waiver program, this study tests implementation strategies for adoption and sustainability of an evidence-based intervention to support disabled and older adults who have difficulty with physical function and daily living tasks. A multi-level implementation strategy bundle will be directed at relationship, coalition, and team building; readiness to implement, leadership, and clinician attitude toward evidence assessments; intervention and facilitation training; interdisciplinary coordination; facilitation; and audit and feedback to support practice change.


Knowledge-to-Action model underpins this 2-arm, 3-year pragmatic mixed method randomized hybrid type III trial in 18 waiver program sites in Michigan. Data will be collected on sites, 775 clinicians (registered nurses, occupational therapists, social workers), and 15,000 disabled and older adults. Consolidated Framework for Implementation Research guides examination of site, clinician, and beneficiary characteristics; clinician attitude and self-efficacy; leadership and readiness to implement; and intervention impact on beneficiary outcomes. Sites will be randomized to either usual waiver care with internal facilitation of the bundle of implementation strategies or usual waiver care with both internal and external facilitation of the bundle. Primary outcomes are site-level adoption and sustainability over 12 months, and intervention effects on these outcomes are hypothesized to be mediated by clinicians’ attitude and self-efficacy. At the beneficiary level, by addressing the individual’s capabilities and home environment, the intervention is hypothesized to improve secondary outcomes of activities of daily living, pain, depression, falls, emergency department visits, and hospitalizations. Baseline site readiness and leadership and stages of implementation at 6 months will be explored as potential moderators. Linear mixed effects models will be used to test intervention effects on primary outcomes, with bias-correcting analytic strategy in mediation analyses. Generalized linear mixed effects modeling will be employed for the analysis of intervention effects on secondary outcomes.


Synthesizing findings within and across the sites, we will specify how leadership, readiness for change, and level of facilitation enhance capacity for adoption and sustainability of an evidence-based intervention in an under-resourced Medicaid setting that cares for disabled and older adults.

Trial registration, NCT03634033. Registered 16 August 2018.

Peer Review reports


There are 962 million people aged 60 and older worldwide [1], and 42% in the USA report problems with physical function, which can lead to difficulty with daily tasks and nursing home placement [2,3,4,5]. Interventions that optimize physical function to support aging-in-place are needed, particularly in settings that care for disabled and older adults. We propose to address this urgent need through the testing of an implementation strategy bundle on adoption and sustainability of an adapted intervention [6] that improves adults’ ability to conduct daily tasks [7,8,9,10]. The implementation strategy bundle includes relationship, coalition, and team building; readiness to implement, leadership, and clinician attitude toward evidence assessments; intervention and facilitation training; interdisciplinary coordination; internal and external facilitation (IF, EF); and audit and feedback. Multi-level approaches are known to support incremental single strategy effects, as well as a synergistic effect from all the strategies [11]. Thus, we test two topics of import to the field of implementation science - adoption and sustainability [12,13,14,15,16] - and explore mechanisms of action [17] to support uptake, while implementing a complex multi-component intervention in a disabled aging population. Real practice change and improved outcomes will not occur without adoption and sustainability [13].

Model guiding implementation

The Knowledge-to-Action [18, 19] model underpins our approach (see Fig. 1) to implementation. Evidence was created and efficacy established by others [7,8,9,10]. In prior work, the evidence-based intervention was selected, barriers to implementation were identified, and the intervention was adapted to the setting and population [6]. This study will evaluate adoption and sustainability of the intervention and beneficiary outcomes.

Fig. 1

Applying the Knowledge-to-Action model to the intervention creation, testing, setting/population, adaptation, barriers, and knowledge use


The overall goal of the study is to implement evidence in a waiver program to improve the ability of disabled or older adults to perform daily tasks. Objectives for this study are to test the deployment of an implementation strategy bundle on adoption and sustainability of the intervention and to examine the effect of the intervention on beneficiary outcomes.


Aim 1: To test the effects of an implementation strategy bundle with IF alone versus an implementation strategy bundle with IF + EF with respect to the site-level outcomes of adoption and sustainability (primary) and beneficiary-level outcomes of activities of daily living/instrumental activities of daily living (ADL/IADLs), pain, depression, falls, emergency department (ED) visits, and hospitalizations (secondary) over the next 12 months.

Aim 2 mechanism-of-action: To determine whether the effects of IF + EF versus IF on primary outcomes are mediated by clinician attitude or self-efficacy at 9 months.

Aim 3: To benchmark the effects of IF and IF + EF on beneficiary outcomes following implementation as compared to pre-intervention.

Exploratory aim 4: To compare the primary and secondary outcomes within 12 months, potential mediators at 9 months, and baseline leadership and readiness for sites with SIC of > 50% versus sites with SIC of < 50% at 6 months in each arm.

Exploratory aim 5: To explore whether baseline site leadership and readiness moderate the impact of IF + EF compared to IF on primary and secondary outcomes within 12 months in order to determine which sites may require facilitation that is more intensive.

Exploratory aim 6: To evaluate clinician satisfaction at 1 month and the cost of implementation and policy impact for IF and IF + EF at 12 months.

Study design

Testing in this study will occur in a 2-arm, 3-year pragmatic hybrid type III [20, 21] mixed method randomized trial design [22,23,24]. Hybrid III trials examine implementation strategy effect (primary) while gathering data on the intervention outcomes (secondary) [21].

Arm 1 includes usual waiver care and an implementation strategy bundle with IF.

Arm 2 includes all components of Arm 1 plus additive EF.


The setting is 18 Medicaid home and community-based (MI Choice) waiver sites in Michigan. The waiver supports low-income, nursing home eligible, disabled, and older adults in the community, providing 19 services (e.g., personal care and meals) through registered nurse (RN) and social work (SW) case management.

Participants, recruitment, contracts, and consents

We have three levels of participation: the site, the clinician, and Medicaid beneficiary, with individualized recruitment plans for each. Figure 2 shows a CONSORT-like flow chart of recruitment by site, clinician, and beneficiary.

Fig. 2

CONSORT-like figure of recruitment and participant status to be tracked. a Site CONSORT-like figure. b Clinician study status CONSORT-like figure. c Beneficiaries study status CONSORT-like figure

Sites. We recruited 18 waiver sites in Michigan, have letters of support, and anticipate all will be contracted.

Clinicians. We will recruit clinicians via a flyer sent by email. In addition, the IF at each site will encourage clinicians to complete the training and use the intervention by providing verbal and written prompts during routine communications (email or face-to-face) and at staff meetings. We expect 80% (620 of 775) of clinicians employed at the sites to consent electronically prior to online training, complete the training, and use the intervention with beneficiaries.

Beneficiaries. The trained clinicians will recruit beneficiaries during a home visit using an algorithm that was designed during prior work and a cognitive aid on a pocket card that will be provided during training. The algorithm is a set of rules the clinician follows to use critical thinking when examining the beneficiary’s need or potential benefit from the intervention. We expect 60% (9,000 of 15,000) of beneficiaries to not opt-out and receive the intervention.

Inclusion criteria

Sites. Eighteen sites under contract by the Michigan Medicaid program which provide waiver care and utilize an electronic health record (EHR) from EHR company are included. Excluded are 2 sites which do not use the EHR.

Clinicians. Clinicians who are employed by the sites who consent and are trained in use of the intervention are included. Excluded are clinicians who do not consent and/or do not complete the training.

Beneficiaries. Beneficiaries who do not opt-out and who may need and benefit from the care will receive the intervention. Excluded are beneficiaries who opt-out.

Evidence-based intervention

The evidence-based intervention reduces the effect of problems with physical function among low-income older adults living in the community by addressing an individual’s capabilities and the home environment [7,8,9,10]. The intervention is a 16-week structured program delivered by occupational therapists (OTs) who conduct 6 home visits and provide assistive devices, RNs who conduct 4 home visits, and a handyman who provides home alterations (e.g., installs devices, environmental modifications, and home repair). The team provides consultation with older adults to help the individuals identify daily activity goals (e.g., taking a shower and walking to the bathroom), evaluates barriers to achieving those goals, and attains outcomes collaboratively. OTs assist older adults to carry out ADLs, IADLs, and discretionary activities that are challenging at home such as functional mobility, meal preparation, bathing, and dressing. RNs target underlying issues that influence ADLs and IADLs, such as pain reduction, improvement in mood, fall prevention, incontinence management, and medication review and management with the help of a pharmacist. In prior work, the intervention was adapted [6] to fit the waiver setting and population: SWs were added to address social and emotional needs; the number, type, and timing of clinician visits were tailored to the beneficiary’s need; RNs were trained in medication management; and training and documentation modes were modified to fit the setting.

Guiding framework

Consolidated Framework for Implementation Research [25, 26] supports our approach to practice change (see Fig. 3). We will examine characteristics (site, clinician, and beneficiary), clinician attitude and self-efficacy, inner setting [27] leadership and readiness to implement, and outer setting [28] policy, as each may impact adoption and sustainability.

Fig. 3

Consolidated Framework for Implementation Research (CFIR) factors influencing implementation strategy bundle and outcomes

Implementation strategies

We will deploy a 9-component implementation strategy bundle (see Table 1). As few studies focus on sustainability of implementation efforts in knowledge translation during implementation, we examine the mechanisms of action of awareness, commitment, confidence, and trust [17] during implementation (see Fig. 4). Materials; procedures; providers; mode, where provided; duration; intensity and dose; and tailoring, if needed, for each strategy are presented. We also delineate which strategies we expect to trigger the mechanisms of action.

Table 1 Implementation strategies deployed, domains affected, definition, actors, actions/target of actions, temporality, dose, and outcome affected
Fig. 4

Relationships among mechanisms-of-action context, mechanism category, and expected outcome of implementation strategy

Strategy 1

We will form both informal and formal relationships to conduct this study. We expect this strategy to prompt the mechanisms of action of trust (see Fig. 4).

Informal relationships. Informal relationships were built between the sites, the investigator, and the state policymakers during 20 years of prior work. During this study, the investigator will continue to interact with the policymakers on a monthly basis, updating on study progress via a brief email (< 120 words); with the sites on a quarterly basis, providing a 1-h continuing education unit (e.g., incontinence, pain management); and with both by attending the 1-h face-to-face meeting when requested.

Formal relationships. For this study, an agreement, memorandum of understanding, or contract (site and university, state and university, and EHR company and university) will be executed prior to data collection and last for the duration of the study. The roles and responsibilities of each partner (site, investigator, state, and EHR company), a data use agreement, and a timeline for the study will be delineated. A key component of the success of the study is the IF. Delineated in the contract is the selection of the IF by the investigator with guidance by the site managers. The IFs are supervisors/employees that work for the waiver site and will conduct facilitation at their site as “Champions.” In addition, we will utilize a waiver employee as the EF, who was an early adopter of the intervention in prior work, as a “Super-Champion” for arm 2.

Strategy 2

Data on readiness to implement, to prompt the mechanisms of action of awareness (see Fig. 4), and leadership, to examine the need to do intermittent booster training for facilitators, will be collected from clinicians in an online 10-min survey distributed prior to training. Within the readiness assessment, we focus on subscales of training, pressure for change, and attributes of growth, efficacy, and influence. Within the leadership assessment, we focus on subscales of proactive, knowledgeable, supportive and perseverant leadership.

Strategy 3

Data on clinician attitude and self-efficacy will be collected in an online 10-min survey distributed prior to training and at 9 months to examine changes over time. In the self-efficacy assessment, we examine if the training prompts improvement. In the attitude assessment, we focus on openness, appeals of the new intervention, willingness to using the intervention, and conflict between clinical experience and research results (divergence). We expect this strategy to prompt the mechanisms of action of confidence during implementation (see Fig. 4).

Strategy 4

Clinicians will be trained in the intervention, and supervisors (IFs and EF) will be trained in facilitation. We expect this strategy to prompt the mechanisms of action of confidence during implementation (see Fig. 4).

Clinicians. RNs, SWs, and OTs will complete a 5.5-h, 20-module, 6-article online training program prior to providing the intervention to beneficiaries. Clinicians are expected to complete the training in its entirety within a 30-day time period and will receive a continuing education certificate upon completion. Modules include training on person-environment fit, person-centered care, and building collaborative relationships and self-efficacy. In addition, other modules include readiness-to-change, motivational interviewing, cognitive behavioral therapy, problem solving, brainstorming, therapeutic communication, reflection, and interdisciplinary coordination. An additional file shows this in more detail (see Additional file 1). The role of each discipline and care to be provided are covered, with in-depth medication management instructions, a case study, and handouts to use during care. A pocket card cognitive aid (see Additional file 2) will be provided to assist clinicians in tailoring care to the beneficiary’s needs and desires. We may need to tailor and/or boost education to a specific site and/or group of clinicians on use of the intervention, with brief (20–30 min) online training modules, after we examine fidelity to the intervention monthly (months 2 to 12).

Facilitators. IFs and EF will complete a 1.5-h, 10-module online training program on facilitation. Modules for the IF include evidence supporting implementation strategies, the facilitator role, quality improvement, problem solving, how to provide feedback, use of reflection, counseling, motivational interviewing, and remediation. An additional file shows this in more detail (see Additional file 3). An implementation toolkit will be provided, and the plan will be reviewed. The module for the EF includes a train-the-trainer approach for facilitation of IFs. We may need to tailor and/or boost education with some or all of the IFs if data on implementation strategy fidelity show implementation is not occurring as expected.

Strategy 5

IFs will form a coalition to support implementation. Monthly for 1 h for 12 months, the IFs will have an online collaborative meeting hosted by the investigator. The purpose of the coalition is to build capacity among the IFs by sharing best practice use of the implementation strategies. The investigator will provide feedback on the number of clinicians trained and fidelity to implementation strategies and the intervention. Discussion will then occur among the group on use of implementation strategies to improve implementation. We expect this strategy to prompt the mechanisms of action of commitment during implementation (see Fig. 4).

Strategy 6

IFs will prompt and lead intermittent interdisciplinary coordination among the clinicians (RN, SW, and OT), as needed, to promote teamwork, brainstorming, and problem solving to support beneficiary goal attainment. We expect interdisciplinary coordination will occur via phone, email, and/or face-to-face for 15 min per month for each beneficiary cared for. However, this is dependent upon the complexity of the beneficiary’s care and is tailored in time and intensity based on the needs of the person. We expect this strategy to prompt the mechanisms of action of commitment during implementation (see Fig. 4).

Strategy 7

IFs will facilitate clinician implementation at each site in arm 1 and 2, and the EF will facilitate IFs in arm 2 tailored to a site’s needs. We expect that an IF will spend between 2 to 5 h a week per month (months 2 to 12) facilitating implementation via face-to-face, email, or phone. Also, we expect that an IF will spend 1 h a month for 12 months interacting with IFs from other sites to learn best practices for implementation. Once the clinicians are trained and using the intervention with the beneficiaries, the IFs will be absorbed within usual IF duties. IF actions include training, coaching, consultation, supervision, modeling, problem solving, and providing feedback, supporting, instructing, demonstrating, and assisting with evaluation. We will use an audit tool with these actions to examine facilitation monthly (months 2 to 12) (see Additional file 4). We expect this strategy to prompt the mechanisms of action of commitment during implementation (see Fig. 4).

Strategy 8

An audit of implementation strategy and intervention fidelity (see Additional file 5) with feedback will occur to assist IF’s work with clinicians in arms 1 and 2. The audit will be conducted by the study team, and feedback will be provided to the IFs weekly via email and reviewed monthly during the online coalition meeting for both arm 1 and 2. We expect this strategy to prompt the mechanisms of action of confidence, commitment, and trust during implementation (see Fig. 4).

Strategy 9

The EFs work with and support the IFs in arm 2. Feedback will be provided to the EF weekly via email by the study team, with a request to provide external facilitation as needed when implementation is not occurring as expected. Similar to IF actions, EF actions include training, coaching, consultation, supervision, modeling, problem solving, and providing feedback, supporting, instructing, demonstrating, and assisting with evaluation. We will use an audit tool with these actions to examine facilitation monthly (months 2 to 12) (see Additional file 4). We expect this strategy to prompt the mechanisms of action of commitment during implementation (see Fig. 4).


In this study, we will measure site, clinician, and beneficiary level data (see Table 2). Site measures include characteristics, leadership, readiness to implement, and implementation completion. Clinician measures include characteristics, attitude toward evidence-based care, self-efficacy, and training completion. Beneficiary measures include characteristics and outcomes. We also measure fidelity to implementation strategies and the intervention. Measures are shown in Table 2, and the data collection plan is shown in Table 3.

Table 2 Measures deployed in study by domain, construct, concept, aim, who/how measured, and who collects
Table 3 Data collection activity, month, where obtained from (state, clinicians, IF, EF, EHR), and what obtained


Characteristics will be collected at baseline. Site characteristics include size; quality scores; number of supervisors, clinicians, and beneficiaries; and years in the waiver. Clinician characteristics include age, race, gender, discipline, and years working at waiver program site. Beneficiary characteristics include age, gender, and race.


We use 6 tools in this trial. Implementation Leadership Scale (ILS) is a 12-item, 4-subscale (Cronbach .93–.97) tool that assesses leadership behaviors and actions and will be collected from clinicians at baseline [29, 30]. Organizational Readiness for Change (TCU-ORC) is a 124-item tool with subscales (needs of clinician, program, training, pressure for change, resources, and attributes) and 3-open ended questions and will be collected at baseline from clinicians (Cronbach .88) [31]. Evidence-Based Practice Attitude Scale (EBPAS) is a 50-item tool with 12 subscales with 4 constructs (openness, appeal, requirements, and divergence) and will be collected from clinicians at baseline and 9 months [32]. General Self-Efficacy (GSE) is a 10-item tool (Cronbach’s alpha .79–.90) and will be collected from clinicians at baseline and 9 months [33]. Minimum Data Set-Home Care (MDS-HC) is a self-reported, person-centered assessment for the collection of beneficiary minimum essential nursing data, with reliability and validity, and used in the waiver since 1993 and will be collected in the EHR as part of usual care. We obtain variables for our study from the MDS-HC [34]. Stages of Implementation (SIC) is an 8-stage tool that examines completion of activities during implementation, adoption, and sustainability and will be collected from the IFs monthly (months 2 through 12) [35, 36].


We will examine fidelity to the implementation strategies and the intervention.

Fidelity to the implementation strategies. An implementation strategy fidelity checklist measures number of consented clinicians, number of completed readiness and leadership assessments, number of clinicians trained, number of IF actions taken, and number of interdisciplinary coordination meetings and will be collected monthly (months 2 through 12). Based on a review of the literature, IF actions will be coded as training, coaching, consultation, supervision, modeling, problem solving, providing feedback, supporting, instructing, demonstrating, and assisting with evaluation. Clinician and facilitator training acceptability will be evaluated by number of clinicians and facilitators who completed training. Costs of implementation will be estimated based on the time spent training and increased number of home visits. Policy impact will be examined in the waiver contract.

Fidelity to the intervention. An intervention fidelity checklist will be used to examine the EHR for those beneficiaries who consent and receive care from trained clinicians. This will include OT assessments at baseline; SW mood assessment, as needed; and RN medication reviews and will be collected monthly (months 2 through 12). In addition, within the person-centered service plan, we will examine the presence of the beneficiary’s desires and needs being documented. Within progress notes documentation, we will examine use of brainstorming, problem solving, and role modeling.


Primary outcomes are implementation strategy impact on adoption and sustainability of the intervention. Stages of implementation completion will be derived via 3 SIC scores. First, the number of stages completed is a simple count of progression through stages; the score is the last stage in which at least one activity was performed. Second, time spent in each stage is calculated by taking the difference between the date of completion of the first activity in the stage and the date of completion of the last activity in the same stage. Skipped activities are not included. If a site skips the last activity in a stage and completes an activity in a subsequent stage, they automatically move to the subsequent stage. However, if they later complete the skipped activity, the duration score is adjusted for the original (earlier) stage to include the activity. Third, the date of completion of all stages is logged within stage 8. For sites that choose to discontinue implementation at any point in the process, the date is logged in the furthest stage that the site enters. In the case where data are summarized before the stage is complete but a site has not discontinued implementation, the site data are treated as being censored, just as it would in a standard time-to-event or survival analysis [37]. Proportion of activities completed is calculated as the number of activities completed divided by the number of possible activities in each stage. Activities in each stage are ordered based on their logical progression up to the last activity the site completes in the stage or completion of the final activity in the stage. Achievement of either activity indicates completion of that stage. Secondary outcomes are intervention effects on beneficiary ADLs, IADLs, pain, depression, falls, ED visits, and hospitalizations.

Sample size and power calculation

Site sample size was based on the number of available sites, with 9 randomized to each arm. Assuming correlation of 0.7 between pairs of 11 repeated measures, the adjusted effect size of 1.2 was detectable with power of 0.80 or greater in two-sided tests at 0.05 level of significance. The effect size is expressed as Cohen's d, difference between trail arm means in standard deviation units. If observed differences are smaller than 1.2 of the standard deviation, statistical significance will not be reached, then the estimate of the effect size will be obtained. For beneficiary outcomes, we assumed 750 beneficiaries per site and conservative intraclass correlation coefficient of 0.01 to obtain the design effect factor of 8.49. In the comparison of the trial arms, the adjusted sample size is 795 per trial arm, which allows to detect effect size as small as 0.14. As larger effects were seen in the prior studies, a third of the standard deviation or larger is often used as a threshold for clinical significance [38, 39]. Small effects could be detected as statistically significant, and the study is powered to detect any meaningful differences between IF and IF + EF on beneficiary outcomes. The tests of mediation effects (aim 2) will have a greater power as further reduction in error variance will occur due to controlling for the mediator. Outcomes (aim 3) are benchmarked against a prior period and have no hypotheses; thus, power considerations are not applicable. Similarly, exploratory aims have no associated hypotheses.


Randomization will occur at the site level. The number of beneficiaries and quality assessment scores will be used to block the 18 sites into a pair of similar attributes. A coin will be flipped to determine arm assignment of each pair. We expect to start one pair each month until all are underway.


Clinicians and beneficiaries will be blinded to arm assignment. Sites, IFs, and research assistants will not be blinded. Success of blinding will be assessed at study end.

Study procedures

After IRB approval, data use agreements will be obtained and contracts will be executed with the sites, the state, and the EHR company. To randomize, the sites’ number of beneficiaries and summed quality data scores (2015–2017) will be obtained from the state and the investigators will match site pairs and assign the trial arm. If the clinicians consent, characteristics, ILS, ORC, GSE, and EBPAS will be collected via survey (baseline). Waiver directors and investigator will select IFs. Clinicians, IFs, and EFs will be trained. The intervention will be provided to beneficiaries and documented in the EHR (12 months post-implementation). Monthly for 11 months following implementation, the SIC (via research assistant phone interview) and intervention fidelity checklist (via project manager chart audit) will be collected. Clinician EBPAS and GSE will be collected again via survey (9 months). We will obtain beneficiary assessment data prior to the intervention (last assessment before implementation), after the intervention (first assessment within 12 months post-implementation), and any assessments that occurred between the two time points after completion of implementation via a data use agreement. Analysis will occur and reports will be written.

Research assistant training

Research assistants will complete human subject training and orientation prior to collecting data or interacting with IFs. As part of orientation, a job description, which includes the role and responsibilities, will be reviewed and signed; a position manual, the manual of procedures, the data safety monitoring plan and board charter, the study protocol, the literature supporting project evidence and approach, and the quality assurance-monitoring plan will be reviewed. Research assistants will conduct dummy data collection until performed correctly, prior to data collection for the trial. Once trained, the research assistant will participate in team meetings and quality assurance activities and complete a weekly report of activities and/or issues that arise.

Analytic approach

Preliminary analysis of the distributions of outcomes, mediators, and potential covariates will be assessed, outliers will be investigated by inspecting the residuals, and models described below will be fit with and without outliers to examine their influence on the results. Analyses will be implemented in SAS 9.4.

Aim 1 primary outcomes (adoption and sustainability) will be analyzed using a linear mixed effects model with repeated measures (SIC and EHR 12-months; IF 9-months). Covariates will include trial arm, randomization variables, and time entered as a class variable to model potentially non-linear patterns. We will use a general linear model to analyze training fidelity. Log-rank test and Cox proportional hazard modeling will be employed to analyze time spent in each implementation stage. Tests of significance of the coefficient of the trial arm variable will yield the formal test of hypothesis for the effect of IF + EF compared to IF alone. Analysis of beneficiary outcomes will employ a generalized linear model with appropriately distributed errors and the random effect of site added to account for nesting of individuals within sites. For counts of falls and health service use, Poisson error distribution will be specified. Alternatively, a zero-inflated Poisson or negative binomial model based on the distribution of the counts will be fit. Explanatory variables, including trial arm, will be evaluated as predictors of zero inflation (whether or not the count is zero) and as predictors of the magnitude of the count when it is not zero. Prior MDS will be used to obtain baseline data for each outcome, which will be included as a covariate to explain the variation in post-intervention outcomes.

Aim 2 determines whether the effects of IF + EF compared to IF alone on adoption and sustainability are mediated by clinician attitude or self-efficacy at 9 months. The model fit will be at the clinician level with site as a random effect to account for nesting of clinicians within sites. To test for mediation, trial arm will be treated as the independent variable, with each potential mediator (one at a time) tested for their effect on the outcome. A bias-corrected bootstrapping analytic strategy [40, 41] will be used to estimate confidence intervals around the indirect effect of the trial arm on the outcome variable through the mediator. To establish mediation, the 95% confidence interval around the indirect must not include 0.

Aim 3 benchmarks the effects of IF + EF compared to IF alone on beneficiary outcomes following implementation as compared to pre-intervention. Analysis will use 2 repeated measures of beneficiary outcomes, prior to and 12 months after the intervention. Time by trial arm interaction will be included. The least square means according to the interaction term will be output from this model, and differences from before and after the intervention will be evaluated for each arm to gauge the magnitude and meaning of improvements.

Exploratory aims. Aim 4 compares potential mediators at 9 months and baseline leadership and readiness for sites with SIC (> 50% to < 50%) at 6 months by arm. General or generalized linear models (as appropriate based on outcome or potential mediator distribution) will include the binary indicator of whether SIC at 6 months (> 50% or < 50%) in interaction with the trial arm variable. Differences between least square means according to SIC level will be tested within each arm, and the effect sizes will be estimated. Aim 5 explores whether baseline site leadership and readiness moderate the impact of IF + EF compared to IF alone on primary and secondary outcomes within 12 months in order to determine which sites may require facilitation that is more intensive. A model similar to aim 1 will be modified to include trial arm by potential moderator interaction. Effect sizes that correspond to the interaction term will be estimated. Aim 6 evaluates clinician satisfaction at 1 month and the cost of implementation and policy impact for IF + EF compared to IF alone at 12 months. We will use descriptive statistics to summarize satisfaction with training and implementation cost. We will use qualitative thematic analysis to analyze the surveys. We will report on the policy in the state contract.


This trial sets an ambitious agenda to explore ways to adopt and sustain evidence across a Medicaid program. There is a significant amount of work that presents challenges and opportunities. First, implementation and evaluation with a hybrid III design in an under-resourced real-world setting force a delicate balance of study design and voluntary participation. Our work with sites in a standardized manner encourages local adaptations to optimize implementation. Second, our selection of volunteer sites may limit heterogeneity in our sample. However, we expect to learn a great deal about stages of implementation to inform this project’s adoption and sustainability that may be generalizable to other settings or populations. Third, our measurement approach involves significant effort in data collection of sites, clinicians, and beneficiaries with surveys, interviews, and extraction of secondary administrative and clinical data across disparate clinician contexts. The challenges of data collection and management may be outweighed by the opportunity to generate a broader understanding in Medicaid settings to facilitate effective use of evidence in under-resourced, complex environments.



Activities of daily living




Evidence-Based Practice Attitude Scale


Emergency department


External facilitation/facilitator


Electronic health record


General Self-Efficacy


Instrumental activities of daily living


Internal facilitation/facilitator


Implementation Leadership Scale


Minimum Data Set for Home Care


Organizational Readiness for Change


Occupational therapist


Registered nurse


Stages of Implementation Completion


Social worker


  1. 1.

    United Nations, Department of Economic and Social Affairs, Population Division. World population prospects: the 2017 revision, volume 1: comprehensive tables. 2017. (ST/ESA/SER.A/399).

    Google Scholar 

  2. 2.

    Manton KG, Gu X, Lamb VL. Change in chronic disability from 1982 to 2004/2005 as measured by long-term changes in function and health in the U.S. elderly population. PNAS. 2006;103:18374–9.

    CAS  Article  Google Scholar 

  3. 3.

    Martin LG, Freedman VA, Schoeni RF, Andreski PM. Trends in disability and related chronic conditions among people ages fifty to sixty-four. Health Aff. 2010;29:725–31.

    Article  Google Scholar 

  4. 4.

    Castora-Binkley M, Meng H, Hyer K. Predictors of long-term nursing home placement under competing risk: evidence from the health and retirement study. J Am Geriatr Soc. 2014;62:913–8.

    Article  Google Scholar 

  5. 5.

    Seeman TE, Merkin SS, Crimmins EM, Karlamangla AS. Disability trends among older americans: national health and nutrition examination surveys, 1988-1994 and 1999-2004. Am J Public Health. 2010;100:100–7.

    Article  Google Scholar 

  6. 6.

    Spoelstra S, Sikorskii A, Gitlin L, Schueller M, Kline M, Szanton SL. Dissemination of CAPABLE model of care in a Medicaid waiver program to improve physical function. J Am Geriatr Soc. 2018;67:363–70.

    PubMed  PubMed Central  Google Scholar 

  7. 7.

    Szanton SL, Wolff JW, Leff B, Thorpe RJ, Tanner EK, Boyd C, et al. CAPABLE trial: a randomized controlled trial of nurse, occupational therapist and handyman to reduce disability among older adults: rationale and design. Contemp Clin Trials. 2014;38:102–12.

    Article  Google Scholar 

  8. 8.

    Szanton SL, Thorpe RJ, Boyd C, Tanner EK, Leff B, Agree E, et al. Community aging in place, advancing better living for elders (CAPABLE): a bio-behavioral-environmental intervention to improve function and health-related quality of life in disabled older adults. J Am Geriatr Soc. 2011;59:2314–20.

    Article  Google Scholar 

  9. 9.

    Szanton SL, Leff B, Wolff JL, Roberts L, Gitlin LN. Home-based care program reduces disability and promotes aging in place. Health Aff. 2016;35:1558–63.

    Article  Google Scholar 

  10. 10.

    Szanton SL. “Decreasing the gap between person and environment: the CAPABLE approach to self-management” Invited speaker for Annual Joint Geriatrics and General Internal Medicine Grand Rounds, Mount Sinai School of Medicine, New York, January 30, 2014. (invited).

  11. 11.

    Stange KC, Breslau ES, Dietrich AJ, Glasgow RE. State-of-the-art and future directions in multilevel interventions across the cancer control continuum. J Natl Cancer Inst Monogr. 2012;2012:20–31.

    Article  Google Scholar 

  12. 12.

    Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:1–19.

    Article  Google Scholar 

  13. 13.

    Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67.

    Article  Google Scholar 

  14. 14.

    Milat AJ, King L, Newson R, Wolfenden L, Rissel C, Bauman A, et al. Increasing the scale and adoption of population health interventions: experiences and perspectives of policy makers, practitioners, and researchers. Health Res Policy Syst. 2014;12:1–11.

    Article  Google Scholar 

  15. 15.

    Chambers DA. Advancing sustainability research: challenging existing paradigms. Public Health Dent. 2011;71:S99–S100.

    Article  Google Scholar 

  16. 16.

    Tricco AC, Ashoor HM, Cardoso R, MacDonald H, Cogo E, Kastner M, et al. Sustainability of knowledge translation interventions in healthcare decision-making: a scoping review. Implement Sci. 2016;11:55.

    Article  Google Scholar 

  17. 17.

    Willis CD, Riley BL, Stockton L, Abramowicz A, Zummach D, Wong G, et al. Scaling up complex interventions: insights from a realist synthesis. Health Res Policy Syst. 2016;14:88.

    Article  Google Scholar 

  18. 18.

    Field B, Booth A, Ilott I, Gerrish K. Using the Knowledge to Action Framework in practice: a citation analysis and systematic review. Implement Sci. 2014;9:172.

    Article  Google Scholar 

  19. 19.

    Graham ID, Tetroe J, the KT Theories Group. Chapter 4.1, Planned action theories. In: Knowledge translation in health care [Internet]. Hoboken: Wiley; 2013. p. 277–87. Available from: [Cited 22 Dec 018].

    Google Scholar 

  20. 20.

    Meffert SM, Neylan TC, Chamber DA, Verdeli H. Novel implementation research designs for scaling up global mental health care: overcoming translational challenges to address the world's leading cause of disability. Int J Ment Health Syst. 2016;8:19.

    Article  Google Scholar 

  21. 21.

    Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50:217–26.

    Article  Google Scholar 

  22. 22.

    Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health. 2011;38:44–53.

    Article  Google Scholar 

  23. 23.

    Green CA, Duan N, Gibbons RD, Hoagwood KE, Palinkas LA, Wisdom JP. Approaches to mixed methods dissemination and implementation research: methods, strengths, caveats, and opportunities. Adm Policy Ment Health. 2015;42:508–23.

    Article  Google Scholar 

  24. 24.

    Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42:533–44.

    Article  Google Scholar 

  25. 25.

    Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    Article  Google Scholar 

  26. 26.

    Breimaier HE, Heckemann B, Halfens RJ, Lohrmann C. The Consolidated Framework for Implementation Research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC Nurs. 2015;14:43.

    Article  Google Scholar 

  27. 27.

    Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manag Rev. 1996;21:1055–80.

    Article  Google Scholar 

  28. 28.

    Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629.

    Article  Google Scholar 

  29. 29.

    Aarons GA, Horowitz JD, Dlugosz LR, Ehrhart MG. Chapter 7, The role of organizational processes in dissemination and implementation research. In: Dissemination and implementation research in health: translating science to practice [Internet]. New York: Oxford University Press; 2012. Available from: [cited 13 Nov 2018].

    Google Scholar 

  30. 30.

    Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9:45.

    Article  Google Scholar 

  31. 31.

    Gagnon MP, Attieh R, Ghandour EK, Légaré F, Ouimet M, Estabrooks CA. A systematic review of instruments to assess organizational readiness for knowledge translation in health care. PLoS One. 2014;9:e114338.

    Article  Google Scholar 

  32. 32.

    Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004;6:61–74.

    Article  Google Scholar 

  33. 33.

    Schwarzer R, Jerusalem M. Measures in health psychology: a user’s portfolio. Causal and control beliefs. Windsor: NFER-NELSON; 1995. Generalized self-efficacy scale. p. 35–7.

    Google Scholar 

  34. 34.

    Fries BE, Shugarman LR, Morris JN, Simon SE, James M. A screening system for Michigan’s home- and community-based long-term care programs. Gerontologist. 2002;42:462–74.

    Article  Google Scholar 

  35. 35.

    Saldana L. The stages of implementation completion for evidence-based practice: protocol for a mixed methods study. Implement Sci. 2014;9:43.

    Article  Google Scholar 

  36. 36.

    Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: the stages of implementation completion (SIC). Implement Sci. 2011;6:116.

    Article  Google Scholar 

  37. 37.

    Kalbfleisch J, Prentice R. The statistical analysis of failure time data, vol. 447. Hoboken: Wiley—Interscience; 2002.

    Google Scholar 

  38. 38.

    Sloan JA, Cella D, Hays RD. Clinical significance of patient-reported questionnaire data: another step toward consensus. J Clin Epidemiol. 2005;58:1217–9.

    Article  Google Scholar 

  39. 39.

    Sloan JA, Frost MH, Berzon R, Dueck A, Guyatt G, Moinpour C, et al. The clinical significance of quality of life assessments in oncology: a summary for clinicians. Support Care Cancer. 2006;14:988–98.

    Article  Google Scholar 

  40. 40.

    Preacher KJ, Selig JP. Advantages of Monte Carlo confidence intervals for indirect effects. Commun Methods Meas. 2012;6:77–98.

    Article  Google Scholar 

  41. 41.

    Biesanz JC, Falk CF, Savalei V. Assessing mediational models: testing and interval estimation for indirect effects. Multivariate Behav Res. 2010;45:661–701.

    Article  Google Scholar 

Download references


We sincerely thank the clinicians at the 18 Medicaid sites for their time and effort.

Availability of data and material

The datasets generated during and/or analyzed will be available upon request from the investigators.


The study is funded by the National Institutes of Aging 1 R15 AG058193-01A1 (9/1/2018 to 8/30/2021).

Author information




SS and MS contributed to the study concept and design, selection of methods, acquisition of sites and subjects and/or data, analysis plan, and preparation of manuscript. AS contributed to the study concept and design, analysis plan, and preparation of manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sandra L. Spoelstra.

Ethics declarations

Ethics approval and consent to participate

Protocol described here was approved by the University 19-061-H Institutional Review Board. Participants will be informed about aims of study and methods for protection of data privacy. Informed consent will be obtained from clinicians and implied by beneficiaries who do not opt out. Reports will contain aggregate data allowing no identification of individual participants.

Consent for publication

This manuscript does not include data from any individual.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Certification program training outline. An outline of the training modules for the certification program. (PDF 72 kb)

Additional file 2:

Pocket card algorithm. A cognitive aid on a packet card that includes a set of rules the clinician will follow when examining the beneficiary’s need or potential benefit from the intervention. (PDF 114 kb)

Additional file 3:

Internal facilitator (IF) training outline. An outline of the modules for the internal facilitator (IF) training program. (PDF 57 kb)

Additional file 4:

Internal facilitator/external facilitator (IF/EF) audit tool. Data collection tool to examine how often the internal facilitators and external facilitators performed facilitation tasks like conducting problem solving, feedback, reflection, counseling, and remediation with a support coordinator. (PDF 56 kb)

Additional file 5:

Implementation strategies and intervention fidelity audit tool. Data collection tool to examine fidelity to the implementation strategies and intervention which includes items like number of consented clinicians, completed surveys, trained clinicians, interdisciplinary meetings, and IF actions and PCSP presence of desire of beneficiary, completed SW and OT assessments and medication reviews, and if brainstorming, problem-solving, and role modeling was conducted. (PDF 49 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Spoelstra, S.L., Schueller, M. & Sikorskii, A. Testing an implementation strategy bundle on adoption and sustainability of evidence to optimize physical function in community-dwelling disabled and older adults in a Medicaid waiver: a multi-site pragmatic hybrid type III protocol. Implementation Sci 14, 60 (2019).

Download citation


  • Adoption
  • Sustainability
  • Implementation
  • Implementation strategies
  • Physical function
  • Community-dwelling
  • Older adults
  • Medicaid waiver
  • Study protocol
  • Randomized controlled trial