Considerable investment is made to generate research knowledge intended to improve the quality and delivery of health care. Knowledge of this type, particularly when it is complex, is frequently conveyed via evidence-based practice interventions, and the costs expended to implement these interventions are similarly substantial [1]. Once intervention implementation “supports” are removed, the initial effects obtained through these interventions are susceptible to natural decay [2,3,4]. The long-term durable sustainment of evidence-based changes to practice is challenging [1, 5, 6].
While sustainability has been identified as “one of the most significant translational research problems of our time” (1: 2), post-implementation studies of practice change sustainability in health care are rare [7,8,9,10,11] and so it follows that our understanding of the factors that influence sustainability is generally poor. Failure to sustain evidence-based changes or innovations to practice means that the intended improvements to care are short-lived, that the practice innovations’ further scale-up and spread is unlikely, and that real losses are incurred on research investment, often made with public funds. This protocol describes a study that aims to contribute to our understanding of the inter-related phenomena of sustainability, sustainment, and spread of evidence-informed, complex practice change interventions.
Unpacking the concept of sustainability
While the concept of sustainability is still maturing [3, 4, 12], work in this area has recently acknowledged a useful distinction between sustainability and sustainment [4, 8]. With a focus on lasting benefits, sustainability generally refers to the extent to which “an evidence-based intervention can deliver its intended benefits over an extended period of time after external support from the donor agency is terminated” ([13]: 118); whereas, sustainment refers to the continued enactment of processes, practices, or work routines that are conveyed and learned through an intervention [4, 8]. While the concept of spread is generally discussed separately [14], we suggest that there is likely a link between spread and the concepts of sustainability and sustainment, given that the spread of the practices and benefits introduced through an intervention, from one part of an organization to other parts or from one organization to another, is unlikely to take place without some degree of retention of these processes and benefits within the originating unit or organization. As with sustainability, the importance of understanding the processes and factors that influence the spread of healthcare innovations, including practice innovations, are highlighted in the implementation literature, albeit separate from the literature on sustainability [9, 15, 16].
Approaches to studying sustainability: fidelity versus adaptation
To date, studies of sustainment and sustainability invoke one of two dominant and competing approaches: the fidelity approach, and the adaptation approach [3, 7]. The fidelity approach focuses on implementation fidelity and is the most common approach used to examine sustainability. Fidelity is defined as the extent to which an intervention program follows the originally intended implementation plan and faithfully delivers the research-informed components of the intervention [11, 17]. This approach contends that deviation from the intended intervention content and delivery protocols during implementation—that is, “program drift” and “low fidelity”—will inevitably lead to diminished benefits/outcomes both during and after implementation, once intervention support is withdrawn [8, 18].
By contrast, the adaptive approach ascribes importance to the co-evolution of the intervention and the organizational context into which it is introduced [7]. This approach suggests that overemphasis on fidelity and adherence, “relative to generalizability and adaptation”, increases the risk of creating interventions—including practice changes and the processes to effect them—that will not “fit” within complex or changeable settings ([8]: 2), and that while changes to the intervention may reduce fidelity they may lead to improved fit to local context and enhanced outcomes/benefits [8, 9, 18, 19].
In this study, fidelity versus adaptation is of interest to us to the extent that it is, or is not, related to post-implementation sustainment and sustainability of practice change. High implementation fidelity during an intervention may contribute to sustained use and benefits. Conversely, the adaptive perspective suggests that sustainability and sustainment is achieved in organizations that are adept at striking a balance between fidelity and responsiveness to the implementation context. The bottom line is that what is done during implementation, in addition to what is done afterwards, is thought to affect the sustainment, sustainability and spread of practice changes conveyed through an intervention—but we do not know precisely how. Work to further our understanding of relationships amongst fidelity, adaptation, sustainment, sustainability, and spread is needed and there is almost no literature on these dynamics. SSaSSy will contribute to this understanding.
Study context
SSaSSy is a post-implementation study of sustainment (continued use), sustainability (lasting benefits), and spread (beyond the initial implementation setting) of the practice changes conveyed through an evidence-informed, Care Aide-led, facilitation-based quality-improvement intervention called SCOPE (Safer Care for Older Persons (in long-term care) Environments) that is the focus of a clinical trial being conducted in in Canadian nursing homes operating in the Provinces of Manitoba (MB), Alberta (AB) and British Columbia (BC) [NCT03426072]. In SCOPE, HCA-led teams lead quality-improvement initiatives focussing on one of three resident care areas identified as priorities by experts in long-term care, i.e., mobility, pain, and behavior [20]. The SCOPE intervention was piloted in nursing homes in AB and BC over 2010-2011. The impacts of the SCOPE intervention are described in several published articles:
In Norton et al. [21], the SCOPE pilot was shown to meet its primary objective of demonstrating the feasibility and utility of implementing the intervention in nursing homes relying upon the leadership of HCAs, and engagement of professional staff and leadership in facilitative roles. Specifically, of the 10 HCA-led QI teams in nursing homes that participated in the SCOPE pilot, 7 succeeded in learning and applying the improvement model and methods for local measurement, with 5 of the 10 teams showing measurable improvement in the chosen clinical areas.
These impacts were corroborated in a follow-up study that examined the sustainability of elements of the SCOPE pilot [22]. In this article, sustained differences between participating/intervention units, and non-participating units, were observed in outcomes relating to quality-improvement activities (i.e., continuation of work according to the improvement model and principles learned in SCOPE), HCA empowerment, and satisfaction with quality of work life.
As part of the SCOPE clinical trial, SCOPE was implemented in 7 units in MB nursing homes over 2017, somewhat earlier than the intervention was implementation in participating BC and AB units/homes. While the data for the MB homes will be analyzed in conjunction with that collected for homes in BC and AB, a recent retrospective qualitative analysis of the implementation experiences [23] of administrative leaders (sponsors), professional staff, and QI team participants in MB homes demonstrates effects akin to those observed in the SCOPE pilot. In addition to accruing observable benefits to residents who were the subjects of the QI initiatives formulated by the HCA-led QI teams in participating units in each MB home, SCOPE was observed to change the expectations and behaviors on the part of administrative leaders, professional staff, and—importantly—HCAs relating to their abilities to conduct small-scale, unit-level, evidence-informed quality-improvement initiatives [23].
Both SCOPE and SSaSSy are situated within a larger program of research, Translating Research in Elder Care (TREC) [24]. TREC was initiated in 2007 and focuses on the influence of organizational context on resident quality of care and safety in 94 nursing homes in the three Western Canadian Provinces [24]. Both SCOPE and SSaSSy rely on TREC’s longitudinal database that includes data on staff behavior, attitudes and quality of worklife; leader behavior; work environment (context); and care unit and nursing home characteristics (e.g., unit size, facility owner-operator model). Data are collected routinely from all levels of nursing home staff, and quality of care measures are collected on a quarterly basis across the 94 homes at the unit level [25] via the Resident Assessment Instrument—Minimum Data Set, version 2.0 (RAI-MDS 2.0).
Study purpose and aims
SSaSSy focusses on a 1-year interval, 1 year after SCOPE implementation concludes. Phase 1 of SSaSSy is a pilot that will occur in 7 units in MB nursing homes where SCOPE was piloted over 2016–2017. The results of this pilot will inform the content of post-implementation “boosters”, designed to sustain or renew the application of the QI techniques and tools—the changes to care practice—conveyed through SCOPE. The relative effectiveness of the boosters compared to a no booster control will be tested in phase 2 in 31 more units in nursing homes in BC and AB, where SCOPE concluded in May 2019.
As a post-implementation study, SSaSSy presents a rare opportunity to systematically contribute to knowledge [22] on the sustainment and sustainability of complex practice changes conveyed through evidence-based interventions, and to examine spread—first, in the SCOPE pilot sites in MB and subsequently in the trial sites in BC and AB.
Specific aims of SSaSSy are:
- 1.
To determine whether fidelity, site- or facility-initiated adaptation of aspects of the intervention, aspects of the implementing unit, and/or other aspects of nursing homes’ operations or structures, are associated with sustainment, sustainability and spread one year following implementation of practice changes conveyed through SCOPE.
- 2.
To explore the relative effects on sustainment, sustainability and intra-organizational spread of high- and low-intensity post-implementation “boosters” compared to “no booster”/natural decay; specifically, the extent to which there are:
- a.
sustained or renewed improvements in resident outcomes in clinical areas of focus targeted by the SCOPE intervention (deteriorating mobility, pain, responsive behavior) (sustainability),
- b.
sustained or renewed changes in staff behaviors (reported use of best practices, use of SCOPE components and processes) (sustainment),
- c.
sustained or renewed changes to staff work attitudes (work engagement, psychological empowerment, burnout, job satisfaction) and outcomes (organizational citizenship behaviors) related to work performance (sustainability),
- d.
sustained or renewed changes to senior leadership support behaviors relating to staff engagement in SCOPE (sustainment),
- e.
indications of spread to other units within the SCOPE intervention sites, and its extent.
- 3.
To compare the costs and effectiveness of each post-implementation support condition.