- Systematic review
- Open Access
Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: a systematic review and theory analysis of selected sustainability frameworks
Implementation Science volume 14, Article number: 108 (2019)
There is growing recognition among healthcare professionals that the sustainability of evidence-based practices (EBPs) within different settings is variable and suboptimal. Understanding why a particular EBP might be sustained in one setting and not another remains unclear. Recent reviews illustrate the need to identify and analyze existing frameworks/models/theories (F/M/Ts) that focus solely on the sustainability of EBPs in specific healthcare settings, such as acute care, to illuminate key determinants and facilitate appropriate selection to guide practice and research.
We conducted a systematic review to extract sustainability frameworks. This involved using two available syntheses of the literature and a systematic search of four databases from January 2015 to July 2018: CINHAL, MEDLINE, Embase, and ProQuest. We included studies published in English, and if they included sustainability F/M/Ts recommended for use in acute care or an unspecified healthcare organization/setting. F/M/Ts explicitly recommended for use in public health and or community settings were excluded. We then conducted a comparative analysis of F/M/Ts using a modified theory analysis approach, to understand the theoretical underpinnings of each F/M/T, their determinants and concepts hypothesized to influence the sustained use of EBPs within an acute care context.
Of 2967 identified citations from the 2 available syntheses and the systematic review, 8 F/M/Ts met the inclusion criteria. We identified 37 core factors, of which 16 were recorded as common factors (occurring within 4 or more of the 8 included F/M/Ts). All factors grouped into 7 main themes: innovation, adopters, leadership and management, inner context, inner processes, outer context, and outcomes.
This systematic review is the first to include a comprehensive analysis of healthcare sustainability F/M/Ts for the sustained use of EBPs in acute care settings. Findings reveal insights into sustainability as a “process or ongoing stage of use” following initial implementation, suggesting this construct should be added to the definition of sustainability. Results provide a resource of available F/M/Ts and hypothesized factors to consider for acute care team members who are planning or currently implementing EBPs with the goal of improving patient outcomes. It also provides a basis for future research on sustainability in acute care.
Over a decade ago, the sustained use of evidence-based practices (EBPs) was identified as a gap in the literature. Evolving debate among experts suggest sustainability should be considered a distinct concept that occurs “(1) after a defined period of time, (2) the program, clinical intervention and/or implementation strategies (hereafter referred to as EBPs) continue to be delivered and/or, (3) individual behavior change (i.e. clinician, patient ) is maintained, (4) the program (EBP) and individual behavior change may evolve or adapt while (5) continuing to produce benefits for individuals/systems ”. Despite growing interest, the timing and understanding of how to sustain the use of EBPs remains a relatively unexplored field of research [2, 3] and least understood part of the translation research process  that has challenged practitioners and researchers alike. Evidence reveals the integration and sustainability of EBPs in clinical practice is “an iterative, dynamic”  and “complex process,”  which poses a significant challenge. Emerging discourse indicates efforts to sustain EBPs in healthcare should be guided by conceptual frameworks, models or theories (hereafter collectively referred to as F/M/Ts) [1, 7,8,9,10,11,12] to better understand the factors that impact sustainability as a distinct concept [13, 14], over time, in a range of distinct healthcare settings [3, 10, 11]. Thus, a critical analysis of existing sustainability F/M/Ts relevant to acute care contexts was conducted as a way to understand the meaning of key concepts, factors, and their relationships to ultimately provide direction for practice and research.
Increasing demand on healthcare organizations to improve patient outcomes [10, 15, 16] in an efficient, cost-effective manner [17, 18]) has resulted in the growing expectation that EBPs be informed by research, be effective and sustainable to inform clinical decision making [19, 20]. In response, healthcare organizations have undertaken a number of quality improvement initiatives . Despite efforts, variable rates of sustained use of EBPs exist ranging from none to full adherence , not only among various healthcare professionals but also within different settings [1, 9, 10, 17, 20,21,22,23,24]. Researchers argue the decay of sustained EBPs [17, 23, 25, 26], also referred to as the “improvement evaporation effect” [25, 26] can be attributed to the limited use of theoretical F/M/Ts [27, 28]. To overcome these challenges and to advance knowledge, researchers [7, 11,12,13,14] recommend the use of F/M/Ts to examine the factors that impact sustainability as a distinct concept, especially in complex acute care environments .
Recent reviews/syntheses reveal a lack of use/empirical testing of existing F/M/Ts [10, 19, 24], highlight several diverse perspectives, applications and constructs deemed useful for sustainability , and few F/M/Ts that focus solely on the sustainability of EBPs within acute settings [10, 24, 29]. Specifically, the majority of sustainability F/MTs and approaches are designed for use in non-specified healthcare settings (37% or 23/62) (e.g., healthcare organizations or systems), followed by 31% (19/62) specified for use in public health, 26% (16/62) in community settings, and only 3% (2/62) primarily focused within acute care . To date, a review that examines how to improve the sustainability of EBPs in acute care settings has not been conducted . Given healthcare expenditures are reported to be the largest in hospitals (36.9% in 2018) , exclusively identifying relevant concepts and factors related to sustainability in this challenging setting will likely be of considerable benefit to research and practice, potentially improving the quality of care and reducing costs. Clearly, a gap exists regarding which existing sustainability F/M/Ts are applicable and what factors are relevant when trying to sustain the use of EBPs primarily in acute care contexts [10, 13].
The aims of this study were to (i) identify existing healthcare F/M/Ts that explicitly address the process of sustained use of research (EBPs/guidelines/innovations/clinical protocols/programs/interventions) and are recommended for use within acute care contexts or unspecified healthcare organization/setting; (ii) compare F/M/Ts, using a theory analysis approach, to identify key concepts and factors that influence/predict the likelihood of successful sustainability of EBPs; and (iii) provide a list of relevant sustainability F/M/Ts, concepts, and core factors to act as a guide for practice and provide direction for future research within acute care contexts.
Search strategy and data sources
Two different data sources and related search strategies were used to identify existing healthcare sustainability F/M/Ts. First, a full text review, abstraction and appraisal of all F/M/Ts included in two existing knowledge syntheses of sustainability in healthcare [1, 10] was conducted to determine overall alignment with the aims and eligibility criteria established for this study given their original purposes, scopes and related methodologies (see Table 1). Second, to identify relevant sustainability F/M/Ts published after the two syntheses, a new systematic search of all published articles, dissertations/theses, systematic and scoping reviews, and concept analyses was conducted using the same eligibility criteria guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) reporting standards [31, 32] (see Additional file 1). The selection of healthcare databases, search terms, and strategy was supported by a health science librarian and peer-reviewed by a second using the PRESS EBC Checklist . A search of CINHAL, MEDLINE, Embase, and ProQuest databases was conducted with results limited to citations published between 1 January 2015 and 3 July 2018, based on end dates of the two syntheses. A hand search of references from included citations was undertaken. Details of key terms and search strategies are available in Additional file 2. Ethical approval was not required for this review.
Eligibility criteria were designed to examine sustainability as a distinct concept, as per Moore et al.’s  definition, and to identify concepts and factors that related solely to the sustained use of EBPs, after the initial rollout, in complex healthcare environments such as acute care [3, 34]. A checklist of inclusion and exclusion criteria was developed to guide selection of citations (see Table 1). During the process, four coauthors (LNP, JS, BD, CB) reviewed a sub-sample of citations  to refine and ensure criteria could be consistently applied. To be eligible, citations needed to be published in English; in a peer-reviewed journal; include sustainability or implementation and sustainability F/M/Ts recommended for use in acute care or an unspecified healthcare organization/setting; and represent the most current/refined version.
A citation was excluded if the F/M/T was not recommended for healthcare; was recommended only for use within a specified setting other than acute care (e.g., public health or community); if it contained only an implementation F/M/T; and if it contained an implementation and sustainability F/M/T without an explicit breakdown of related sustainability factors. Notably, this study was not designed to examine the influence of implementation on sustainability.
Data collection process and analysis
A data collection form was piloted by four coauthors (LNP, BD, CB, JS) with 50 randomly selected citations to ensure comprehensiveness prior to screening. The form required minimal modification. To ensure inclusiveness, level 2 full text screening of all citations was conducted in two steps: (i) screening of results from two syntheses was completed by one reviewer (LNP) and reviewed by four coauthors (BD, IG, CB, JS); (ii) screening of systematic review results was completed by two independent reviewers (LNP, IM). Final decisions regarding inclusion were made jointly by LNP and coauthors (BD, IG, CB, JS). Disagreements were resolved through discussion and consensus.
A theory analysis of the identified F/M/Ts was undertaken as a means of understanding their theoretical underpinnings, paying particular attention to key concepts/factors influencing sustained EBP use within acute care . According to Walker and Avant , theory analysis involves consideration of seven elements: (i) determining origins, (ii) examining meaning of concepts and their relationships; (iii) analyzing logical adequacy of concepts and relational statements to determine predictive ability to generate hypotheses, (iv) determining usefulness for practice and predicting outcomes, (v) defining generalizability across settings, (vi) defining the degree of parsimony and language clarity, and (vii) determining testability (see Table 2). Modifications to the theory analysis elements/tool included adding a subjective rating scale for both parsimonious (full or partial) and language (clear, somewhat unclear, unclear). Analysis involved entering findings into a master chart to facilitate comparisons. All factors identified in the appraisal were then extracted and collated. Qualitative content analysis was completed by identifying and placing all related and similar factors together (identified as core factors) and then into broad themes, which were inductively identified from F/M/Ts [37, 38]. Factors cited in four or more F/M/Ts within each theme were identified as common factors.
Of the 2967 citations identified, 8 met the inclusion criteria (e.g., four from Moore et al. , three from Lennox et al. , one from the new systematic review) and were eligible for theory analysis (Fig. 1). Rationale for excluded citations is documented in Additional file 3. Most F/M/Ts containing both implementation and sustainability phases did not explicitly provide a detailed breakdown of the sustainability concepts/factors and were excluded. Those that did were recommended for use in community and/or public health settings and were excluded.
Framework/model/theory characteristics and quality appraisal
The F/M/Ts were published between 2005 and 2016; the majority (n = 6) published after 2010. The originators are from Europe (n = 4) [25, 39] [40, 41]; North America (n = 3) [13, 34, 42]; and Australia (n = 1)  (see Table 3). Various methodological approaches used by originators to develop F/M/Ts included focused systematic or literature reviews (n = 4) [25, 34, 40, 42]; integrative reviews of frameworks or theory [41, 43]; a Bayesian research co-production approach ; and a concept analysis . The F/M/Ts were reported to be based on theoretical and empirical work of scholars from different fields of study/disciplines with varying theoretical perspectives on sustainability. Specifically, these included the diffusion of innovations theory [42, 43], organizational change theory , organizational and management theory [25, 39, 42], ecological theory [34, 43], total quality improvement theory , psychological theory , theory of routines , and multiple healthcare discipline theories . Six F/M/Ts were designed to be operationalized to guide practice and/or research at an organizational or unit/departmental level. Two were specified for use at the project/initiative level [34, 39]. Three F/M/Ts were explicitly recommended for use in a hospital [13, 40, 41], and the remaining five were recommended for use in any unspecified healthcare organization/setting. Some F/M/Ts were intended for multiple audiences, namely, researchers [13, 25, 34, 39,40,41,42] practitioners [13, 34, 39,40,41,42], policy-makers , administrators , and funders/grantors . The most common motivation was to add to the body of evidence/knowledge to either guide research or better understand how to successfully sustain effective improvements in practice.
Meaning of the F/M/T
To examine how originators defined the constructs of sustainability, conceptual definitions for sustainability were mapped to the five constructs of a comprehensive definition recently published by Moore et al.  (see Table 4). Notably, one author did not provide nor reference an explicit definition of sustainability . Two definitions included all five established constructs [25, 39], and four definitions included all but one construct [13, 34, 40, 41]. Similar to Moore et al.  findings, the most commonly described construct for sustainability was “continued delivery or use,” which was combined equally with the “evolution or adaption” construct cited in seven out of eight definitions.
Differing from Moore et al.  findings where most publications did not define the timeline for sustainability, the “after a period of time” construct was included in 75% (6 out of 8) of the definitions. However, these time-related references were undefined and unquantified. The “continued benefits” construct occurred in five out of eight definitions signifying the importance of the perceived goal to enhance outcomes (on individual, unit, organization, system level). The “maintain behavior change in individuals” construct reflected how a broad range of EBPs may interact with individuals or teams to maintain behavior change for sustainability. Although this was the least commonly described construct, it occurred in half the definitions: two F/M/Ts recommended for use in acute care [40, 41] and two for use in unspecified settings [25, 39].
Two similar constructs of sustainability currently not included in the Moore et al.  definition emerged during the analysis: defining sustainability as a “process,” [13, 25, 34, 39, 40, 43] or as a “stage/phase of ongoing use” post-implementation [41, 42]. These views were supported by several theoretical perspectives given F/M/T origins, revealing a new construct that describes the nature of sustainability to be “ongoing/continuous and process-like.”
Synthesis of factors and themes
Initially, 152 sustainability factors were extracted from the 8 F/M/Ts. Qualitative analysis identified 37 core factors, which grouped into 7 themes: (1) characteristics of the innovation/EBP; (2) adopter/user factors influencing sustained use (3) leadership and management influences/factors; (4) inner context (practice setting/organization) factors where EBPs are delivered; (5) inner processes/infrastructure factors that support the EBPs (e.g., processes, methods, systems, structures, or strategies); (6) outer context or broader system factors; and (7) outcomes descriptions without defined factors. Further synthesis identified 16 common factors (occurring in four or more F/M/Ts), which are highlighted with an asterisk in Table 5.
A subgroup analysis comparing the themes and factors among the specified acute care F/M/Ts [13, 40, 41] with those recommended for unspecified healthcare settings [25, 34, 39, 42, 43] was conducted. Results are available in Additional file 4 and Table 5. Notably, originators collectively identified all seven themes within both subgroups. Only three out of 37 core factors were uniquely identified among all F/M/Ts: two core factors were separately identified in two different F/M/Ts within the acute care subgroup (e.g., behavioral change strategies , financial funds, and non-financial resources ), and one core factor was identified within the unspecified setting subgroup (e.g., barrier identification ). Given minimal subgroup differences, all F/M/Ts were included in the theory analysis.
The themes were defined by terms used by originators. The “adopter” theme is defined as a stakeholder, staff, user, adopter, actor, or individual using the innovation/EBP. Of note, the Sustainability of Innovation Theoretical Framework (hereafter Fox SITF)  and Sustainability of Healthcare Innovations Framework (hereafter Fleiszer SHIF)  focused exclusively on the presence and influence of champions. The “inner context” theme refers to the context, practice setting or organization, while the “inner process” theme includes processes, methods, systems, structures or strategies used within the context. The “innovation” theme, defined as a new process, change, product, practice, or programme in six F/M/Ts, is not evident in two F/M/Ts [40, 41]. Similarly, the “leadership and management” theme refers to leadership style, approach, behaviors, engagement, support, or feedback in six F/M/Ts [13, 25, 34, 39, 41, 42]. The “outer context” theme, referencing conditions, context, systems or environment external to the inner context, is not evident in three F/M/Ts [39,40,41]. The outcome theme is described in four F/M/Ts as “outcomes on a spectrum from high to nil” , sustained “teamwork behaviors” , “consequences” , or “continuation of benefits” .
Inclusiveness of themes and factors
Three F/M/Ts [13, 25, 34] contain all seven themes with one F/M/T  containing six themes. The inclusiveness of 6–7 themes in 50% (4 out of 8) F/M/Ts highlights the importance of all themes and related factors for the sustainability of EBPs within acute care contexts. The innovation , leadership and management [34, 40, 41, 43], outer context [39,40,41], and outcome [34, 40, 41, 43] themes were not evident in all F/M/Ts. The Framework and a Measurement Instrument for Sustainability of Work Practice (hereafter Slaghuis FMIS-WP)  contains only three themes and related factors as it represents a portion of a larger conceptualization on sustainability unpublished. The 37 core factors primarily are distributed among 6 themes, given the outcome concept/factors are undefined. All F/M/Ts contain core factors from the adopter, inner context, and process themes. Fifty-seven percent (21 out of 37) of the core factors are contextual contingent including inner context, inner processes, and outer context core factors thus highlighting the influence context may have on the sustainability of EBPs in acute care. One F/M/T contained all 16 common factors .
All originators described the relationship between the factors as non-discrete or dynamic, which may interact either in varied combinations or degrees on different levels. How this occurs, however, was not made explicit by definition/statements. The use of arrows to imply direction or potential influence between concepts/factors was used in seven F/M/Ts. Uniquely, the National Health Service Sustainability Model (hereafter Maher NHS-SM)  originators used three overlapping colored circles representing broad concepts to illustrate a level of dynamic interaction among the related factors within the concepts. The use of arrows or circles failed to clarify how the interactions between factors occurred. The Dynamic Sustainability Framework (hereafter Chamber DSF)  originators specified a “dynamic relationship” that exists between and among the three concepts (e.g., innovation, practice setting, broader system) and changes over time, but how to interpret this was unclear. Uniquely, the DCOM Framework with Realistic Evaluation (hereafter Frykmann DCOMF)  originators used relational statements to identify key influences impacting relationships between factors not evident in other F/M/Ts, namely, four mechanisms of behavior change: direction, competence, opportunity, and motivation. All originators recommended further testing to seek greater clarity about relationships between concepts. Fleiszer SHIF  originators suggest their framework is representative of a mid-range theory, and further understanding of the relationship between concepts and factors is essential.
Key assumptions underlying the F/M/Ts include (i) the concept of sustainability is only partially mature , dynamic [34, 40], or ambiguous having different meanings in different contexts ; (ii) sustainability considers change (either strategic and/or incremental) as a central influence [25, 34, 39,40,41, 43]; (iii) evolving fit and/or adaption of the EBP is expected [13, 25, 34, 39, 40]; and (iv) success overtime is based on whether or not the EBP remains beneficial [13, 25, 34, 39, 41, 42].
All originators provided schematic representations illustrating key concepts/factors claiming to be operational and able to guide sustainability efforts and future research. Four F/M/Ts depict unidirectional graphical representations that assume a continuum or processual stance focusing on the EBP and its ongoing implementation process in context influenced by internal or external factors [13, 25, 34, 41] thus implying the goal of maximizing the fit between the EBP and the context. Originators of the remaining schematics provided a simple, high-level representation depicting the interplay among the set of factors [39, 40, 42, 43]. Notably, in all schematics, each factor category was represented as equal relative to one another given their image size. In fact, originators contend the relative significance of the factors cannot be determined a priori, except the Maher NHS-SM  where relative weighting within and among the factors is provided based on empirical evidence.
To date, evidence of further testing of four F/M/Ts has occurred [13, 34, 39, 40]. Notably, the Maher NHS-SM  has been empirically tested in the USA, Canada, UK, South Africa [14, 21, 45, 46] and in low- to middle-income countries . All originators recommended practical testing (application and evaluation) in multiple contexts using different methodologies to broaden conceptual understanding and further development/refinement. Specifically, research using a systemic and process-orientated lens to uncover the complexities and dynamics of the concept was recommended .
Parsimonious and language clarity
Five F/M/Ts were subjectively rated by coauthors (LNP, BD, IG, CB, JS) as parsimonious, with clear language, terminology, explicit definitions for factors, and without repetitions noted [13, 25, 39, 41, 42]. The remaining three F/M/Ts were rated as partially parsimonious based on the lack of completeness  or the use of vague definitions and concept relational statements [34, 43].
Originators claimed all F/M/Ts as operational and capable of guiding research and practice to explore factors influencing the sustainability of healthcare EBPs. Originators of four F/M/Ts explicitly provided either testable hypotheses [34, 39, 42] or testable scales for the concepts . The chambers DSF  originators proposed seven tenets related to the ongoing improvement of EBPs emphasizing a “central goal of continuously optimizing the fit between the innovation and the dynamic (changing) delivery context to achieve maximum benefit” . The Model for Sustaining Innovations (hereafter Racine MSI)  originators provided 12 propositions, which align with three main factor categories (innovation legitimacies, intermediary functions, conditions of local adopters) but assert it does not predict the likelihood of sustainability. Conversely, the Maher NHS-SM  originators defined 10 measurable factors, which are weighted within and among each other, providing a testable hypothesis and a prediction of sustainability for the improvement. originators of the four remaining F/M/Ts [13, 25, 41, 43] identified measurable factors/variables to guide research and data collection without explicitly defining the impact of the factors for outcomes concept but rather state it can vary based on the innovation, conditions, and contexts.
Minimal inconsistencies related to the content within the “adopters” and “outcomes” themes were noted among all F/M/Ts by coauthors (LNP, BD, IG, CB, JS). Specifically, within three F/M/Ts, the “adopters” theme was not identified as distinct but rather considered part of the inner context [13, 34] or inner processes  themes. In the Fleiszer SHIF , the deliberate positioning of individual characteristics within the inner context versus distinct, similar to the leadership and management theme, was not explicit. Chambers DSF  originators did not identify “staff” separately but rather part of the “practice setting” or inner context. The staff/team member is not explicitly identified as a separate theme by originators of Frykman DCOM , yet the entire framework is focused on revealing how behavior change interventions influence the sustainability of staff/teamwork behaviors. The failure to distinguish adopters, either as individual [2, 48] or collective agency  influences, as a separate theme by originators is inconsistent with other F/M/Ts noted in recent syntheses [10, 29] and this study. Furthermore, originators of four F/M/Ts, identified “outcomes” as a theme [13, 25, 34, 41] represented by the combined influence of factors from within their frameworks. Outcome factors were undefined in all F/M/Ts.
Originators claimed the F/M/Ts have multidisciplinary relevance and practicality to inform health professionals, administrators, policy-makers, and/or funders to identify inadequacies, refine theory, and ensure the development of the concept. Uniquely, originators of Racine MSI  contend their model provides a “blueprint or agenda”  with clear practical implications. Maher NHS-SM  originators assert their model is intended to provide a platform for quality improvement for all healthcare disciplines. Other originators indicated their F/M/T can be used across multiple healthcare settings [34, 41], for nursing-specific settings [13, 43] or at the micro-level of work practice . Originators of Sustaining Organizational Change Framework (hereafter Buchanan’s SOCF)  presented a practical guide outlining a range of potential influences/factors at different levels of analysis.
Two F/M/Ts provided tools [39, 40]. Maher NHS-SM  includes a manual, user guide, diagnostic tools, videos, and an interactive option, all of which can be used to assess and predict the likelihood of the sustainability of change in clinical practice using a systematic approach. The Slaghuis FMIS-WP  includes an instrument to measure sustained changed work practices related to improvement processes, which originators have tested [40, 50].
The Slaghuis FMIS-WP , Fleiszer SHIF , and Frykman DCOMF  were all designed to guide practice and research in acute care settings. Specifically, Slaghuis FMIS-WP  and Frykman DCOMF  were designed for changing complex healthcare environments (hospitals) where high turnover and interdependence between multiple professionals often exists. The Fleiszer SHIF  was designed for use in diverse hospital nursing contexts at the unit/organizational level. The Fox SITF  was recommended for use in unspecified nursing contexts. The Maher NHS-SM  was designed to guide practice and research at the project/initiative level and has been tested in several non-specified healthcare settings  including hospitals , community settings , and globally . The Buchanan SOCF , Racine MSI , and Chambers DSF  were designed for use in non-specified healthcare contexts for a broad range of interventions at a project/initiative level  or unit/organizational level [25, 42].
This systematic review is the first to include a comprehensive analysis of healthcare sustainability F/M/Ts with a primary focus on identifying key concepts influencing the sustained use of EBPs in acute care contexts. Our search revealed the vast majority of F/M/Ts relating to sustainability were designed specifically for use in community and public health settings, which is congruent with the current literature [3, 10]. Notably, only three F/M/Ts were primarily focused on the sustainability of EBPs within acute care settings [13, 40, 41], and five were recommended for use in non-specified healthcare organizational/settings [25, 34, 39, 42, 43]. Recommended target domains for use across disciplines imply general learning can be gathered to inform sustainability for practice and research using an interdisciplinary approach. Addressing sustainability challenges from a variety of theoretical perspectives and disciplines is equally pivotal to understanding this concept in acute care as reported in other healthcare sectors .
The two most commonly described constructs cited in the F/M/Ts for sustainability were
“continued delivery or use” and “evolution or adaption” constructs. The prominence of these constructs emphasizes the continuous use and evolutionary nature of sustained EBPs in context over time and is congruent with Moore et al.’s  previous developed definition of sustainability. Furthermore, this analysis provides insight into sustainability as a “process” or “stage/phase” of ongoing/continuous use of EBPs post-implementation. This finding is congruent with researchers who argue sustainability is not an all or nothing “phase or endgame”  nor an “outcome”  but rather a “process of managing and supporting the evolving EBP” overtime . Some contend it is a “matter of degree of sustained change” [18, 53] to be viewed as a “continuous phase”  or a “continuum”  or a “process” . The importance of this construct is consistent with a recent review , ultimately adding new knowledge to the current definition . The shift in perspective of sustainability as a “process or ongoing/continuous stage/phase” [3, 10], together with the EBPs’ evolutionary nature and dynamic interaction/influence among the factors overtime [3, 34], highlights the complexity of planning and measuring sustainability and the need to consider how strategies for sustainment overtime differ from implementation and/or potentially overlap.
Results provide a resource of eight F/M/Ts and hypothesized factors that can be used by acute care team members planning or currently implementing EBPs with the goal of improving patient outcomes. Our synthesis of the concepts/factors revealed 37 core factors which cluster around 7 themes specifically defined by the F/M/T originators to be relevant to acute care settings. Four F/M/Ts containing all [13, 25, 34] or most  of the themes provide a knowledge base for practitioners and researchers to evaluate the sustained use of EBPs within their acute care setting. Four themes align with those deemed useful in any setting by Lennox et al.  (e.g., (i) initiative design and delivery = inner processes, (ii) people involved = adopters, (iii) organizational setting = inner context, (iv) external environment = outer context), and three add to the current knowledge, namely, leadership and management, characteristics of the innovation, and outcomes. The equal distribution of core factors among six of the seven themes (excluding outcome) signifies the relative importance of each theme for the sustainability of EBPs in acute care. Notably, several factors support the conceptualization of sustainability as “a dynamic construct that allows for adaptation in response to new or changing populations, evidence, policies, or other contextual influences” . The combined contextual factors (57% or 21/37) influencing sustainability related to acute care contexts include (i) the integration of four layers of context factors influencing the sustained use of complex organizational change practices (e.g., individual, interpersonal relationships, internal context, wider infrastructure system) ; (ii) attention to the complexity, multi-layered, ever-changing organizational setting [13, 25, 34]; (iii) the adaptability of the innovation/EBP to context [13, 34, 39], and (iv) the dynamic process of routinization of innovations/EBPs as a source of change . Arguably, contextual factors impacting sustainability within and among departments or sites will likely provide insight into why the sustained use of EBPs may vary within the same acute care setting. In turn, this likely will affect the strategies needed for sustainment.
Differences amongst F/M/Ts lie in the overall structures, the degree of refinement, substantiation to date, and identified gaps. Each F/M/T reflects a different conceptualization of sustainability evident in the varied schematics. The use of vague/minimal terminology defining concepts/factors and their relationships increases the potential for multiple interpretations. Sustainability outcomes were depicted in three F/M/Ts as a range (e.g., decay to sustainability to development , a spectrum from high to nil  or as an ongoing stage/phase of implementation ). The Chambers DSF  defined outcomes as the “continuation of intended benefits.” The outcome theme is not explicitly defined in the remaining four F/M/Ts [39, 40, 42, 43]. Consistent with other researchers , we recommend future inquiry focus on articulating sustainability outcomes.
Identified gaps among the eight F/M/Ts were revealed by examining their concepts/factors and tools. Variation existed related to the inclusiveness of each factor and labeling of themes. For example, the absence of any type of ‘financial factor’ in the Racine MSI  to guide stakeholders offers little insight into how this factor influences sustainability. Additionally, Fleiszer SIHF  did not include a separate “adopter theme” but instead recognized the role of leadership and management as distinct. A lack of focus on facilitation as a factor either explicitly or implicitly or its inclusion and the perceived need for it is not evident in most F/M/Ts [25, 34, 39, 41, 42], except the Fleiszer SIHF . Originators of the Chambers DSF  were distinct in their acknowledgement of the “dynamic relationship” between three “changing” concepts (innovation, practice setting, broader system), their focus on “benefits beyond helping patients” and the “fit of the innovation” with existing routines/processes. Despite this acknowledgement, the potential risk of overlooking the impact on patient outcomes has been recognized . Slaghuis FMIS-WP  originators claim their framework is part of a larger unpublished framework. Lastly, only two of the F/M/Ts included tools to measure changed work practices [39 40]. To date, minimal evidence for instrument reliability and validity is available for these tools.
This systematic review is the first to include a comprehensive analysis of healthcare sustainability F/M/Ts for EBPs with a primary focus on acute care context. Seven themes primarily related to acute care were identified, four that align with a current review , and three that add to current knowledge (e.g., characteristics of the innovation, leadership and management, and outcomes). By identifying factors and themes/constructs relevant to acute care settings, this work has the potential to aid sustainability for those planning or currently implementing EBPs. The analysis offers insight into sustainability as a “process” or “ongoing stage of implementation” adding to the current definition. For the first time, factors (mechanisms) influencing the sustainability of behavior changes in an acute care setting (see Table 5) are integrated into a synthesis adding to the current knowledge base . Additionally, the modified theory analysis criteria can be used as a tool to guide practitioners, researchers, and students in the appraisal of emerging or existing F/M/Ts, related concepts, and factors.
There are limitations to consider when interpreting the results of this review. First, a systematic review was conducted for conceptual F/M/Ts related to the sustainability of healthcare EBPs from January 1, 2015, to July 3, 2018. Frameworks/models/theories prior to these dates were identified from two existing knowledge synthesis, dated 1946 to March 2017 inclusively. Inclusion criteria varied within each synthesis, and therefore, there is a risk some F/M/Ts may have been missed. Second, the new systematic review, designed to identify recently published F/M/Ts included four key databases, known to focus on healthcare and/or implementation science, among the 14 combined databases used within the two syntheses. There could be sustainability F/M/Ts in databases restricted to the social sciences or organizational management literature that may have been missed. However, healthcare was the primary focus. Third, the qualitative analysis of the main themes and related factors was conducted independently by one reviewer, then analyzed/reviewed by coauthors. Analysis using a deductive approach might draw different conclusions. Lastly, interpretations made as part of the theory analysis are based on the reviewers’ subjective appraisal . These items are clearly marked in Table 2.
Sustainability is an emerging field of study. Given the ever-changing nature and complexity of acute healthcare settings and related costs, it is imperative practitioners and researchers consider the use of sustainability F/M/Ts to guide their practice and inquiry to ensure EBPs are sustained effectively, continue to inform clinical decisions and contribute to improved patient outcomes. Principally, selecting one of the eight sustainability F/M/Ts proactively to plan, evaluate and interpret findings is recommended. Then consider the context level for F/M/T use, specify the goals of sustainability, and determine if the concepts and factors listed apply . We also recommend future inquiry adopt the use of mixed methodologies to explore the complex relationship between implementation factors and outcomes (including sustainability), and determine their level of influence using Proctor’s Framework . Additionally, using a theory analysis approach to examine F/M/Ts containing both implementation and sustainability could provide new insight into the relationship of factors over time (e.g., early, mid-process, and long-term) and/or the potential impact of implementation on the sustainability phase.
Availability of data and materials
All data generated or analyzed during this study are included in this published article (and its supplementary information files).
- Buchanan SOCF:
Buchanan Sustaining Organizational Change Framework
- Chamber DSF:
Chamber Dynamic Sustainability Framework
- Fleiszer SHIF:
Fleiszer Sustainability of Healthcare Innovation Framework
- Fox SITF:
Fox Sustainability of Innovation Theoretical Framework
- Frykman DCOMF:
Frykman Direction, Competence, Opportunity and Motivation Framework
- Maher NHS-SM:
Maher National Health Services Sustainability Model
- Racine MIS:
Racine Model for Sustaining Innovations in their effectiveness
- Slaghuis FMIS-WP:
Slaghuis A Framework and a Measurement Instrument for Sustainability of Work Practice in long term care
Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12(1):1–8.
Ament SMC, de Groot JJA, Maessen JMC, Dirksen CD, van Der Weijden T, Kleijnen J. Sustainability of professionals’ adherence to clinical practice guidelines in medical care: a systematic review. BMJ Open. 2015;5(12):e008073.
Shelton RC, Cooper BR, Stirman SW. The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care. Annu Rev Public Health. 2018;39(1):55–76.
Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, Padek M. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10:88.
Straus S. Knowledge Translation. In: Straus SE, Tetroe J, Graham ID, editors. Health Care : Moving from Evidence to Practice (2nd Edition). Chichester, West Sussex: Wiley; 2013.
Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. American journal of public health. 2012;102(7):1274.
Graham ID, Tetroe J. Some Theoretical Underpinnings of Knowledge Translation. Acad Emerg Med. 2007;14(11):936–41.
Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. American journal of public health. 2011;101(11):2059.
Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research.( Systematic review). Implementation. Science. 2012;7:17.
Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018;13(1):27.
Cowie J, Campbell P, Dimova E, Nicoll A, Duncan EAS. Improving the sustainability of hospital- based interventions: a study protocol for a systematic review. BMJ Open. 2018;8(9):e025069.
Nilsen P. Making sense of implementation theories, models, and frameworks. Implementation Sci. 2015;10:53. https://doi.org/10.1186/s13012-015-0242-0.
Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. The sustainability of healthcare innovations: a concept analysis. Journal of Advanced Nursing. 2015;71(7):1484–98.
Higuchi KS, Downey A, Davies B, Bajnok I, Waggott M. Using the NHS sustainability framework to understand the activities and resource implications of Canadian nursing guideline early adopters. Journal of Clinical Nursing. 2013;22(11-12):1707–16.
Heslop L, Lu S. Nursing- sensitive indicators: a concept analysis. Journal of advanced nursing. 2014;70(11):2469.
Doran D, Harrison MB, Laschinger H, Hirdes J, Rukholm E, Sidani S, et al. Relationship between nursing interventions and outcome achievement in acute care settings. Research in Nursing & Health. 2006;29(1):61–70.
Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability science: an integrated approach for health- programme planning. Lancet. 2008;372(9649):1579–89.
Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community- based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education Research. 1998;13(1):87–108.
Yost J, Thompson D, Ganann R, Aloweni F, Newman K, McKibbon A, et al. Knowledge Translation strategies for enhancing Nnurses’ evidence- informed decision making: a scoping review. Worldviews Evid Based Nurs. 2014;11(3):156–67.
Yost J, Ganann R, Thompson D, Aloweni F, Newman K, Hazzan A, et al. The effectiveness of knowledge translation interventions for promoting evidence- informed decision- making among nurses in tertiary care: a systematic review and meta- analysis. Implement Sci. 2015;10(1):98.
Doyle C, Howe C, Woodcock T, Myron R, Phekoo K, McNicholas C, Saffer J, Bell D. Making change last: applying the NHS institute for innovation and improvement sustainability model to healthcare improvement. Implement Sci. 2013;8:127.
Dückers M, Wagner C, Vos L, Groenewegen P. Understanding organisational development, sustainability, and diffusion of innovations within hospitals participating in a multilevel quality collaborative. Implement Sci. 2011;6(1):18.
Scheirer MA. Is Sustainability Possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320–47.
Tricco AC, Ashoor HM, Cardoso R, Macdonald H, Cogo E, Kastner M, et al. Sustainability of knowledge translation interventions in healthcare decision- making: a scoping review. Implement Sci. 2016;11(55):55.
Buchanan D, Fitzgerald L, Ketley D, Gollop R, Jones JL, Lamont SS, et al. No going back: a review of the literature on sustaining organizational change. Int J Manag Rev. 2005;7(3):189–205.
Buchanan DA, Fitzgerald L, Ketley D. The sustainability and spread of organizational change : modernizing healthcare. London, New York: New York : Routledge, 2006.; 2006.
Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. Journal of Clinical Epidemiology. 2005;58(2):107–12.
Sales A, Smith J, Curran G, Kochevar L. Models, strategies, and tools. Theory in implementing evidence- based findings into health care practice. J Gen Intern Med. 2006;21(Suppl 2):S43.
Geerligs L, Rankin NM, Shepherd HL, Butow P. Hospital- based interventions: a systematic review of staff-reported barriers and facilitators to implementation processes. Implement Sci. 2018;13(1).
Canadian Insitutue for Health Information (CIHI). National Health Expenditure Trends. Ottawa, ON, CIHI. 1975, 2018. https://www.cihi.ca/en/health-spending/2018/national-health-expenditure-trends.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta- Analyses: The PRISMA Statement. J Clin Epidemiol. 2009;62(10):1006–12.
Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349.
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75(C):40–6.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117.
Meleis AI. Theoretical nursing : development and progress. Fifth Edition.. ed. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins, 2012; 2012.
Walker LO. Strategies for theory construction in nursing. In: Avant KC, editor. . 4th ed. Upper Saddle River, N.J: Pearson Prentice Hall, c2005; 2005.
Elo S, Kyngäs H. The qualitative content analysis process. Journal of Advanced Nursing. 2008;62(1):107–15.
Graneheim U, Lindgren B-M, Lundman B. Methodological challenges in qualitative content analysis: A discussion paper. Nurse Educ Today. 2017;56:29.
Maher L, Gustafson D, Evans A. NHS Sustainability Model and Guide. Coventry, UK: NHS Institute of Innovation and Improvement; 2010.
Slaghuis SS, Strating MM, Bal RA, Nieboer AP. A framework and a measurement instrument for sustainability of work practices in long-term care. BMC Health Serv Res. 2011;11:314.
Frykman M, Von Thiele SU, Muntlin Athlin Å, Hasson H, Mazzocato P. The work is never ending: uncovering teamwork sustainability using realistic evaluation. J Health Organ Manag. 2017;31(1):64–81.
Racine DP. Reliable effectiveness: a theory on sustaining and replicating worthwhile innovations. Adm Policy Mental Health. 2006;33(3):356–87.
Fox AG, G.;Osborne, S. A theoretical framework to support research of health service innovation. Aust Health Rev. 2015;39(1):70-75.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations. Milbank Q. 2004;82(4):581–629.
Ford HJ, Krahn AD, Wise AM, Oliver AK. Measuring Sustainability Within the Veterans Administration Mental Health System Redesign Initiative. Qual Manag Health Care. 2011;20(4):263–79.
Molfenter T, Ford JH 2nd, Bhattacharya A. The development and use of a model to predict sustainability of change in health care settings. Int J Inf Syst Change Manag. 2011;5(1):22–35.
Youngleson M, Provost L, Maher L. Adapting the NHS Sustainability Model for a low-to-middle income setting. Coventry: NHS Institute for Innovation and Improvement; 2010.
Squires JE, Estabrooks CA, Gustavsson P, Wallin L. Individual determinants of research utilization by nurses: a systematic review update. Implement Sci. 2011;6(1):1.
May C. Towards a general theory of implementation.(Report). Implement Sci. 2013;8(1).
Slaghuis SS, Strating MMH, Bal RA, Nieboer AP. A measurement instrument for spread of quality improvement in healthcare. Int J Qual Health Care. 2013;25(2):125–31.
Higuchi KS, Davies BL, Edwards N, Ploeg J, Virani T. Implementation of clinical guidelines for adults with asthma and diabetes: a three- year follow- up evaluation of nursing care. J Clin Nurs. 2011;20(9-10):1329.
Goodman MR, Steckler MA. A model for the institutionalization of health promotion programs. Fam Community Health. 1989;11(4):63–78.
Rogers EM. Diffusion of innovations. 5th ed. New York: Free Press, 2003; 2003.
Graham ID, Logan J. Innovations in knowledge transfer and continuity of care. Can J Nurs. 2004;36(2):89.
Lynch EA, Mudge A, Knowles S, Kitson AL, Hunter SC, Harvey G. "There is nothing so practical as a good theory": a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18(1):857.
Proctor E, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges. Adm Policy Ment Health. 2009;36(1):24–34.
Acknowledgement to Sharon Straus, Julie Bain, Alekhya Mascarenhas Johnson, and Julia Moore for their contribution to initial literature review which supported the background work for the review of existing syntheses in this review. Further acknowledgement to Natalie LeClair for her assistance in the selection of databases, search terms and search strategy for the new systematic review and to Marie-Cécile Domecq for her assistance with the peer review of the electronic search. Acknowledgement to both Marie-Cécile Domecq and Isabelle Castonguay for their assistance in the extraction of databases into working files for screening, Caitlin Pawlett for her assistance formatting the Additional files, and Professor Jan Lundquist for assistance in the proofreading of the final manuscript. IDG is a recipient of a CIHR Foundation Grant, FDN#143237.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
PRISMA 2009 Checklist—Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: A systematic review and theory analysis of selected sustainability frameworks.
Concept key terms and search strategy.
Qualitative analysis of concepts and factors for sustainability frameworks/models/theories.
About this article
Cite this article
Nadalin Penno, L., Davies, B., Graham, I.D. et al. Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: a systematic review and theory analysis of selected sustainability frameworks. Implementation Sci 14, 108 (2019). https://doi.org/10.1186/s13012-019-0952-9
- Theories; Sustainability
- Evidence-based practices/guidelines/programs/interventions