Skip to main content
Download PDF
Download ePub
  • Systematic review
  • Open access
  • Published:

Built to last? Barriers and facilitators of healthcare program sustainability: a systematic integrative review

Implementation Science volume 18, Article number: 62 (2023) Cite this article

Abstract

Objective

To identify barriers and facilitators associated with the sustainability of implemented and evaluated improvement programs in healthcare delivery systems.

Data sources and study setting

Six academic databases were searched to identify relevant peer-reviewed journal articles published in English between July 2011 and June 2022. Studies were included if they reported on healthcare program sustainability and explicitly identified barriers to, and facilitators of, sustainability.

Study design

A systematic integrative review guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. Study quality was appraised using Hawker’s Quality Assessment Tool.

Data collection/extraction methods

A team of reviewers screened eligible studies against the inclusion criteria and extracted the data independently using a purpose-designed Excel spreadsheet. Barriers and facilitators were extracted and mapped to the Integrated Sustainability Framework (ISF). Frequency counts of reported barriers/facilitators were performed across the included studies.

Results

Of the 124 studies included in this review, almost half utilised qualitative designs (n = 52; 41.9%) and roughly one third were conducted in the USA (n = 43; 34.7%). Few studies (n = 29; 23.4%) reported on program sustainability beyond 5 years of program implementation and only 16 of them (55.2%) defined sustainability. Factors related to the ISF categories of inner setting (n = 99; 79.8%), process (n = 99; 79.8%) and intervention characteristics (n = 72; 58.1%) were most frequently reported. Leadership/support (n = 61; 49.2%), training/support/supervision (n = 54; 43.5%) and staffing/turnover (n = 50; 40.3%) were commonly identified barriers or facilitators of sustainability across included studies. Forty-six (37.1%) studies reported on the outer setting category: funding (n = 26; 56.5%), external leadership by stakeholders (n = 16; 34.8%), and socio-political context (n = 14; 30.4%). Eight studies (6.5%) reported on discontinued programs, with factors including funding and resourcing, poor fit, limited planning, and intervention complexity contributing to discontinuation.

Conclusions

This review highlights the importance of taking into consideration the inner setting, processes, intervention characteristics and outer setting factors when sustaining healthcare programs, and the need for long-term program evaluations. There is a need to apply consistent definitions and implementation frameworks across studies to strengthen evidence in this area.

Trial registration

https://bmjopen.bmj.com/content/7/11/e018568.

Peer Review reports

Background

Healthcare system sustainability is the ongoing capacity to deliver affordable and effective care that contributes to better health outcomes over time. There are many threats and challenges to the sustainability of healthcare systems across the world, including an ageing population, increasing costs of delivering healthcare, costly new medical technologies and growing consumer demand [1,2,3,4]. Crises, including natural disasters, large-scale accidents, epidemics, and pandemics add further challenges to already over-stretched healthcare systems [5, 6]. A sustainable healthcare system is one that is also resilient, such that it constantly adapts and endures despite these ever-changing pressures while maintaining performance in terms of health outcomes [7].

As healthcare systems strive towards delivering value-based care within these challenging contexts, improvement programs to increase quality, safety, effectiveness, and efficiency of healthcare have proliferated [8]. Quality improvement programs are now ubiquitous across healthcare sectors and facilities. Such programs are important to support innovations where new and more effective health technologies, models of care delivery, and financing are adopted and ideally, while old ineffective and inefficient ones are phased out.

To sustain the benefits from innovations in healthcare, innovations must be empirically evaluated to ensure they are indeed effective and deliver the outcomes that they promise, at scale and across different contexts. However, large-scale innovations are rare in healthcare systems and most innovations consist of improvement projects that tend to be short-term and proscribed—implemented in single centres or regions [1, 2]. Despite its importance, the sustainability of implemented improvement programs is under-researched with a limited evidence base to support decisions [9]. For example, discontinuing effective programs because of a lack of ongoing investment is wasteful and unethical [10]. Our understanding of how and why programs implemented in the real-world are sustained or discontinued is also limited, often because evaluations of improvement programs are almost always performed over the short term [11, 12]. The need for continuing investment in effective programs is well recognised as an important factor of sustainability, however, it is not the only factor [11, 12]. Shelton et al. [12] proposed the Integrated Sustainability Framework (ISF), which identifies important factors that help or hinder program sustainability. The ISF includes inner contextual factors (i.e., program champions, leadership/support, organisational resources/funding, staffing/turnover) and outer contextual factors (i.e., socio-political context, funding environment, external leadership, and values, needs and priorities), characteristics of the interventions or programs (i.e., perceived benefit/need, adaptability, and fit with context and population), characteristics of people or institutions implementing these programs (i.e., implementer/provider characteristics, implementer skills/expertise) and the processes used for implementation (i.e., partnership/engagement, training/supervision, program evaluation/data, adaptation) [12].

Healthcare system sustainability as applied to programs that are implemented in the healthcare delivery system are poorly defined and understood conceptually. In their systematic review of 125 studies of program sustainability published up until 2011, Stirman et al. [11] identified gaps in the application of definitions of sustainability when developing and implementing programs. Studies included in their review seldom reported definitions in sufficient detail to be able to assess sustainability in a systematic manner [11]. A recent systematic integrative review [1] and scoping review [2] also found gaps and inconsistencies around the definitions of program sustainability, with less than 30% of studies providing the definitions in both reviews. Sustainability was often discussed as an extension of implementation, with many studies reporting that at the end of a 1-to-2-year implementation project, the program “was sustainable” without providing long-term outcomes or specific measures of sustainability [1].

Rationale

The factors that act as barriers/facilitators for the sustainability of healthcare programs are inadequately reported and are poorly understood. The determinants of successful implementation are often reported; however, these are likely to be quite different to the factors related to sustainability [10, 13]. For example, factors such as trialability, intervention fidelity and factors associated with the inner setting are often talked about with reference to successful implementation. Different factors are more likely to be important for health program sustainability and program scaling, including outer setting factors such as socio-political and funding environment, external leadership, and values, needs and priorities of communities and populations, data and evaluation to demonstrate value and to support adaptations as contexts change [13]. Stirman and colleagues [11] identified gaps in research on public health program adaptations and factors that drive sustainability such as organisational context and capacity, processes and characteristics of implemented programs. A deeper understanding of barriers to, and facilitators of, program sustainability is essential to support the development, implementation, and evaluation of innovative healthcare programs, to support decision-making around program continuation, adaptation, scale-up, and diffusion, and to maximise the long-term benefits of programs. It is similarly important to understand the encountered barriers and contexts that lead to program discontinuation. To develop this understanding, a review and synthesis of current knowledge, guided by a theoretical framework such as the ISF [12] is needed.

Objectives

Our primary objective was to build on the studies of Stirman et al. [11] and Braithwaite et al. [1] to identify the barriers and facilitators associated with the sustainability of implemented and evaluated improvement programs in healthcare delivery systems, and then to map them to the ISF. Our study also aimed to discern the extent to which the discontinuation of healthcare programs was reported in the literature and to identify factors that led to these programs being discontinued.

Methods

The review forms part of a body of research investigating the sustainability of healthcare programs, seeking to bring it up to date [1]. The search strategy, study selection and quality assessment mirror those outlined in a published integrative review on this topic [1], with the present updated review conducted in June 2022. The data relating to barriers and facilitators, the analysis and synthesis of data, the results, and the implications and conclusions drawn from these findings are unique to the current review. The review was guided by the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) statement (Table S1) [14].

Protocol and registration

The published protocol for this review can be found at https://bmjopen.bmj.com/content/7/11/e018568 [15]. Modifications to the protocol have been previously published and the details of the updated search are described here [1].

Search strategy

The search strategy was developed in consultation with two medical librarians and included six academic databases: CINAHL, EMBASE, Ovid MEDLINE, Emerald Management, Scopus and Web of Science [1]. Additional studies were identified by hand searching reference lists of relevant systematic reviews. The search strategies for all databases are provided in Table S2.

Study selection

The selection process has been previously described [1]. The reviewers had varying degrees of experience in conducting systematic reviews, and 13 out of the 16 authors had previously been an author on at least one systematic review study. A blinded review of 5% of titles and abstracts was undertaken, and discrepancies were discussed among the reviewers (KL, LT, HA, JHD, GL, EM, KH, AC, CLS, LVB, LAE, and GD), with two reviewers (YZ and JB) acting as arbitrators, until a consensus was reached. The remaining screening of abstracts and titles was undertaken in Rayyan [16], a web and mobile app for systematic reviews, according to the inclusion criteria with records randomly allocated among the reviewers [1]. Publications were assessed against the following inclusion criteria: (1) English-language, (2) peer-reviewed journal article, (3) primary empirical research, (4) published July 2011–June 2022, (5) healthcare setting, (6) evaluation of program, (7) assessment of program sustainability, and (8) focus on changes/improvements to the healthcare system.

Studies reporting on public health programs including population-based prevention programs, community-level outcomes only, or patient-level outcomes only were excluded. Studies that did not identify barriers to, or facilitators of, program sustainability, and studies that reported barriers or facilitators of implementation only were also excluded. Studies included at the abstract-review stage were re-assessed against the inclusion criteria during full-text review. Based on our previous integrative review [1] of 92 studies and drawing on the work of Stirman et al. [11], Shelton et al. [12], and Scheirer and Dearing [10], health program sustainability was conceptualised from a systems or organisational view-point. Therefore, studies were included if they reported on the following:

  1. a)

    Evaluation of a program after funding had ended, or after the initial staff training or implementation phase; and

  2. b)

    Explicitly assessed sustainability, either using qualitative, quantitative, or mixed methods, for example, stakeholders’ views of sustainability, evidence of ongoing care delivery under the program, ongoing funding; or,

  3. c)

    Longitudinal studies, for example, evaluations conducted over multiple time points.

Data collection processes and data items

Following previously-described work [1], a purpose-designed Excel spreadsheet was utilised for data extraction. The spreadsheet was piloted by reviewers on two studies. The remaining studies were randomly assigned to the reviewers for data extraction. Verification of the accuracy and meanings of the extracted data was undertaken independently by seven reviewers (LT, AC, PNAD, CLS, NH, and YZ). Any discrepancies were resolved through team discussions during regular meetings (over 10 group meetings were held).

Data analysis and synthesis

It was often not possible to classify factors influencing sustainability in a binary way, i.e., as either a barrier or facilitator. For example, the degree to which a program was sustained may have been influenced by a high (facilitator) or low (barrier) level of leadership. As such, barriers and facilitators were conceptualised as part of a single construct, representing two ends of a spectrum.

Barriers/facilitators of program sustainability were synthesised using the ISF [12]. The ISF embodies 36 “emerging factors” grouped together under five categories: outer setting, inner setting, intervention characteristics, processes, and implementer and population characteristics. Shelton et al. [12] did not provide definitions for each of their emerging factors making it difficult for the reviewers to classify some of the barriers/facilitators. To overcome this challenge, working definitions for the ISF emerging factors were developed by two reviewers (LT and HA) based on relevant literature and other frameworks including Weiner et al. [17] and the Consolidated Framework for Implementation Research (CFIR) [18]. The proposed definitions were discussed with the broader review team before being applied during data analysis and interpretation (Table 1). Frequency counts were then performed for each emerging factor (Table 2), in addition to a qualitative narrative synthesis. For the purpose of this review, emerging factors will henceforth be referred to as “barriers/facilitators”.

Table 1 Definitions/operationalisations of emerging factors in the Integrated Sustainability Framework
Full size table
Table 2 Number of included papers reporting on barriers to and/or facilitators of program sustainabilitya, organised according to the Integrated Sustainability Framework [12]
Full size table

To identify critical barriers to program sustainability and how these might be overcome, one part of our analysis focused on programs that were discontinued. To identify critical facilitators of longer-term sustainability, the review also focussed on reports of programs that were sustained for at least 5 years after funding, training, or the implementation period ended, rather than the more commonly reported time points of 1 to 3 years at the end of trial funding when it is difficult to separate factors related with implementation from those related with sustainability [1]. There are currently no specific agreed or pre-determined time points at which a program is deemed to be sustainable. Thus, informed by the literature, especially published reviews [1, 11, 12, 20, 21] and, after team discussions, we concentrated on programs that had been sustained for 5 years or longer after funding, staff training or the implementation period or the trial had ended.

Quality assessment

The quality of included studies was assessed by ten reviewers (KL, LT, HA, JHD, GL, AC, PNAD, CLS, GD, and NS) using Hawker’s Quality Assessment tool [22] and Lorenc et al.’s quality ratings [23] (low, medium, high). A blinded quality assessment of a randomly selected 6% (n = 7) sample of included studies was conducted to ensure consistency of ratings among the reviewers. The remaining studies were randomly assigned to individual reviewers and any queries were discussed and resolved in a team meeting.

Results

Study selection

A total of 11,443 studies were screened after duplicates were removed. At the title/abstract review stage, 10,845 out of the 11,443 were excluded, leaving 598 studies progressing to full-text review, with 124 studies being retained for data analysis and synthesis (Fig. 1). The main reasons for exclusion at full-text review were that the publication did not assess sustainability or discontinuation of a program (n = 185), did not focus on change improvements in the healthcare system (n = 91), or no evaluation of a program was reported (n = 102).

Fig. 1
figure 1

PRISMA flowchart of studies identified for relevant studies for inclusion in the review

Full size image

Study characteristics and quality assessment

Fifty-two studies (41.9%) reported qualitative results, 37 (29.8%) used mixed methods and 35 (28.2%) reported quantitative results. Most studies were longitudinal with assessment of outcomes at different time points (n = 55; 44.4%), in addition to case studies (n = 32; 25.8%), and cross-sectional studies (n = 30; 24.2%). Thirty-five countries were covered by the 124 studies, with seven including more than one country. The majority of studies originated from the USA (n = 43; 34.7%), Canada (n = 14; 11.3%), the United Kingdom (n = 11; 8.9%), and Australia (n = 11; 8.9%). Eighty-four studies scored 30–36 points (high quality), 35 scored 24–29 points (moderate quality), and 5 scored less than 24 points and were considered low quality on Hawker’s Quality Assessment Tool [22, 23] (Table S3). No studies were excluded as all studies were deemed as providing sufficient information related to sustainability.

Barriers and facilitators

Table 2 summarises the number of studies that explicitly referred to the barriers/facilitators under the ISF as defined in Table 1. Detailed data for each barrier/facilitator including frequency counts are provided in Table S4. The most commonly identified barriers/facilitators were related to the ISF inner setting category (n = 99; 79.8%) and the processes category (n = 99; 79.8%), (Table 2).

Inner setting

Barriers/facilitators related to the inner setting were reported by 99 (79.8%) studies (Table 2). Organisational factors commonly identified by studies as influencing intervention sustainability were leadership (n = 61; 61.6%), staffing/turnover (n = 50; 50.5%), climate/culture (n = 42; 42.4%), and funding (n = 37; 37.4%). Committed leadership, especially from formally appointed leaders, was reported to play a critical role in sustaining interventions [24, 25]. Brewster et al. [26] described the presence of a small number of key staff members to maintain an intervention in place and demonstrable commitment by management over time as important facilitators. Staff turnover was often cited as a barrier to continuation of interventions, which related to the need to re-train new staff whilst dealing with staff shortages to deliver the program [25, 27,28,29].

The climate/culture of the inner setting were also commonly discussed, as these factors created a supportive environment for change implementation. Supportive work culture [30] and providing rewards and recognition [27, 31] were recognised facilitators. Conversely, factors including lack of a unified identity and poor accountability [32], and staff and institutional resistance [33, 34] acted as barriers to sustainability. Availability of funding facilitated the delivery of an intervention, independent of the cost of the intervention itself, however, inadequate resources to support and expand interventions presented a barrier to continuation [35].

Processes

The processes related to program sustainability were reported by 99 (79.8%) studies and included the availability of training/supervision (n = 54; 54.5%), program evaluations and data (n = 40; 40.4%), and communication (n = 40; 40.4%) (Table 2). The provision of regular staff training about new programs, for both newly hired staff and experienced staff members, was an identified facilitator for program sustainability [25, 35,36,37] whereas a lack of training was identified as a barrier [25, 35, 38]. The importance of program evaluation and regular feedback of data concerning program outcomes to staff members involved in the implementation and to stakeholders, was highlighted by at least seven studies [34, 39,40,41,42,43,44]. Positive program outcomes that were regularly communicated and visible to staff members were linked with program sustainability [37, 45,46,47] and the converse hindered program sustainability [48, 49].

Efficient and ongoing communication among stakeholders involved in the program [24, 31, 50] and strong collaborative partnerships facilitated program sustainability [51,52,53]. Clear roles and responsibilities as well as mutual trust among stakeholders to fulfil their unique responsibilities were also reported as facilitators [50, 54]. Decreased communication among stakeholders after program implementation was a recognised barrier, for example, limited communication after implementation of a multidisciplinary hospital‐based surgical program threatened sustainability in some locations [55].

Intervention characteristics

Intervention characteristics were reported in 72 studies (58.1%), with a good “fit” and alignment of interventions with existing systems and local contexts facilitating sustainability reported in 37 studies (51.4%). For example, facility-based consultations in eye care interventions in Ghana were more likely to be routinised due to a high level of compatibility with the hospitals’ mandate, whereas outreach activities were less likely to be sustained due to a low level of compatibility and lack of role clarity [56]. Intervention adaptations to overcome challenges in resource-limited settings bolstered sustainability by improving fit with the population’s needs and context. For example, across Uganda, shifting intervention delivery from physicians to other staff (nurses and pharmacists) in an anti-retroviral therapy program and adopting greater task sharing with non-physician staff, supported program sustainability [57]. Urquhart et al. [58] also reported the importance of adapting interventions to improve fit with cancer survivors’ needs in different settings, including transitioning to online delivery and tailoring of tools.

Furthermore complex interventions were less sustainable [24, 59], while simpler interventions were reported to be more sustainable [26, 53, 60].

Outer setting

Factors related to the outer setting category were reported by 46 (37.1%) studies, with funding (n = 26; 56.5%), external leadership by stakeholders (n = 16; 34.8%) and socio-political context (n = 14; 30.4%) reported most frequently (Table 2). Support and leadership from external stakeholders facilitated intervention sustainability [35, 61, 62]. External contextual factors, such as funding environment and socio-political context, were found to both positively and negatively influence intervention sustainability. For example, Bond et al. [35] found that adequate financing facilitated sustainability, whereas Fleizer et al. [32] and Olumide et al. [29] reported that insecure sources of funding challenged the continuation of programs. A high dependence on time-limited funding from external donors created a barrier to sustainability, especially when other funding sources were not planned for in the longer term [62, 63].

Ongoing involvement of leadership in the region [49] and at national level [61] were thought to facilitate sustainability while a lack of support from governments created barriers to program sustainability [38, 64]. Furthermore, the socio-political context was reported as an influencing factor by 14 studies. For example, De Neve et al. [64] found that political turnovers and instability led to discontinuation of a program as ‘political actors’ and priorities changed. A mismatch between the program activities and values, priorities and needs were also identified as a barrier [64,65,66]. Socio-political factors were also reported as important facilitators. For example, political and financial stability and perceived value of the implemented programs among external stakeholders were thought to support the sustainability of HIV/AIDS relief programs [67]. In addition, joint planning between the donor, non-governmental organisations, health facilities and government enabled stakeholders, especially local governments, develop a better understand their health system needs and therefore to sustain effective health investments [67]

Implementer and population characteristics

Factors related to implementer and population characteristics were reported by 44 studies (35.5%), and half of them reported that general attitudes of implementers of new programs (n = 23; 52.3%) both positively and negatively affected sustainability (Table 2). Implementers being realistic in their expectations, including realistic timelines, adequate resourcing, ongoing engagement with staff delivering the program, and with program recipients, facilitated sustainability [65]. Staff members’ beliefs about the advantages of programs facilitated sustainability [47, 55, 68], whereas negative attitudes and fear of change were barriers [37, 69, 70]. Staff members who perceived that a new program would have negative consequences for their autonomy and workload was identified as a barrier [27, 69].

The right skills and level of expertise were also identified as implementer characteristics that positively influenced the delivery of sustained programs. For example, a rural volunteer program in Canada underlined that volunteer coordinators with sufficient skills and expertise, who also trained and mentored others, was an important facilitator [36]. However, Fox et al. [48] found that underuse of highly experienced and skilled staff might lead to job dissatisfaction and staff attrition, posing a barrier. For example, emergency nurses were concerned about deskilling and underutilisation after acquiring new high-level skills which were not required to care for low acuity patients [48].

Identifying barriers that resulted in program discontinuation, and facilitators of long-term program sustainability

Eight of the 124 studies (6.5%) explicitly referred to the discontinuation of programs [24, 29, 35, 71,72,73,74,75]. Table 3 summarises factors associated with discontinuation, including lack of financial viability; workforce issues (lack of trained workforce, strict role boundaries, competing demands on staff time, poor preparation, training and planning); and lack of engagement and misunderstandings between implementers and staff expected to deliver the program. Misalignment with existing policies and workflows, lack of ongoing support from the implementation team and multiple changes being implemented at the same time also contributed to program discontinuation (Table 3).

Table 3 Summary of studies reporting on programs that were discontinued, illustrating common barriers of sustainability (n = 8)
Full size table

A total of 29 studies (23.4%) reported that the programs had been sustained for 5 years or longer (Table S5). Five example studies reporting on programs that were sustained for 5 years or more are summarised in Table 4. These five example programs were selected because they demonstrate a wide variety of factors that supported program sustainability. Eight of the 29 studies (27.6%) reported that program adaptability and/or adaptation were the key facilitators of long-term program sustainability. For example, most health facilities implementing a multi-site anti-retroviral therapy (ART) scale-up program in Uganda modified and tailored the intervention in order to improve fit with their resource-constrained conditions thereby fostering long-term sustainability between 2004 and 2014 [57]. Another study by Oliveira et al. [76] highlighted that ongoing monitoring and adaptation of the Family Health Program (FHP) in response to critical events were deemed as strategic facilitating factors for the sustainability of the program for 12 years.

Table 4 Five examples describing programs sustained for 5 years or more and illustrating common facilitators of sustainabilitya
Full size table

Multi-site studies demonstrated the importance of understanding the local contexts and several studies reported that the programs were sustained in one context but not in another. Vidgen et al. [24] demonstrated such contextual differences by highlighting that the decision to outsource a program to an external provider under a limited time contract was a barrier to sustainability. On the other hand, Zakumumpa et al. [25, 75] aptly demonstrated both sustainability and discontinuation in different sites to show significant barriers (Table 3) and facilitators (Table 4) related to local context.

Discussion

Our systematic integrative review demonstrated that the literature on the sustainability of innovations or improvement programs in healthcare is developing. Barriers and facilitators of healthcare program sustainability were identified and mapped to the ISF using our working definitions, with the most prevalent barriers/facilitators relating to inner setting (79.8%) and processes (79.8%). The review identified important gaps including limited long-term program evaluations. Studies often claimed program sustainability even at 1 or 2 years after implementation. Longer-term evaluations are needed to confirm such claims as few studies (n = 29; 23.4%) reported on program sustainability 5 years or more after implementation. Short-term evaluations were common which is not surprising given the approach often taken by health organisations and governments when implementing improvement programs in the healthcare system [79]. Short-term funding limits capacity to rigorously evaluate, adapt, sustain and scale programs over the longer-term.

Leadership and support emerged as a key influencer in program sustainability. Consistent with the findings from the systematic reviews by Cowie et al. [80] and Penno et al. [81] using Lennox’s consolidated framework [82], our review suggests that the support of leaders plays a critical role in achieving sustained programs. Enthusiasm and support of leaders, however, is not enough to effectively support and sustain healthcare programs without considerable skill, expertise, and capacity of these leaders. Ambitious leadership without sufficient managerial skills and technical experience can negatively impact sustainability of healthcare programs due to a loss of focus on the program after implementation [83].

Workforce issues, such as high staff turnover, were identified as common barriers to sustainability. This finding is consistent with other reviews [21, 80]. Our review identified that program discontinuation could be attributed to staff turnover associated with lack of adequate training and trained staff, lack of incentives and recognition, and competing priorities [29, 75]. In contrast, Shelton et al. [12] reported that the nature and influence of processes, including staff training, were barriers/facilitators less often reported to be associated with sustainability. Ninety-nine studies (79.8%) included in our review reported at least one factor under the processes category, with more than half mentioning training/support/supervision. The role of training was essential to equip staff with skills and knowledge required to deliver program interventions and to maintain fidelity [34, 62]. Ensuring adequate time and resources to train staff as required, and not only at the beginning of implementation, should be considered in planning new programs that are intended to last. New programs require new roles and new role descriptions, which should be developed, maintained, and updated to ensure role clarity, responsibility, and scope within the program and the context within which the program is being sustained.

Much of the literature about healthcare system sustainability is focussed on factors that make programs and systems last over time. This, of course, is sensible, however it provides a one-sided view. One of the unique aspects of our review is the analysis of factors that led to the discontinuation of programs. Understanding why programs cease is a critical complement to our understanding of factors that make programs sustainable. Increasingly, it is being recognised that many programs may continue despite becoming ineffective, inefficient, or no longer needed [84]. Therefore, the strident quest for sustainability of all implemented programs may be inappropriate and may in itself contribute to the wider system unsustainability, as maintaining ineffective, inefficient or defunct health programs can contribute to waste or low-value care [84, 85].

A greater emphasis in the literature on reporting on ineffective or unsustainable programs would enrich our understanding of factors associated with program sustainability and may also prevent others from wasting efforts and investment. However, there were only eight studies out of the 124 included in our review (6.5%) that reported on discontinuation. Publication bias, where negative results are less likely to be published, may be a factor skewing the literature towards successful ongoing programs [86]. This limited literature restricts our understanding of the factors that lead to discontinuation, or how the decision to discontinue was influenced, made, or planned. All eight studies describing discontinued programs, or parts of programs, argued that their programs should have been sustained; however, scaling-up healthcare programs must be accompanied by appropriate long-term monitoring and ongoing evaluation to ensure that decisions to sustain, adapt or discontinue are evidence-based [84].

In line with our review, two other systematic reviews applied the ISF to assess barriers/facilitators related to program sustainability [20, 21]. One of these, by Shoesmith et al. [21] provided specific factor definitions as applied in the context of schools and/or childcare services. In line with the results of Shoesmith et al. [21] and Braithwaite et al. [1] our review also found that factors predominantly related to the inner setting were reported to facilitate intervention sustainment, including leadership support. Another review by Hall et al. [20] applied the factor definitions developed by Shoesmith et al. [21] in a review of clinical, public health, and community health services. Hall et al. [20] focused on the measurement of sustainability and determinants of sustainability, and found that 28 individual measures were used among 223 articles, but only 2 of these measures specifically assessed sustainability as an outcome and one assessed both sustainability outcomes and determinants. This is an important gap also illustrated in our previous review [1], where only 6 out of 92 included studies (6.5%) reported using measures of program sustainability in healthcare delivery settings. This demonstrates a need for clear and consistent definitions and measures of sustainability that are relevant and applicable to the field being studied, in our case, healthcare. Whilst undertaking this review, it became necessary to develop working definitions guided by the literature that could be applied by the review team to operationalise the ISF and ensure consistency of interpretation of emerging factors associated with program sustainability. These new definitions should be considered and improved upon as new research emerges, to improve the consistency of assessments using the ISF.

Strengths and limitations

Our review builds on our previous work [1] and extends the work of others [2, 11, 12] by describing in detail the factors contributing to program sustainability in healthcare settings, whilst guided by a published framework, the ISF. This review enhances evidence about the sustainability of healthcare programs by identifying and mapping barriers to and/or facilitators of a sustainability framework that support sustainability or contribute to program discontinuation. The analysis of factors that led to the discontinuation of programs is also a strength and provides important learnings to guide future healthcare program planning to avoid known barriers. Few reviews have specifically addressed program discontinuation, possibly because there are fewer publications about discontinued programs, potentially due to the known publication bias to publish positive results [86].

The application of the ISF as an underpinning theory to map barriers and facilitators of sustainability is another strength of our review, which builds on this framework by providing a working definition of each emerging factor. Importantly, we conceptualised the factors on a continuum rather than binary facilitators or barriers, for example policy and legislation may be a barrier in some settings and a facilitator in others and this may change over time. Although working definitions of emerging factors were developed for this review, they have not been applied by others to determine their validity, consistency, and applicability to the study of sustainability of healthcare programs. Further work is needed to ensure the longer-term usability or adaptation of these definitions.

Double-blinded abstract and full-text reviews were conducted in addition to many team meetings to ensure consistency of study screening, inclusion and interpretation. Interpretation of how studies conceptualised sustainability required significant discussion by the whole team especially in studies that reported on sustainability at 12 months or less after implementation—where implementation and sustainability may have been conflated. Therefore, the decision to concentrate on programs that were sustained for 5 years or more adds further validity to our findings.

The large variety of definitions of sustainability, and the failure to provide definitions of sustainability for studies that report on sustainability, has been continuously highlighted as a major limitation in previous reviews on sustainability [12, 21, 80], including in our previous reviews [1, 2]. Reviewing the literature on healthcare program sustainability was made more challenging by the large body of literature on environmental sustainability, limiting the pace of screening for inclusion and exclusion. Moreover, the heterogeneous nature of the current included studies, including lack of clear definitions or inconsistent definitions, made synthesis of the literature challenging. The exclusion of grey literature, studies published in languages other than English may mean that other relevant studies could have been missed, limiting the comprehensiveness of the evidence synthesis.

Implication for practice and research

Our findings add to the understanding of which factors hampered or facilitated the sustainment of healthcare programs and complements previous reviews on program sustainability. The sustainability of healthcare system improvements in our review mapped to the inner setting and processes category of the ISF with leadership/support, training/support/supervision, and staffing/turnover being the most frequently reported barriers/facilitators. In line with other reviews [80, 81], the results suggest that these barriers/facilitators should be prioritised in the sustainability phase of programs and considered in light of organisational readiness and ongoing resources for program delivery. However, it should be noted that this suggestion is based on the findings from the 124 studies in our review, which were mainly located in high-income countries.

Aligned with the literature on program implementation, factors related to the inner setting and processes were commonly reported in studies included in our review. However, the outer contextual factors such as the socio-political context, funding environment, external leadership, and values, needs and priorities of stakeholders and populations were addressed in over a third of the selected studies. Furthermore, the importance of process factors that to some extent overlap with those related to the outer setting, such as partnerships and engagement with stakeholders, effective communication with stakeholders and evaluations and data were discussed, especially in studies reporting on programs that were sustained for more than 5 years. As implementation and sustainability are on a continuum, groups designing and planning health programs should consider these outer setting and process factors that have been reported to impact health program sustainability.

Despite rapidly growing literature about healthcare program sustainability, there remains a lack of conceptual clarity in defining and assessing sustainability. Furthermore, frameworks such as the Consolidated Framework for Implementation Research [18] and RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) [87] have been applied predominantly in the context of program implementation rather than sustainability. Applying these frameworks can be challenging when discerning the barriers/facilitators contributing to implementation as opposed to sustainability. Hall et al. [20] recommended the careful consideration of measures of determinants of sustainability that align with the construct of interest, such as objective and settings, to ensure robustness and relevancy of the program evaluation to sustainability. Moreover, it is important for future studies evaluating program sustainability to provide operational definitions of sustainability and clear evaluation timeframes as well as being explicit about theoretical frameworks to underpin their work.

Conclusions

Strong leadership and stakeholder engagement, supportive organisational culture/climate, intervention fit with context and policy, intervention simplicity, adaptability, and fit with need and context, were all important factors in program sustainability. Adequate resourcing including ongoing availability of funding, training for staff, and low staff turnover, as well as constrained intervention costs, and alignment with organisational or broader policy or strategy were also strongly associated with program sustainability. Our review identifies the need for greater use of clear definitions of program sustainability and the application of validated frameworks in future research in this field. To that end, this review provided a working definition of each factor in the ISF to ensure consistency in defining barriers/facilitators associated with sustainability. Furthermore, a greater understanding of the factors associated with discontinuation of healthcare programs is needed, and this can only occur if negative outcomes are published to address the likely publication bias towards positive findings.

Availability of data and materials

All data generated or analysed in this study are included in the published article.

Abbreviations

ISF:

Integrated Sustainability Framework

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analysis

RE-AIM:

(Reach, Effectiveness, Adoption, Implementation, Maintenance)

References

  1. Braithwaite J, Ludlow K, Testa L, et al. Built to last? The sustainability of healthcare system improvements, programmes and interventions: a systematic integrative review. BMJ Open. 2020;10(6):e036453. https://doi.org/10.1136/bmjopen-2019-036453.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Zurynski Y, Herkes-Deane J, Holt J, et al. How can the healthcare system deliver sustainable performance? A scoping review. BMJ Open. 2022;12(5):e059207. https://doi.org/10.1136/bmjopen-2021-059207.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Braithwaite J, Mannion R, Matsuyama Y, Shekelle PG, Whittaker S, Al-Adawi S. Healthcare systems: future predictions for global care. Boca Raton: CRC Press; 2018. p. 600.

  4. World Economic Forum. Sustainable health systems—Visions, strategies, critical uncertainties and scenarios. Switzerland: World Economic Forum Geneva; 2013.

    Google Scholar 

  5. Hanefeld J, Mayhew S, Legido-Quigley H, et al. Towards an understanding of resilience: responding to health systems shocks. Health Policy Plan. 2018;33(3):355–67. https://doi.org/10.1093/heapol/czx183.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Nuzzo JB, Meyer D, Snyder M, et al. What makes health systems resilient against infectious disease outbreaks and natural hazards? Results from a scoping review. BMC Public Health. 2019;19(1):1310. https://doi.org/10.1186/s12889-019-7707-z.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Braithwaite J, Tran Y, Ellis LA, Westbrook J. Inside the black box of comparative national healthcare performance in 35 OECD countries: issues of culture, systems performance and sustainability. PLoS One. 2020;15(9):e0239776. https://doi.org/10.1371/journal.pone.0239776.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Hibbert PD, Basedow M, Braithwaite J, Wiles LK, Clay-Williams R, Padbury R. How to sustainably build capacity in quality improvement within a healthcare organisation: a deep-dive, focused qualitative analysis. BMC Health Serv Res. 2021;21(1):588. https://doi.org/10.1186/s12913-021-06598-8.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88. https://doi.org/10.1186/s13012-015-0274-5.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–67. https://doi.org/10.2105/ajph.2011.300193.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17. https://doi.org/10.1186/1748-5908-7-17.

    Article  Google Scholar 

  12. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39(1):55–76. https://doi.org/10.1146/annurev-publhealth-040617-014731.

    Article  PubMed  Google Scholar 

  13. Palinkas LA, Spear SE, Mendon SJ, et al. Conceptualizing and measuring sustainability of prevention programs, policies, and practices. Transl Behav Med. 2019;10(1):136–45. https://doi.org/10.1093/tbm/ibz170.

    Article  PubMed Central  Google Scholar 

  14. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Braithwaite J, Testa L, Lamprell G, et al. Built to last? The sustainability of health system improvements, interventions and change strategies: a study protocol for a systematic review. BMJ Open. 2017;7(11):e018568. https://doi.org/10.1136/bmjopen-2017-018568.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. https://doi.org/10.1186/s13643-016-0384-4.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Weiner BJ, Amick H, Lee S-YD. Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev. 2008;65(4):379–436. https://doi.org/10.1177/107755870831780.

    Article  PubMed  Google Scholar 

  18. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15. https://doi.org/10.1186/1748-5908-4-50.

    Article  Google Scholar 

  19. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2(1):1–9. https://doi.org/10.1186/1748-5908-2-40.

    Article  Google Scholar 

  20. Hall A, Shoesmith A, Doherty E, et al. Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review. Implement Sci. 2022;17(1):1–28. https://doi.org/10.1186/s13012-022-01252-1.

    Article  Google Scholar 

  21. Shoesmith A, Hall A, Wolfenden L, et al. Barriers and facilitators influencing the sustainment of health behaviour interventions in schools and childcare services: a systematic review. Implement Sci. 2021;16(1):62. https://doi.org/10.1186/s13012-021-01134-y.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Hawker S, Payne S, Kerr C, Hardey M, Powell J. Appraising the evidence: reviewing disparate data systematically. Qual Health Res. 2002;12(9):1284–99. https://doi.org/10.1177/1049732302238251.

    Article  PubMed  Google Scholar 

  23. Lorenc T, Petticrew M, Whitehead M, et al, eds. Crime, fear of crime and mental health: synthesis of theory and systematic reviews of interventions and qualitative evidence. Public Health Res. 2014;2(2). https://doi.org/10.3310/phr02020.

  24. Vidgen HA, Love PV, Wutzke SE, et al. A description of health care system factors in the implementation of universal weight management services for children with overweight or obesity: case studies from Queensland and New South Wales Australia. Implement Sci. 2018;13(1):1–15. https://doi.org/10.1186/s13012-018-0801-2.

    Article  Google Scholar 

  25. Zakumumpa H, Kwiringira J, Rujumba J, Ssengooba F. Assessing the level of institutionalization of donor-funded anti-retroviral therapy (ART) programs in health facilities in Uganda: Implications for program sustainability. Glob Health Action. 2018;11(1):1523302. https://doi.org/10.1080/16549716.2018.1523302.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Brewster AL, Curry LA, Cherlin EJ, Talbert-Slagle K, Horwitz LI, Bradley EH. Integrating new practices: a qualitative study of how hospital innovations become routine. Implement Sci. 2015;10(1):168. https://doi.org/10.1186/s13012-015-0357-3.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Cranley LA, Hoben M, Yeung J, Estabrooks CA, Norton PG, Wagg A. SCOPEOUT: sustainability and spread of quality improvement activities in long-term care-a mixed methods approach. BMC Health Serv Res. 2018;18(1):1–10. https://doi.org/10.1186/s12913-018-2978-0.

    Article  Google Scholar 

  28. Gunaratnam P, Schierhout G, Brands J, et al. Qualitative perspectives on the sustainability of sexual health continuous quality improvement in clinics serving remote Aboriginal communities in Australia. BMJ Open. 2019;9(5):e026679. https://doi.org/10.1136/bmjopen-2018-026679.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Olumide AO, McGuire C, Calhoun L, Speizer I, Babawarun T, Ojengbede O. Factors promoting sustainability of NURHI programme activities in Ilorin and Kaduna, Nigeria: findings from a qualitative study among health facility staff. BMJ Open. 2020;10(10):e034482. https://doi.org/10.1136/bmjopen-2019-034482.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Hoben M, Ginsburg LR, Norton PG, et al. Sustained effects of the INFORM cluster randomized trial: an observational post-intervention study. Implement Sci. 2021;16(1):1–14. https://doi.org/10.1186/s13012-021-01151-x.

    Article  Google Scholar 

  31. Schuller KA, Kash BA, Gamm LD. Studer Group®’s evidence-based leadership initiatives: comparing success and sustainability in two health systems. J Health Organ Manag. 2015;29(6):684–700. https://doi.org/10.1108/JHOM-10-2013-0211.

    Article  PubMed  Google Scholar 

  32. Fleiszer AR, Semenic SE, Ritchie JA, Richer MC, Denis JL. A unit-level perspective on the long-term sustainability of a nursing best practice guidelines program: an embedded multiple case study. Int J Nurs Stud. 2016;53:204–18. https://doi.org/10.1016/j.ijnurstu.2015.09.004.

    Article  PubMed  Google Scholar 

  33. Ford JH 2nd, Krahn D, Wise M, Oliver KA. Measuring sustainability within the Veterans Administration Mental Health System Redesign initiative. Qual Manag Health Care. 2011;20(4):263–79. https://doi.org/10.1097/QMH.0b013e3182314b20.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Kavanagh AY, O’Brien LJ, Maloney SR, Osadnik CR. Barriers and facilitators to adopting functional maintenance initiatives for acutely hospitalised older adults. Disabil Rehabil. 2020;42(26):3808–15. https://doi.org/10.1080/09638288.2019.1610802.

    Article  PubMed  Google Scholar 

  35. Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. Long-term sustainability of evidence-based practices in community mental health agencies. Adm Policy Ment Health. 2014;41(2):228–36. https://doi.org/10.1007/s10488-012-0461-5.

    Article  PubMed  Google Scholar 

  36. Duggleby W, Pesut B, Warner G, et al. Sustainability of a rural volunteer program (Nav-CARE): a case study. Rural Remote Health. 2020;20(4):6112. https://doi.org/10.22605/RRH6112.

    Article  PubMed  Google Scholar 

  37. Fossey J, Garrod L, Tolbol Froiland C, Ballard C, Lawrence V, Testad I. What influences the sustainability of an effective psychosocial intervention for people with dementia living in care homes? A 9 to 12-month follow-up of the perceptions of staff in care homes involved in the WHELD randomised controlled trail. Int J Geriatr Psychiatry. 2019;34(5):674–82. https://doi.org/10.1002/gps.5066.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Jabeen I, Qureshi A, Waqar MA. Development of palliative care services at a tertiary care teaching hospital in pakistan: retrospective analysis of existing palliative care program. J Pain Symptom Manage. 2022;64(2):178–85. https://doi.org/10.1016/j.jpainsymman.2022.03.018.

    Article  PubMed  Google Scholar 

  39. Huang Y-H, Ma L, Sabljak LA, Puhala ZA. Development of sustainable community paramedicine programmes: a case study in Pennsylvania. Emerg Med J. 2018;35(6):372–8. https://doi.org/10.1136/emermed-2017-207211.

    Article  PubMed  Google Scholar 

  40. Tejedor-Sojo J, Chan KN, Bailey M, et al. Improving bronchiolitis care in outpatient settings across a health care system. Pediatr Emerg Care. 2019;35(11):791–8. https://doi.org/10.1097/PEC.0000000000001966.

    Article  PubMed  Google Scholar 

  41. Tjia J, Field T, Mazor K, et al. Dissemination of evidence-based antipsychotic prescribing guidelines to nursing homes: a cluster randomized trial. J Am Geriatr Soc. 2015;63(7):1289–98.

    Article  PubMed  Google Scholar 

  42. Agarwal HS, Saville BR, Slayton JM, et al. Standardized postoperative handover process improves outcomes in the intensive care unit: a model for operational sustainability and improved team performance. Crit Care Med. 2012;40(7):2109–15. https://doi.org/10.1097/CCM.0b013e3182514bab.

    Article  PubMed  Google Scholar 

  43. Agarwal S, Gallo JJ, Parashar A, et al. Impact of lean six sigma process improvement methodology on cardiac catheterization laboratory efficiency. Cardiovasc Revasc Med. 2016;17(2):95–101. https://doi.org/10.1016/j.carrev.2015.12.011.

    Article  PubMed  Google Scholar 

  44. Benn J, Burnett S, Parand A, Pinto A, Vincent C. Factors predicting change in hospital safety climate and capability in a multi-site patient safety collaborative: a longitudinal survey study. BMJ Qual Saf. 2012;21(7):559–68. https://doi.org/10.1136/bmjqs-2011-000286.

    Article  PubMed  Google Scholar 

  45. Palinkas LA, Ell K, Hansen M, Cabassa L, Wells A. Sustainability of collaborative care interventions in primary care settings. J Soc Work. 2010;11(1):99–117. https://doi.org/10.1177/1468017310381310.

    Article  Google Scholar 

  46. Rubin FH, Neal K, Fenlon K, Hassan S, Inouye SK. Sustainability and scalability of the hospital elder life program at a community hospital. J Am Geriatr Soc. 2011;59(2):359–65. https://doi.org/10.1111/j.1532-5415.2010.03243.x.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Verma JY, Amar C, Sibbald S, Rocker GM. Improving care for advanced COPD through practice change: experiences of participation in a Canadian spread collaborative. Chron Respir Dis. 2018;15(1):5–18. https://doi.org/10.1177/1479972317712720.

    Article  PubMed  Google Scholar 

  48. Fox A, Gardner G, Osborne S. Nursing service innovation: a case study examining emergency nurse practitioner service sustainability. J Adv Nurs. 2018;74(2):454–64. https://doi.org/10.1111/jan.13454.

    Article  PubMed  Google Scholar 

  49. Morgan D, Kosteniuk J, O’Connell ME, et al. Barriers and facilitators to development and implementation of a rural primary health care intervention for dementia: a process evaluation. BMC Health Serv Res. 2019;19(1):1–18. https://doi.org/10.1186/s12913-019-4548-5.

    Article  CAS  Google Scholar 

  50. Stirman SW, Matza A, Gamarra J, et al. System-Level Influences on the sustainability of a cognitive therapy program in a community behavioral health network. Psychiatr Serv. 2015;66(7):734–42. https://doi.org/10.1176/appi.ps.201400147.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Gramlich L, Nelson G, Nelson A, Lagendyk L, Gilmour LE, Wasylak T. Moving enhanced recovery after surgery from implementation to sustainability across a health system: a qualitative assessment of leadership perspectives. BMC Health Serv Res. 2020;20(1):1–11. https://doi.org/10.1186/s12913-020-05227-0.

    Article  Google Scholar 

  52. Thota R, Gill DM, Brant JL, Yeatman TJ, Haslem DS. Telehealth is a sustainable population health strategy to lower costs and increase quality of health care in rural Utah. JCO Oncology Practice. 2020;16(7):e557–62. https://doi.org/10.1200/Jop.19.00764.

    Article  PubMed  Google Scholar 

  53. Waiswa P, Wanduru P, Okuga M, et al. Institutionalizing a regional model for improving quality of newborn care at birth across hospitals in eastern Uganda: a 4-year story. Glob Health Sci Pract. 2021;9(2):365–78. https://doi.org/10.9745/GHSP-D-20-00156.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Zafar MA, Nguyen B, Gentene A, et al. Pragmatic challenge of sustainability: long-term adherence to COPD care bundle maintains lower readmission rate. Jt Comm J Qual Patient Saf. 2019;45(9):639–45. https://doi.org/10.1016/j.jcjq.2019.05.011.

    Article  PubMed  Google Scholar 

  55. Ament SMC, Gillissen F, Moser A, et al. Factors associated with sustainability of 2 quality improvement programs after achieving early implementation success. A qualitative case study. J Eval Clin Pract. 2017;23(6):1135–43. https://doi.org/10.1111/jep.12735.

    Article  PubMed  Google Scholar 

  56. Blanchet K, James P. Can international health programmes be sustained after the end of international funding: the case of eye care interventions in Ghana. BMC Health Serv Res. 2014;14(1):77. https://doi.org/10.1186/1472-6963-14-77.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Zakumumpa H, Bennett S, Ssengooba F. Modifications to ART service delivery models by health facilities in Uganda in promotion of intervention sustainability: a mixed methods study. Implement Sci. 2017;12(1):1–14. https://doi.org/10.1186/s13012-017-0578-8.

    Article  Google Scholar 

  58. Urquhart R, Kendell C, Cornelissen E, et al. Identifying factors influencing sustainability of innovations in cancer survivorship care: a qualitative study. BMJ Open. 2021;11(2):e042503. https://doi.org/10.1136/bmjopen-2020-042503.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Kastner M, Sayal R, Oliver D, Straus SE, Dolovich L. Sustainability and scalability of a volunteer-based primary care intervention (Health TAPESTRY): a mixed-methods analysis. BMC Health Serv Res. 2017;17(1):514. https://doi.org/10.1186/s12913-017-2468-9.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Casati B, Bjugn R. Structured electronic template for histopathology reporting on colorectal carcinoma resections: five-year follow-up shows sustainable long-term quality improvement. Arch Pathol Lab Med. 2012;136(6):652–6. https://doi.org/10.5858/arpa.2011-0370-OA.

    Article  PubMed  Google Scholar 

  61. Dharmayat KI. Sustainability of ‘mHealth’interventions in sub-Saharan Africa: a stakeholder analysis of an electronic community case management project in Malawi. Malawi Med J. 2019;31(3):177–83. https://doi.org/10.4314/mmj.v31i3.3.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Kenu A, Kenu E, Bandoh DA, Aikins M. Factors that promote and sustain the use of traditional, complementary and integrative medicine services at LEKMA hospital, Ghana, 2017: an observational study. BMC Complement Med Ther. 2021;21(1):1–10. https://doi.org/10.1186/s12906-020-03185-y.

    Article  Google Scholar 

  63. Ali SM, Rehal S. Public health practitioners’ perspective on the sustainability of the tuberculosis control programme at primary health care level in Pakistan. East Mediterr Health J. 2021;27(9):899–905. https://doi.org/10.26719/emhj.21.044.

    Article  PubMed  Google Scholar 

  64. De Neve J-W, Garrison-Desany H, Andrews KG, et al. Harmonization of community health worker programs for HIV: A four-country qualitative study in Southern Africa. PLoS Med. 2017;14(8):e1002374–e1002374. https://doi.org/10.1371/journal.pmed.1002374.

    Article  PubMed  PubMed Central  Google Scholar 

  65. MacLean S, Berends L, Mugavin J. Factors contributing to the sustainability of alcohol and other drug interventions in Australian community health settings. Aust J Prim Health. 2013;19(1):53–8. https://doi.org/10.1071/py11136.

    Article  PubMed  Google Scholar 

  66. Song Y, MacEachern L, Doupe MB, et al. Influences of post-implementation factors on the sustainability, sustainment, and intra-organizational spread of complex interventions. BMC Health Serv Res. 2022;22(1):666. https://doi.org/10.1186/s12913-022-08026-x.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Chiliza J, Laing R, Feeley FG III, Borba CP. Program sustainability post PEPFAR direct service support in the Western Cape, South Africa. PLoS One. 2021;16(5):e0251230. https://doi.org/10.1371/journal.pone.0251230.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Popowich AD, Mushquash AR, Pearson E, Schmidt F, Mushquash CJ. Barriers and facilitators affecting the sustainability of dialectical behaviour therapy programmes: a qualitative study of clinician perspectives. Couns Psychother Res. 2020;20(1):68–80. https://doi.org/10.1002/capr.12250.

    Article  Google Scholar 

  69. Mahomed OH, Asmall S, Voce A. Sustainability of the integrated chronic disease management model at primary care clinics in South Africa. Afr J Prim Health Care Fam Med. 2016;8(1):e1–7. https://doi.org/10.4102/phcfm.v8i1.1248.

    Article  PubMed  Google Scholar 

  70. Butow P, Williams D, Thewes B, et al. A psychological intervention (ConquerFear) for treating fear of cancer recurrence: views of study therapists regarding sustainability. Psychooncology. 2019;28(3):533–9. https://doi.org/10.1002/pon.4971.

    Article  PubMed  Google Scholar 

  71. Lean M, Leavey G, Killaspy H, et al. Barriers to the sustainability of an intervention designed to improve patient engagement within NHS mental health rehabilitation units: a qualitative study nested within a randomised controlled trial. BMC Psychiatry. 2015;15(1):209. https://doi.org/10.1186/s12888-015-0592-9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Peterson AE, Bond GR, Drake RE, McHugo GJ, Jones AM, Williams JR. Predicting the long-term sustainability of evidence-based practices in mental health care: an 8-year longitudinal analysis. J Behav Health Serv Res. 2014;41(3):337–46. https://doi.org/10.1007/s11414-013-9347-x.

    Article  PubMed  Google Scholar 

  73. Pomey M-P, Clavel N, Amar C, et al. Wait time management strategies for total joint replacement surgery: sustainability and unintended consequences. BMC Health Serv Res. 2017;17(1):629–629. https://doi.org/10.1186/s12913-017-2568-6.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Seppey M, Ridde V, Toure L, Coulibaly A. Donor-funded project’s sustainability assessment: a qualitative case study of a results-based financing pilot in Koulikoro region, Mali. Glob Health. 2017;13(1):86. https://doi.org/10.1186/s12992-017-0307-8.

    Article  Google Scholar 

  75. Zakumumpa H, Bennett S, Ssengooba F. Accounting for variations in ART program sustainability outcomes in health facilities in Uganda: a comparative case study analysis. BMC Health Serv Res. 2016;16(1):1–13. https://doi.org/10.1186/s12913-016-1833-4.

    Article  Google Scholar 

  76. Oliveira SRA, Medina MG, Figueiro AC, Potvin L. Strategic factors for the sustainability of a health intervention at municipal level of Brazil. Cad Saude Publica. 2017;33(7):e00063516. https://doi.org/10.1590/0102-311x00063516.

    Article  PubMed  Google Scholar 

  77. Ament SM, Gillissen F, Maessen JM, et al. Sustainability of short stay after breast cancer surgery in early adopter hospitals. Breast. 2014;23(4):429–34. https://doi.org/10.1016/j.breast.2014.02.010.

    Article  CAS  PubMed  Google Scholar 

  78. Magadzire BP, Marchal B, Ward K. Improving access to medicines through centralised dispensing in the public sector: a case study of the Chronic Dispensing Unit in the Western Cape Province, South Africa. BMC Health Serv Res. 2015;15(1):513. https://doi.org/10.1186/s12913-015-1164-x.

    Article  PubMed  PubMed Central  Google Scholar 

  79. Kuipers P, Humphreys JS, Wakerman J, Wells R, Jones J, Entwistle P. Collaborative review of pilot projects to inform policy: a methodological remedy for pilotitis? Aust New Zealand Health Policy. 2008;5:1–4. https://doi.org/10.1186/1743-8462-5-17.

    Article  Google Scholar 

  80. Cowie J, Nicoll A, Dimova ED, Campbell P, Duncan EA. The barriers and facilitators influencing the sustainability of hospital-based interventions: a systematic review. BMC Health Serv Res. 2020;20(1):1–27. https://doi.org/10.1186/s12913-020-05434-9.

    Article  Google Scholar 

  81. Penno LN, Davies B, Graham ID, et al. Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: a systematic review and theory analysis of selected sustainability frameworks. Implement Sci. 2019;14(1):1–16. https://doi.org/10.1186/s13012-019-0952-9.

    Article  Google Scholar 

  82. Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018;13(1):1–17. https://doi.org/10.1186/s13012-017-0707-4.

    Article  Google Scholar 

  83. Nahimana E, Magge H, Bizimana F, et al. Sustainability assessment of a district-wide quality improvement on newborn care program in rural rwanda: a mixed-method study. Ann Glob Health. 2021;87(1):40. https://doi.org/10.5334/aogh.3205.

    Article  PubMed  PubMed Central  Google Scholar 

  84. Grimshaw JM, Patey AM, Kirkham KR, et al. De-implementing wisely: developing the evidence base to reduce low-value care. BMJ Qual Saf. 2020;29(5):409–17. https://doi.org/10.1136/bmjqs-2019-010060.

    Article  PubMed  PubMed Central  Google Scholar 

  85. Norton WE, Chambers DA. Unpacking the complexities of de-implementing inappropriate health interventions. Implement Sci. 2020;15(1):2. https://doi.org/10.1186/s13012-019-0960-9.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Sridharan L, Greenland P. Editorial Policies and publication bias: The importance of negative studies. Arch Intern Med. 2009;169(11):1022–3. https://doi.org/10.1001/archinternmed.2009.100.

    Article  PubMed  Google Scholar 

  87. Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. 2001;44(2):119–27. https://doi.org/10.1016/s0738-3991(00)00186-5.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Ms. Mary Simons and Mr. Jeremy Cullis, clinical librarians at Macquarie University, who consulted with the research team to develop the search strategy, as well as Ms. Johanna Holt, the AIHI Coordinator, for helping to refine the study selection criteria. We would also like to thank Dr. Margie Campbell, a Research Fellow at the Centre for Health Economics Research and Evaluation, University of Technology Sydney, for identifying additional studies to include in the review via snowballing.

Funding

This work was supported by the NHMRC Partnership Centre for Health System Sustainability (Grant ID 9100002) and NHMRC Investigator Grant APP1176620.

Author information

Authors and Affiliations

  1. Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, North Ryde, Australia, Level 6, 75 Talavera Rd, NSW, 2109

    Yvonne Zurynski, Kristiana Ludlow, Luke Testa, Hanna Augustsson, Jessica Herkes-Deane, Karen Hutchinson, Gina Lamprell, Elise McPherson, Ann Carrigan, Louise A. Ellis, Putu Novi Arfirsta Dharmayani, Carolynn L. Smith, Lieke Richardson, Genevieve Dammery, Nehal Singh & Jeffrey Braithwaite

  2. NHMRC Partnership Centre for Health System Sustainability, Australian Institute of Health Innovation, Macquarie University, North Ryde, Australia

    Yvonne Zurynski, Louise A. Ellis, Putu Novi Arfirsta Dharmayani, Carolynn L. Smith, Genevieve Dammery & Jeffrey Braithwaite

  3. Centre for Health Services Research, The University of Queensland, Brisbane, Australia

    Kristiana Ludlow

  4. Procome Research Group, Medical Management Centre, Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Solna, Sweden

    Hanna Augustsson

Authors
  1. Yvonne Zurynski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristiana Ludlow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luke Testa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hanna Augustsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jessica Herkes-Deane
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Karen Hutchinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gina Lamprell
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Elise McPherson
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ann Carrigan
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Louise A. Ellis
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Putu Novi Arfirsta Dharmayani
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Carolynn L. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Lieke Richardson
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Genevieve Dammery
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Nehal Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Jeffrey Braithwaite
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JB conceptualised the study and with YZ led the team’s work. KL, LT, JHD, and EM developed the search strategy. KL, LT, JHD, HA, GL, EM, KH, AC, LAE, PNAD, CLS, LR, GD, and NS conducted the abstract review, full-text review and data extraction, with JB and YZ acting as arbitrators. The quality assessment of included articles was conducted by KL, LT, JHD, HA, GL, KH, AC, LAE, PNAD, CLS, LR, GD, and NS. KL and LT undertook the synthesis of data. KL, LT, YZ, KH, and PNAD wrote the draft manuscript, with all authors providing feedback and approving the final version.

Corresponding author

Correspondence to Yvonne Zurynski.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

PRISMA Checklist.

Additional file 2: Table S2.

Search strategies used for systematic integrative review.

Additional file 3: Table S3.

Quality assessment using Hawker’s Quality Assessment Tools. 

Additional file 4: Table S4.

Barriers and facilitators using the ISFR2.

Additional file 5: Table S5.

Long-term sustained programs (n= 29). 

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zurynski, Y., Ludlow, K., Testa, L. et al. Built to last? Barriers and facilitators of healthcare program sustainability: a systematic integrative review. Implementation Sci 18, 62 (2023). https://doi.org/10.1186/s13012-023-01315-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-023-01315-x

Keywords

Implementation Science

ISSN: 1748-5908