Toward systematic reviews to understand the determinants of wait time management success to help decision-makers and managers better manage wait times
Implementation Science volume 8, Article number: 61 (2013)
Long waits for core specialized services have consistently been identified as a key barrier to access. Governments and organizations at all levels have responded with strategies for better wait list management. While these initiatives are promising, insufficient attention has been paid to factors influencing the implementation and sustainability of wait time management strategies (WTMS) implemented at the organizational level.
A systematic review was conducted using the main electronic databases, such as CINAHL, MEDLINE, and Cochrane Database of Systematic Reviews, to identify articles published between 1990 and 2011 on WTMS for scheduled care implemented at the organizational level or higher and on frameworks for analyzing factors influencing their success. Data was extracted on governance, culture, resources, and tools. We organized a workshop with Canadian healthcare policy-makers and managers to compare our initial findings with their experience.
Our systematic review included 47 articles: 36 related to implementation and 11 to sustainability. From these, we identified a variety of WTMS initiated at the organizational level or higher, and within these, certain factors that were specific to either implementation or sustainability and others common to both. The main common factors influencing success at the contextual level were stakeholder engagement and strong funding, and at the organizational level, physician involvement, human resources capacity, and information management systems. Specific factors for successful implementation at the contextual level were consultation with front-line actors and common standards and guidelines, and at the organizational level, financial incentives and dedicated staffing. For sustainability, we found no new factors. The workshop participants identified the same major factors as found in the articles and added others, such as information sharing between physicians and managers.
Factors related to implementation were studied more than those related to sustainability. However, this finding was useful in developing a tool to help managers at the local level monitor the implementation of WTMS and highlighted the need for more research on specific factors for sustainability and to assess the unintended consequences of introducing WTMS in healthcare organizations.
For the past two decades, access to healthcare services has been a critical issue in most Organisation for Economic Co-operation and Development (OECD) countries . Long waits for core specialized healthcare services have consistently been identified as a key barrier to access to care [2, 3], and governments and organizations have responded with a range of strategies to better manage wait times. For example, in September 2004, Canada’s First Ministers committed $5.5 billion to timely access in five healthcare areas over a 10-year period . In June 2005, the Supreme Court of Canada (in the Chaoulli Decision) struck down Quebec’s ban on private insurance for Medicare-covered services in a bid to reduce wait times in that province. In April 2007, Canada’s federal government announced it would provide $612 billion to provinces that would commit to respecting maximum wait times for at least one medical procedure in their jurisdiction . From these initiatives, it is clear that over the past decade Canadian decision-makers have consistently seen the centralization of programs at the federal and provincial levels as the means to solve problems regarding wait lists and wait times . While these initiatives are promising [7–9], we believe insufficient attention has been paid to how healthcare organizations themselves have implemented strategies to reduce wait lists and wait times . We therefore conducted a systematic review, supplemented by a workshop with experts, to better understand the key organizational and contextual level factors that influence the success or failure of wait time management strategies (WTMS) undertaken by organizations, not only in their implementation but also in their sustainability over the longer term. Given that WTMS implementation and sustainability have not been defined in the literature as such, we borrowed from the change theory literature on the life cycle of typical change initiatives , where the implementation phase is defined as the period when a new intervention is introduced and the sustainability phase as the period during which it is consolidated. In the systematic review and expert workshop, we focused on factors that were considered to influence the success of WTMS implementation and sustainability; we did not focus directly on the impact of these factors on reducing wait times. Our systematic review explored five questions:
Are there existing frameworks to identify potential explanatory factors for WTMS success at the organizational level?
What contextual and organizational factors can influence the success of WTMS implementation at the organizational level for scheduled care?
What contextual and organizational factors can influence the success of WTMS sustainability at the organizational level for scheduled care?
Are those factors valid for the Canadian context?
How can identifying such factors contribute to developing a tool to support managers in implementing and sustaining WTMS at the organizational level?
The systematic review was limited to published articles. Six electronic medical databases were searched: CINAHL, the Cochrane Central Register of Controlled Trials, the Cochrane Database of Systematic Reviews, Embase, Healthstar, and MEDLINE (including MEDLINE In-Process). We consulted 19 electronic non-medical databases, which included general Canadian databases (Canadian Business & Current Affairs; Canadian Newsstand; Canadian Periodical Index; Canadian Research Index; Repères), math and engineering databases (Compendex; IEEE Xplore; Inspec; MathSciNet), economics databases (EconLit & ERIC [CBSCO Host]), and sociological (Communications Abstracts; ERIC; PAIS; Social Sciences Abstracts; Sociological Abstracts) and multidisciplinary (ABI Inform; ProQuest Dissertations & Theses; Scopus) databases. We used the search terms ‘waiting lists’ [or] ‘wait time’ [or] ‘queues’, combined using the Boolean operator and with terms reflecting management issues, such as ‘policies’, ‘information systems’, ‘budgets’, ‘health priorities’, ‘patient referral’, ‘health care delivery’, and ‘personnel management’. Additional search strategies to complement the search outlined above were developed for both medical and non-medical databases to ensure that we captured all relevant articles on WTMS. One example of the search strategies used is given in Table 1. This search strategy was developed and used for the Cochrane Central Registry of Controlled Trials, Healthstar, and MEDLINE medical databases.
The systematic review was carried out over two periods of time. The first search, conducted in 2005, covered articles published between 1990 and 2005, and the second search, conducted in 2011, covered those published between 2006 and 2011. We conducted a workshop in March 2009 with healthcare experts to validate the first findings of the systematic literature review and to ensure they were robust. We conducted our review in two steps because most of the articles retrieved in the first period were related to WTMS implementation, but workshop participants felt it would be important also to include articles related to WTMS sustainability. Consequently, we extended our research until 2011 to capture such articles. We also explored together how the factors identified could be developed into best practices to create a tool for managers in healthcare organizations responsible for WTMS. Participants were selected because they had responsibility for healthcare provision, health system leadership, and wait time management, were in a position to provide experience-based critiques, and could potentially apply the research findings in their context.
The data from the systematic review are presented here in narrative form because of the heterogeneity of study designs in the articles examined for our systematic review. The articles reviewed used a wide variety of study designs and approaches, including case studies, pre-post comparisons studies, prospective studies, simulation/modeling studies, and time series studies. We combined the findings of the two periods, specifically identifying factors that emerged in relation to WTMS implementation or sustainability. We adapted Parson’s social system action theory to classify the factors . This framework, which we used in previous work to identify contextual and organizational factors , was also useful to identify a priori coding categories , which is helpful when conducting reviews of qualitative studies. This framework (Figure 1) is composed of different factors that can be present at the contextual and organizational levels: ‘governance and leadership’, defined as ‘the conduct of collective action from a position of authority’ ; ‘culture’, which consists of ‘underlying beliefs, values, norms and behaviours’ [13, 16]; ‘resources’, which may be human, financial, infrastructural, or informational; and ‘tools’, referring to instruments or procedures. The organizational level corresponds to the specific context of WTMS initiatives within healthcare organizations, whereas the contextual level refers to the broader context of WTMS initiatives, such as policies, laws, or regulations from provincial, regional, or other health authorities.
We applied the following four inclusion criteria:We had no inclusion criteria related to study design; all study designs were included.
Articles referring to wait time management for scheduled care: diagnostic imaging and elective surgery were selected for additional searches to complement the main search and to ensure all relevant articles were captured. More specifically, we selected articles that: described a model or framework of factors influencing WTMS success or failure at the organizational level; referred to an organizational initiative that specifically addressed wait time or wait lists; referred to higher level (national or provincial) strategies or policies that addressed WTMS;
Articles published in peer-reviewed journals;
Articles published in French or in English;
Articles published between 1990 and 2011.
Articles that focused on transplantation, emergency care, and psychiatric care were excluded because their wait time management differs significantly from that for scheduled care. We also excluded opinion papers and papers with no mention of the study design.
Data extraction was qualitative and was performed by five members of the research team using SRS (TrialStat). The software monitored the Kappa data extraction from the five extractors, and discussions among extractors were regularly organized if necessary to reduce any inconsistency between reviewers. In addition, the principal investigator reviewed all abstracted data. The extraction form was a custom-designed template that had been tested on a sample of articles. The following main categories of data were extracted: jurisdiction of WTMS, clinical area of WTMS, definition of wait time or wait list, description of WTMS, article objectives, study design, theoretical framework used, and factors influencing WTMS (categories of factors and their positive or negative influence on WTMS). Because the links between the factors and the success of the WTMS strategies were not systematically analyzed in the articles or quantified in any heterogeneous way, we decided not to include such analyses. Whenever possible, reviewers used quotes to describe wait time initiatives, and for all factors identified in the articles, added page and paragraph references.
A search protocol for the systematic review was approved by the Canadian Institutes of Health Research and is available through the principal investigator.
The search yielded 12,779 references; after elimination of duplicates, 6,176 remained. Four levels of screening were conducted in duplicate by two reviewers using the web-based systematic reviewing platform SRS (TrialSat). The first level excluded all articles obviously irrelevant based on abstracts. The second level excluded articles in non-peer reviewed journals and those written in languages other than English and French. With 619 references remaining, the third level consisted of reviewing the titles and abstracts with the same criteria used in the second level, this time excluding articles not directly linked to wait time management. For the remaining 244 full articles, in the fourth level of screening, the research team applied the specific inclusion criteria for articles described above. In the end, 47 articles were retained and abstracted (Figure 2).
Types of research designs and countries of provenance
Of these 47 articles, the vast majority (n = 45) were empirical studies, and the remaining two used a theoretical approach. Approximately one-third of the empirical studies were single case studies (n = 17), some were pre-/post-strategy comparisons (n = 9), others combined a case study with a survey (n = 7), some were multiple case studies (n = 6), a few were simulation/modelling (n = 3), and the rest were time series (n = 3). Two-thirds of these 47 articles were from Canada (n = 15) and the United Kingdom (n = 15). The rest were from Australia (n = 4), Sweden (n = 4), the Scandinavian countries (n = 4), New Zealand (n = 3), and the United States (n = 2). One explanation for the low number of articles from the US might be that the country faced fewer challenges regarding wait times than did other OECD countries.
Characteristics of wait time management strategies
From the systematic review, we identified different types of WTMS in terms of the levels at which they were launched, i.e., at the local level in a healthcare organization or at a broader system level (national, provincial, or regional). At the local level, strategies included work reorganizations (n = 8), increases in capacity (n = 2), implementation of pre-operative clinics (n = 2), and development of data collection software, including simulations of wait lists (n = 3) and booking systems (n = 1). At the national level, strategies included booking systems (n = 6), maximum wait time guarantees (n = 4), and development and use of prioritization tools (n = 11).
Table 2 summarizes the different WTMS identified in the articles. These covered a broad range of clinical areas, including orthopaedics (n = 7), eye care (n = 7), and cardiac care (n = 6), and many covered general surgery (n = 23) or spanned multiple specialties (n = 4). Our search uncovered no studies in medical imaging, even though this area was specifically included in our research question, as wait times in this area have been identified as a problem.
Frameworks on factors that can influence WTMS
Among all articles abstracted, only two, by the same authors, offered a framework for studying factors that influenced the implementation of WTMS, in this case the implementation of a booking system [17, 18]. Published in 2003, the study collected and analyzed quantitative and qualitative data from 20 pilot organizations in the United Kingdom to identify essential conditions for successful implementation of quality improvement initiatives in healthcare. The framework grouped the factors under six categories. Two of the categories, ‘national context’ and ‘local context’, acknowledged the need to look at the system on more than one level. This is also acknowledged in our model/framework. The authors also identified the categories ‘culture’ and ‘capacity’, which we have named ‘culture’ and ‘resources’. Another of their categories is ‘roles of physicians’, which, as in our study, emphasizes doctors’ key role. Lastly, their category ‘mechanisms of change’ covers a collection of factors, all of which have been classified in various dimensions in our own model.
Empirical factors influencing WTMS
In the following sections, we begin by considering the factors associated with implementation at the local and contextual levels before presenting the factors associated with WTMS sustainability. A synthesis of all the factors is presented in Table 3.
Implementation phase of WTMS: local-level factors
Leadership, related to governance, was the most frequently cited factor, found in 11 articles [17–27]. The other recurrent factor, seen in five articles [19, 20, 24, 28, 29], was the establishment of a dedicated decision-making and management structure to design and implement WTMS. Finally, accountability was also an important factor in four articles, two of which identified clinician accountability as key in implementing a successful strategy [22, 28], while three stressed the accountability of hospital boards and management [22, 23, 25].
Definitely the most recurrent cultural factors in the articles were related to physicians’ involvement and attitudes toward WTMS. Of the 13 articles that mentioned this factor [17, 18, 26, 30–39], seven stressed the fact that it can be a major barrier for WTMS implementation [18, 26, 30, 32, 33, 36, 38]. Lack of trust between managers and clinicians was cited as another important factor [17, 20, 22, 25, 40] that can have a negative impact. On a more positive note, some authors pointed out that WTMS implementation is also facilitated by an organizational culture of quality improvement [24–28].
The most common local-level resource factors mentioned in the literature were those having to do with flexibility and adequacy. Insufficient infrastructure resources were mentioned as a limiting factor in several articles [17, 18, 20, 31, 37, 40–44]. Capacity constraints, in relation to both operating rooms and post-surgery beds, hindered WTMS implementation . One study reported that surgical capacity had to be increased to meet wait time guarantees . Concerning human resources, many articles underscored the importance of having dedicated staffing. More specifically, several articles identified appropriate levels of dedicated staffing as a success factor for WTMS [17, 19, 20, 22, 23, 31, 41, 42],[44–51], and some reported on the positive impact of using non-medical staff (physician assistants, nurse practitioners, anesthesia assistants) on reducing wait times [24–26]. Concerning financial resources, some articles cited incentives at the individual or the unit level as contributing to successful outcomes. In Ontario, offering financial incentives for physicians to use tools specially developed for assigning priorities to surgical patients seemed to have a positive impact on reducing wait times [23, 39]. Disincentives were also powerful, as in Sweden, where hospitals that did not meet wait time guarantees ran the risk of being forced to send patients elsewhere at the hospitals’ expense [33, 52].
The most commonly cited tool was information management systems designed to meet the high information demands in WTMS initiatives. Examples included databases for recording information [18, 19, 25, 30, 31, 39, 41, 44],[53, 54] and scheduling software . Interestingly, it was noted that overly complex systems constituted barriers to WTMS implementation . Thus, simple, user-friendly, and effective solutions for information presentation were lauded, as was system flexibility in the case of information systems implemented across a region or country [31, 55]. It was also important that those concerned be able to access relevant data easily  and that the data be ‘clean’ .
Training and support were another important tool cited. The increased use of quantitative information made it important for staff to be trained to analyze basic statistics and time series data, and to acquire a basic understanding of spreadsheets [31, 55]. Some articles pointed out the key role of peri-operative coaching teams in helping hospitals improve both clinical and organizational processes/practices for wait time management [24–27, 39].
Implementation phase of WTMS: contextual-level factors
The first contextual-level governance factor highlighted by the systematic review was the need for high-level (i.e., central) coordinating, reporting, and monitoring structures [17, 21, 23–27, 32, 35, 37, 44, 54]. Several articles mentioned the need for such structures to provide guidelines and advice. Structures such as the UK’s National Patients’ Access Team  and Healthcare Commission  or Ontario’s Critical Care Expert Panel  positively affected outcomes at the organizational level. For example, the Critical Care Expert Panel provided advice on ‘methods to support a well-functioning critical care system that will minimize surgical delays’ . Reporting was also cited as a success factor [25, 32]; this refers to the healthcare organization’s obligation to monitor and present its data to the Ministry of Health or to an independent structure in charge of monitoring wait time. In particular, one article pointed out that, in Ontario, reports issued by the Ministry of Health on WTMS achievements  offer a certain degree of accountability to different bodies representing patients and more generally to the population. Still, those authors suggest that communication efforts should go further, providing more targeted information to educate the various stakeholders: boards, administrators, clinicians, and hospital staff .
Another positive contextual-level government factor identified in the literature was the involvement of stakeholders, including professional associations [22, 25, 26], partners such as national or provincial health ministries [22, 24–27, 54], patients, and other stakeholders in the health network [22, 56]. Methods for involving stakeholders, which included soliciting them for information on data , advice on existing processes, ideas for improvements [25, 26, 32], and recommendations [24, 26], as well as ensuring systems for reporting , were seen as helpful for WTMS implementation.
Another factor was related to accountability agreements. As highlighted in several articles [22–25, 44, 54], accountability agreements signed between government and hospital boards are a key element of Ontario’s wait time strategy. These agreements identify the hospital actors and make them accountable for equitable access to services in their organization.
One cultural factor seen at the contextual level was consultation with front-line actors. This factor surfaced as having contributed to the successful implementation of WTMS initiatives [17, 22–27, 32, 34, 36, 37]. An example of this was consultation with expert panels as part of Ontario’s Wait Time Strategy. Indeed, a key element of the strategy involved seeking front-line actors’ (clinicians, administrators, and researchers) advice on improving access [22–27]. These consultations helped to involve experts in the strategy and to enlist their support and involvement.
Public awareness was also identified as a success factor in WTMS implementation. Several articles described government efforts to make available to patients clear information about wait time strategies [22, 23, 25, 39, 54]. These initiatives empowered patients by sharing knowledge about wait times .
Developing a performance culture was also an important factor identified in the literature [22–25, 27, 54]. Common levers used by the Ministry of Health to create a performance culture included introducing financial incentives for physicians, monitoring data, and conducting assessments, audits, and hospital surveys, all with the aim of improving practices/process efficiency and effectiveness.
Funding was by far the most recurrent positive contextual resource factor [17, 18, 22–27, 30, 32, 34, 35, 37],[40, 45, 48, 49, 52–54]. Higher level funding was provided for specific initiatives, such as strategies to address backlog [25, 26, 30, 32, 40, 49], initiatives to make practices more efficient and effective [23, 25], recruitment of medical staff , and specific purchases, such as information systems equipment for data collection [22, 23, 25] and new or updated medical equipment [22, 23, 25, 54].
Financial incentives based on performance were another contextual resource factor described in some articles [22, 23, 25, 26]. For example, in Ontario, hospitals were expected to meet a series of conditions to obtain funding for wait time cases.
At the contextual level, tools included instruments or procedures that affected more than one organization, such as the development and implementation of standards and guidelines [22–27, 33–35, 40, 44, 52, 56]. Some procedures involved standardizing equipment across hospitals (e.g. MRI, CT scanners) [25, 26]. Others were related to standardization of practices [23, 25, 26]. Standardized wait time indicators were also identified as a contextual level tool [23–27]. Finally, collection and standardization of data was the most recurrent factor cited [22, 23, 25, 26, 33–35]. Implementation of a central registry was also seen as a tool to significantly reduce wait times [33–35, 37].
Training programs, whether in specific professional skills or management skills, were also mentioned in a few articles as a useful tool for implementing WTMS [17, 18]. Finally, the development of communication tools such as websites was also cited in some articles [22, 23, 25]. For example, in Ontario, the implementation of a website made it possible to provide the population, first, with general information on the province’s wait time strategy and, later, on specific medical specialties as part of the strategy.
Sustainability phase of WTMS: local-level factors
Accountability was the factor most frequently cited as a key component of WTMS sustainability [22, 57, 58]. Those working at the local level pointed out that accountabilities have to be explicit and must apply to all local stakeholders, including the general public and patients, healthcare providers, administrators, managers, and government. One article emphasized that each of these groups has a critical role to play in furthering the strategy’s momentum and developing a sustained culture for the future .
Leadership was also cited in some articles as an important factor in sustaining a WTMS [22, 24, 57], with the point being made that leadership must be continuously provided by all local champions, such as boards of hospitals and local health organizations, hospital administrations, and medical management.
Physician involvement was cited as a cultural factor influencing WTMS sustainability [22, 54]. According to these articles, it is important that physicians maintain momentum in meeting wait time accountability targets. One article described an open management style that supported physicians and engaged them in activities to improve quality and access . Finally, the need for cooperation between physicians and managers was also mentioned .
Certain human resources issues that pose significant challenges to improving system capacity were identified in one article . The authors stressed that, to ensure WTMS sustainability, hospitals must develop effective recruitment and retention strategies in the relevant professions, where there have often been long-standing issues regarding supply. Wait time issues cannot be solved by improving clinical efficiency alone; any solution must also incorporate new ways of working. One article, for instance, described an innovative initiative to expand the role of physiotherapists in joint replacement programs .
Finally, the need to provide patients with alternative treatment options was presented as a key dimension of sustainable WTMS .
At the organizational level, the tool most frequently cited was the use of a performance or information management system [22, 24, 58–61]. These articles pointed out hospital managers’ need for appropriate information to assess their wait time performance, as well as tools for managing capacity, booking patients, and mapping peri-operative processes in order to predict maximum patient flow.
Finally, training and support of human resources was another factor presented in the literature . In particular, this article mentioned the positive role of training staff in the use of common software and in data management.
Sustainability phase of WTMS: contextual-level factors
Some articles considered an informed public to be an important factor in sustainability [22, 27]. Indeed, these articles highlighted the media’s important role in educating the public by communicating the content of public wait time websites, informing the public on how to use wait time information to discuss treatment options with care providers, and writing about the impact that consumer choices, behaviors, and expectations have on wait time.
Financial incentives were identified as a key factor in sustaining WTMS [22, 25]. These articles highlighted the need to align hospital and physician funding mechanisms, since their different methods of payment represented a barrier to the sustainability of such strategies. Indeed, fee-for-service remuneration motivates physicians to treat as many patients as possible, whereas hospitals receiving global budgets are motivated to treat as few patients as possible. These articles outlined the need to align physician remuneration with government objectives to sustain wait time strategies. Performance-based payment was seen as an incentive to perform more procedures as efficiently and effectively as possible. Finally, the articles highlighted the need for appropriate funding for these measures [27, 62].
Electronic information systems were cited as a key to sustaining wait time initiatives [22–24, 61, 63]. Indeed, the availability of wait time information was seen as enabling decision-makers to assess the impact of investments in WTMS on access and on efficiency of service delivery. Another article also highlighted standardization of wait time data as a key sustainability factor .
Public websites were cited as important tools for sustaining WTMS. Indeed, in some cases, communicating to the public about wait times included providing information not only on wait times for scheduled procedures but also on wait times for surgeons practicing such procedures .
Table 3 summarizes all the factors cited.
Workshop on WTMS
In March 2009, we organized a workshop that brought together 20 experts representing a cross-section of Canadian healthcare opinion and thought leaders. Participants were selected who had system and organizational level responsibility for healthcare provision, WTM leadership and/or health services research and education, and were in a position to provide experience-based critiques of WTMS. Participants included care providers, health system managers, policy-makers, and representatives of national associations of healthcare professionals and accreditation bodies. More recently, in a workshop at the Taming of the Queue conference in 2013, we were able to present and once again validate the results from our overall study.
The objectives of the 2009 workshop were to:
Review findings from our systematic review with an expert group of individuals who could potentially apply the research results;
Validate research findings with experts from different domains to ensure that the findings were sound and to explore how identification of the various factors could be developed into best practices;
Discuss ways to disseminate the research findings so as to be most helpful to policy-makers and managers;
Solicit suggestions for future research opportunities and research settings.
Workshop participants were asked to compare their experiences of WTMS and to identify success factors and constraints. Participants listed constraints and barriers to WTMS implementation and sustainability and ranked them in order of severity. Then they identified factors that improve WTMS implementation and sustainability. From this workshop, five main themes emerged. First, these healthcare managers and decision-makers had more to say about negative factors, namely constraints and barriers to WTMS, than about positive or success factors. Second, workshop participants brought forward new factors not identified in the literature that were specific to the Canadian context, particularly with regard to WTMS sustainability (Table 4). Third, they reported the same main factors for WTMS implementation as those identified in our systematic review, including leadership solutions at both the local and contextual levels (governance factors), payment incentives for physicians at the local level and funding at the contextual level (resource factors), change management support tools at the local level and standardized data at a higher level (tool factors). Table 4 presents the key barriers to both implementation and sustainability of WTMS initiatives. Fourth, they identified 10 main factors that improve WTMS implementation and sustainability (Table 5). This helped us to conceptualize, build, and test a tool to help managers implement and sustain WTMS at the organizational level. Finally, they stressed the fact that WTMS initiatives had been implemented across Canada ever since the September 2004 meeting of Canada’s First Ministers, and therefore by that time (2009) they had hoped to be seeing more studies on sustainability. Given that the first period covered by our systematic review ended in 2005, this last observation prompted us to extend our review in hopes of finding data on sustainability beyond that point. Ultimately, however, the data uncovered in the second portion of the systematic review did not really satisfy those expectations.
From this systematic review of 47 articles, we were able to identify key factors influencing WTMS, especially for implementation. Few studies explored the determinants of WTMS sustainability [22–26, 57–63]. With regard to evidence-based healthcare programs and practices, two recent systematic reviews looked at factors that influence the sustainability of such initiatives [64, 65]. Most of the studies reviewed there did not define the notion of sustainability in clear terms, and most did not use a rigorous research design to evaluate research findings on factors impacting sustainability . Although that review found the same main determinants of sustainability as emerged in our review, the factors identified were less accurate than those we were able to show. For example, that review did not identify subcategories for factors related to the cultural dimension, such as performance culture, quality improvement, or public awareness and empowerment. Furthermore, the studies reviewed did not always link the determinants of healthcare innovations’ sustainability to their impacts on particular goals or on benefits for patients.
Despite the lack of evidence in the literature on the notion of sustainability and its determinants, the discussions in the workshop helped to shed light on this notion. Indeed, the workshop allowed us to identify additional constraints and barriers related to WTMS sustainability (see Table 4). Moreover, the workshop participants’ notions concerning implementation differed from those identified in reviewed articles in two ways. First, the participants identified factors that negatively impacted WTMS implementation, in contrast to the success factors found in articles. They also identified factors, again negative, that were especially related to sustainability and were not in the literature. Most of those were related to governance at the local level, such as limited information sharing between professionals and managers, lack of time allocated to WTMS, and lack of partnership between professionals, especially physicians, and managers (see Table 4). Despite these differences, many of the factors identified in the literature and the workshop were similar, including those relating to implementation (see Table 4). As such, this process highlighted the importance of taking into account the potential limitations of using a theoretical framework for data extraction .
Because the workshop experts were healthcare managers and decision-makers directly involved in wait time reduction strategies in Canada, their input helped validate the findings and robustness of our systematic review, in addition to complementing our findings with sustainability factors that were rarely addressed in the literature.
We should also point out that an article published by our team in 2010 on a study of success factors in WTMS implementation showed the same main findings as did our systematic review . That study used data from interviews with key physicians, managers, policy-makers, and researchers from across Canada who had worked on initiatives to reduce wait times. The same governance factors were noted in that article, as were cultural factors related to physician involvement in WTMS and the need for a quality improvement culture. That study also highlighted the importance of aligning contextual-level and local policies. Funding levels and financial incentives were addressed both in that article and in our systematic review. As well, subsequent to that 2010 article and inspired by it, a very interesting tool was developed to assess the success of WTMS implementation, which was applied in an orthopedic program  (see Table 6).
Some of the factors identified in our study have been mentioned in other Canadian and international studies published in the grey literature. For example, the Canadian Centre for Policy Alternatives (CCPA) report in December 2005  mentioned coordination, dedicated evaluation tools, commitment and cooperation by all stakeholders, and an innovative culture of performance as the main factors that can enhance the success of a WTMS. The March 2006 report of the Association of Canadian Academic Healthcare Organizations  and the June 2006 Final Report of the Federal Advisor on Wait Times  also mentioned some of the factors identified in our study, such as those related to physicians’ involvement in WTMS. Another publication, by the Health Council of Canada , reported that interviews with leaders of initiatives to reduce wait times in Canada had revealed a number of key success factors. Unsurprisingly, these are similar to the factors identified in our study and include: support from governments leaders; strong program leadership that brings together administrative and clinical champions; full-time staff who are dedicated to making the program work; information systems that enable programs to centralize wait lists, to track wait times in local areas and province-wide, and to share this information publicly; adequate funding for the introduction of information systems and effective program leadership; and a broad, comprehensive approach to the many large and small changes required to reduce wait times for care .
At the international level, a study conducted by the King’s Fund in the UK  examined organizational characteristics that sustained WTMS improvements. Once again, some factors identified in our study were also in the King’s Fund report, such as the need for reliable information, strong leadership, and incentives, although our findings were more extensive.
Limitations of the systematic review
Our focus on the factors themselves, rather than on their impact on reducing wait times, was both a strength and a weakness. However, there are few studies on wait time reduction and on the success factors linked to WTMS impact on wait time. We did not establish a scale to measure the quality of evidence. Nor did we assess the quality of the research design for each study. Rather, we focused on our main objective, which was to identify the factors at different levels, and we used the conceptual framework to categorize each factor and to abstract all the articles in a consistent manner.
Another limitation of our review was that it included only English and French language studies and those published in peer-reviewed journals. Nevertheless, we were also able to consult the grey literature and to discuss it in the workshop and in subsequent presentations at conferences.
This is the first published review on factors that can influence success in implementing or sustaining WTMS in healthcare organizations. The conceptual framework used in this study is one that we previously used for a published study on success factors of WTMS implementation, which was based on interviews with physicians, healthcare managers, and policy-makers . Our framework, inspired by Parson’s widely recognized and robust four-quadrant model , was useful for organizing and analyzing the factors identified in the literature as influencing the implementation and sustainability of WTMS at both the systemic and local levels.
The paucity of articles with evidence on sustainability points strongly to the need for more research in that area related to impact on WTMS. Since our systematic review did not explore how these individual factors influenced reduction of wait times, future research should focus on success factors that help to both implement and sustain WTMS and that ensure reduction of wait times.
Nevertheless, our results can be useful to healthcare decision-makers and managers seeking to develop best practices based on success factors for WTMS implementation and sustainability. One example of such use is the tool presented in Table 5, which is a checklist of factors developed to support managers at the organizational level implementing a WTMS in one of the main orthopedic services of Ontario’s Eastern Regional Health Authority. This tool was shown, in a study, to have been a successful method of knowledge translation that contributed to the sustainability of that initiative .
Marie-Pascale Pomey, MD, MSc, PhD, is an associate professor in the Department of Health Administration, Public Health Research Institute, Faculty of Medicine, University of Montreal. Pierre-Gerlier Forest, PhD, is president of the Pierre Elliott Trudeau Foundation. Claudia Sanmartin, PhD, is a senior researcher in the Health Analysis Division, Statistics Canada. Carolyn DeCoster, PhD, RN, MBA, is executive director for Clinical & Zone Analytics, Data Integration, Measurement & Reporting, Alberta Health Services. Nathalie Clavel, MHA, is a research assistant, IRSPUM. Elaine Warren, RN, MSc, is regional director of Surgical Services, Eastern Health, and program director of Critical Care, Eastern Health. Madeleine Drew, MHA, is a research assistant, IRSPUM. Tom Noseworthy, MD, PhD, is an associate chief medical officer for Strategic Clinical Networks and Clinical Care Pathways, Alberta Health Services.
Wait time management strategies
Cumulative Index to Nursing and Allied Health Literature
Medical Literature Analysis and Retrieval System Online.
Siciliani L, Hurst J: Tackling excessive wait times for elective surgery: a comparative analysis of policies in 12 OECD countries. Health Policy. 2000, 72 (2): 201-215.
Canadian Institute for Health Information: Health care in Canada 2004: annual report. 2004, Ottawa: Canadian Institute for Health Information
Canada S: Access to health care services in Canada, January to December 2005. 2004, Ottawa: Statistics Canada
Health Canada: First Minister’s meeting on the future of health care 2004: a 10-year plan to strengthen health care. 2004, Ottawa
Prime Minister of Canada: Canada’s new government announces patient wait times guarantees. 2007, Ottawa, Press release, 4 April 2007
Health Council of Canada: Wading through wait times: what do meaningful reductions and guarantees mean?. 2007, Toronto
Trypuc J, Hudson A, Macleod H: Ontario wait time strategy: part 1. Healthc Q. 2006, 9 (2): 44-51.
Glynn PA, Donnelly LM, Calder DA, Brown JC: The Saskatchewan Surgical Care Network: toward timely and appropriate access. Hosp Q. 2003, 7 (1): 44-48.
Western Canada Waiting List Project, list of published papers 2008.http://www.wcwl.org/library/published_papers/?id=published_papers.6,
Priest A, Rachlis M, Cohen M: Why wait? Public solutions to cure surgical waitlists.http://healthcoalition.ca/wp-content/uploads/2010/01/why_wait_surgical_waitlists.pdf,
Senge P, Kleiner A, Roberts C, Ross R, Roth G, Smith B: Dance of change: challenges to sustaining momentum in learning organizations. 1999, New York: Doubleday
Parsons T: Social systems and the evolution of action theory. 1977, New York: Free Press
Pomey MP, Forest PG, Sanmartin C, De Coster C, Drew M: Wait time management strategies for scheduled care: what makes them succeed?. Healthcare Policy. 2010, 5 (3): 66-81.
Dixon-Woods M: Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011, 9: 39-10.1186/1741-7015-9-39.
Hatchuel A: Prospective et gouvernance: quelle théorie de l’action collective?. Prospective pour une gouvernance démocratique. Edited by: Heurgon E, Landrieu J. 2000, Paris: Editions de l’Aube, 29-42.
Shortell SM, Levin DZ, O’Brien JL, Hughes EF: Assessing the evidence of CQI: is the glass half empty or half full?. Hosp Health Serv Adm. 1995, 40 (1): 4-24.
Ham C, Kipping R, McLeod H: Redesigning work processes in health care: lessons from the National Health Service. Milbank Q. 2003, 81 (3): 415-439. 10.1111/1468-0009.t01-3-00062.
McLeod H, Ham C, Kipping R: Booking patients for hospital admissions: evaluation of a pilot programme for day cases. BMJ. 2003, 327 (7424): 1147-10.1136/bmj.327.7424.1147.
Maddison P, Jones J, Breslin A, Barton C, Fleur J, Lewis R, McSweeney L, Norgain C, Smith S, Thomas C, Tillson C: Improved access and targeting of musculoskeletal services in northwest Wales: targeted early access to musculoskeletal services (TEAMS) programme. BMJ. 2004, 329 (7478): 1325-1327. 10.1136/bmj.329.7478.1325.
Rozich JD, Resar RK: Using a unit assessment tool to optimize patient flow and staffing in a community hospital. Jt Comm J Qual Improv. 2002, 28 (1): 31-41.
Worthington D: Hospital waiting list management models. J Oper Res Soc. 1991, 42 (10): 833-843.
Trypuc J, MacLeod H, Hudson A: Developing a culture to sustain Ontario’s wait time strategy. HealthcarePapers. 2006, 7 (1): 8-24.
Trypuc J, Hudson A, MacLeod H: Evaluating outcomes in Ontario’s wait time strategy: part 4. Healthc Q. 2007, 10 (2): 58-67.
Trypuc J, Hudson A, MacLeod H: The pivotal role of critical care and surgical efficiencies in supporting Ontario’s wait time strategy: part3. Healthc Q. 2006, 9 (4): 37-45.
Trypuc J, Hudson A, MacLeod H: Ontario’s wait time strategy: part 1. Healthc Q. 2009, 9 (2): 44-51.
Trypuc J, Hudson A, MacLeod H: Expert panels and Ontario’s wait time strategy: part 2. Healthc Q. 2006, 9 (3): 43-49.
Cheng SM, Irish JC, Thompson LJ: Contract management of Ontario’s cancer surgery wait time strategy. Healthc Q. 2007, 10 (4): 51-58.
Pearson K, Meyer H: Waiting times: the search for equitable solutions. Case study: mater misericordiae adult public hospital continuum of care model. Aust Health Rev. 1991, 19 (4): 93-99.
Dexter F, Macario A, Traub RD, Hopwood M, Lubarsky DA: An operating room scheduling strategy to maximize the use of operating room block time: computer simulation of patient scheduling and survey of patients' preferences for surgical waiting time. Anesth Analg. 1999, 89 (1): 7-20.
Botten G, Grepperud S, Nerland SM: Trading patients. Lessons from Scandinavia. Health Policy. 2004, 69 (3): 317-327. 10.1016/j.healthpol.2003.12.014.
Cromwell DP, Mays L: Can a PC-based model assist the management of waiting lists? Observations from a case study. J Qual Clin Pract. 1999, 19 (3): 173-178. 10.1046/j.1440-1762.1999.00304.x.
Gauld R, Derrett S: Solving the surgical waiting list problem? New Zealand's 'booking system'. Int J Health Plann Manage. 2000, 15 (4): 259-272. 10.1002/hpm.596.
Hanning M: Maximum waiting-time guarantee - An attempt to reduce waiting lists in Sweden. Health Policy. 1996, 36 (1): 17-35. 10.1016/0168-8510(95)00791-1.
Hanning M, Spangberg UW: Maximum waiting time - a threat to clinical freedom? Implementation of a policy to reduce waiting times. Health Policy. 2000, 52 (1): 15-32. 10.1016/S0168-8510(00)00060-9.
Hefford B, Holmes A: Booking systems for elective services: the New Zealand experience. Aust Health Rev. 1999, 22 (4): 61-73. 10.1071/AH990061. discussion 74-77
Leach P, Rutherford SA, King AT, Leggate JRS: Generic waiting lists for routine spinal surgery. J R Soc Med. 2004, 97 (3): 119-120. 10.1258/jrsm.97.3.119.
Lundstrom M, Stenevi U, Thorburn W: Assessment of waiting time and priority setting by means of a national register. Int J Technol Assess Health Care. 1996, 12 (1): 136-140. 10.1017/S0266462300009454.
Ramchandani M, Mirza S, Sharma A, Kirkby G: Pooled cataract waiting lists: views of hospital consultants, general practitioners and patients. J R Soc Med. 2002, 95 (12): 598-600. 10.1258/jrsm.95.12.598.
Stainkey LA, Seidl IA, Johnson AJ, Tulloch GE, Pain T: The challenge of long waiting lists: how we implemented a GP referral system for non-urgent specialist' appointments at an Australian public hospital. BMC Health Serv Res. 2010, 10: 303-10.1186/1472-6963-10-303.
Channer KS: Waiting list management in cardiology: the New Zealand experience. Clin Manag. 2001, 10 (2): 65-71.
Kingston R, Carey M, Masterson E: Needs-based waiting lists for hip and knee arthroplasty. Ir J Med Sci. 2000, 169 (2): 125-126. 10.1007/BF03166915.
Tandon S, Machin D, Jones TM, Lancaster J, Roland NJ: How we do it: head and neck cancer waiting times. Clin Otolaryngol Allied Sci. 2005, 30 (3): 279-282. 10.1111/j.1365-2273.2005.00997.x.
Taylor K, Dangerfield B, Le Grand J: Simulation analysis of the consequences of shifting the balance of health care: a system dynamics approach. J Health Serv Res Policy. 2005, 10 (4): 196-202. 10.1258/135581905774414169.
Wright JG, Li K, Seguin C, Booth M, Fitzgerald P, Jones S, Leitch KK, Willis B: Development of pediatric wait time access targets. Can J Surg. 2011, 54 (2): 107-110. 10.1503/cjs.048409.
Bourne RB, Sibbald WJ, Doig G, Lee L, Adolph S, Robertson D, Provencher M: The Southwestern Ontario Joint Replacement Pilot Project: electronic point-of-care data collection. Can J Surg. 2001, 44 (3): 199-202.
Brunenberg DE, van Steyn MJ, Sluimer JC, Bekebrede LL, Bulstra SK, Joore MA: Joint recovery programme versus usual care: an economic evaluation of a clinical pathway for joint replacement surgery. Medical Care. 2005, 43 (10): 1018-1026. 10.1097/01.mlr.0000178266.75744.35.
Karvonen S, Ramo J, Leijala M, Holmstrom J: Productivity improvement in heart surgery - a case study on care process development. Prod Plann Contr. 2004, 15 (3): 238-246. 10.1080/09537280410001670322.
Mills RP, Heaton JM: Waiting list initiatives: crisis management or targeting of resources?. J R Soc Med. 1991, 84 (7): 405-407.
Mobb GE, Pugh F, Peeling B: How long is your waiting list? Experience of a urological waiting list initiative. J R Soc Med. 1994, 87 (3): 140-142.
Shankar PJ, Achuthan R, Haray PN: Colorectal subspecialization in a DGH. The way forward!. Colorectal Dis. 2001, 3 (6): 396-401. 10.1046/j.1463-1318.2001.00279.x.
Spark E, Sampath SA, Barry K, Johnson R: Pre-operative health screening for total knee replacement. J R Coll Surg Edinb. 1996, 41 (5): 325-328.
Hanning M, Lundstrom M: Assessment of the maximum waiting time guarantee for cataract surgery. The case of a Swedish policy. Int J Technol Assess Health Care. 1998, 14 (1): 180-193. 10.1017/S026646230001062X.
Bhatti TS, Harradine K, Davies B, Heather BP, Earnshaw JJ: First year of a fast track carotid duplex service. J R Coll Surg Edinb. 1999, 44 (5): 307-379.
Walley P, Silvester K, Steyn R: Knowledge and behaviour for a sustainable improvement culture. HealthcarePapers. 2006, 7 (1): 26-33. discussion 74-77
Cromwell D, Mays L: Waiting list statistics as performance indicators: observations on their use in hospital management. Aust Health Rev. 1998, 21 (4): 15-27. 10.1071/AH980015.
Mullen PM: Waiting lists in the post-review NHS. Health Serv Manage Res. 1994, 7 (2): 131-145.
Collins-Nakai R, Adams O, Saulnier M: A prescription for Ontario’s wait time strategy. HealthcarePapers. 2006, 7 (1): 46-50. discussion 74-77
Carruthers C: Sustaining the wait time strategy. HealthcarePapers. 2006, 7 (1): 51-54. discussion 74-77
Hamilton J, Grant B: A welcome link for patients who are waiting. Can Nurse. 2006, 102 (3): 24-27.
Glynn PA: Sustaining change: the imperative for patient access strategies. HealthcarePapers. 2006, 7 (1): 55-57. discussion 74-77
Martalog J, Bains S: Turning data into meaningful information. Healthc Q. 2009, 12 Spec No Ontario: 76-77.
Pfaff S, Guerriero L, Martalog J, Arscott L, Fontaine S, Laforet J: Transitioning initial success into sustainable results: the future of the WTIS. Healthc Q. 2009, 12 Spec No Ontario: 84-88.
Thabet R, Dummett M: Building a sustainable system: the making of the WTIS. Healthc Q. 2009, 12 Spec No Ontario: 43-51.
Buchanan D, Fitzgerald L, Ketley D, Gollop R, Jones JL, Lamont SS, Neath A, Whitby E: No going back: a review of literature on sustaining organizational change. Int J Manag Rev. 2005, 7 (3): 189-205. 10.1111/j.1468-2370.2005.00111.x.
Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M: The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012, 7: 17-10.1186/1748-5908-7-17.
Warren E: Final intervention report: an evidence-based approach to improving orthopedic flow and wait times. 2012, Ottawa: Canadian Health Services Research Foundation (CHSRF)
Rachlis MM: Public solutions to health care wait lists. 2005, Ottawa: Canadian Centre for Policy Alternatives (CCPA)
Association of Canadian Academic Healthcare Organizations: Wait Watchers II; measuring progress of wait time management strategies across ACAHO members. 2006, Ottawa: Association of Canadian Academic Healthcare Organizations
Postl BD: Final report of the Federal Advisor on Wait Times. 2006, Ottawa: Health Canada
Appleby J, Harrison R, Boyle S, Devlin N, Harley M, Harrison T, Locock L, Thorlby R: Sustaining reductions in waiting times: identifying successful strategies. 2005, London: King’s Fund
This research was funded by CIHR (IHSPR) through two operating grants and an emerging team grant. The co-authors thank Donna Riley for her significant contributions in translation and editing, and her careful re-reading, which helped bring greater clarity and focus to the article.
The authors affirm that they have no competing financial or non-financial interests to declare.
MPP was the PI responsible for leading the project. MPP, PGF, CS, CD, NC, and MD identified and analyzed the articles. MPP, MD, and NC wrote the first draft of the article. PGF, CS, CD, EW, and TN were involved in writing the final draft. All authors read and approved the final manuscript.
About this article
Cite this article
Pomey, MP., Forest, PG., Sanmartin, C. et al. Toward systematic reviews to understand the determinants of wait time management success to help decision-makers and managers better manage wait times. Implementation Sci 8, 61 (2013). https://doi.org/10.1186/1748-5908-8-61