Summary of study findings
In this study, we investigated the supports that three types of health system organizations (i.e
., RHAs, hospitals and PCPs) in two Canadian provinces (i.e
., Ontario and Quebec) currently have in place to facilitate EIDM. Based on thematic analysis of the data obtained from 57 interviews in 25 organizations, we found four main factors related to use of research in decision making.
“The organizational climate” was identified as one of the most important elements that could impact the use of research in decision-making, and within this element, developing and implementing an infrastructure or positions for encouraging knowledge use was identified as the most important. It was evident that participants recognized the value and importance of EIDM; however, they did not have either the time or skills to search for the research themselves, and therefore having a staff member or unit that can search and summarize the relevant research was useful.
”The linkage and exchange efforts” within and across organizations and networks was highlighted as essential since it facilitated ease of access to necessary research, enhanced dialogues between researchers and users, and assisted in the establishment of a culture that valued research evidence, even if the research was not immediately relevant. It essentially provided a network of contacts and experts that could be accessed to obtain relevant research to incorporate into the decision-making process.
“The facilitating pull efforts,” most specifically, a technical infrastructure and the ability to access research evidence when and where it is needed, was also identified as an essential element that, if properly in place (i.e., limited access restrictions), can facilitate the use of research in decision-making.
The pull efforts,” and more specifically, providing or enabling staff to participate in training programs, ensured that there were individuals within the organization who valued the use of research evidence in the decision-making process and also had the skills to acquire, assess, adapt and apply the evidence. This also ultimately fed into the organizational culture and the value of incorporating research into decision-making.
The findings of this study suggest that organizational commitments, coupled with the necessary infrastructures, tools and expertise, are essential supports needed to move healthcare organizations toward EIDM. These organizational efforts have to be sustained and evaluated to ensure that the supports align with decision-makers’ needs for evidence at the management level.
Summary of differences in responses
Interviewing multiple respondents and positions within various healthcare organizations enabled the cross-validation of the data. As was demonstrated in the results section and in the Tables, the majority of the coded responses were in alignment, and the data between Ontario and Quebec is fairly aligned. There were limited large differences in responses, and when there were differences of greater than 50%, it was understandable due to different position types (lending to different viewpoints or exposure in the organizational hierarchy) or organization types (lending to different purposes and perspectives). For example, a difference was found for the ‘training provided by the library staff’ element, between the library managers (100%), senior managers (15%), and knowledge brokers (27%). This could be because, although it is available and offered by librarians, maybe it is not visible or properly marketed to senior management, and therefore different position types may have different views. Difference in organization types may also lead to differences in responses; for example, there was one difference between PCPs (22%) and RHAs (88%) with respect to ‘being part of a regional, provincial or national network.’ We hypothesize that this is most probably because RHAs operate on a regional/provincial level and exchange ideas and programs with different RHAs, whereas PCPs operate at a much more local/hospital level and would not be part of broader networks.
Relation to other studies
To our knowledge, this is the first qualitative study with a comprehensive framework of possible supports for EIDM. Studies do exist that examine either: one type of health service organization i.e., mental health services and laboratory units; a small number of interventions; or focus on decisions made at the clinical level and not at the management level [19–23]. In our scoping review, which served as the background for the guiding framework for this research, we did not identify any studies discussing the effects of a full research knowledge infrastructure on the use of evidence by managers and policy-makers, but we did uncover 25 qualitative studies and one randomized control trial that addressed different components of a potential research knowledge infrastructure . Studies, like the 26 that we identified in the scoping review, continue to be published, such as a recent one from a member of our research team examining the availability of scientific journals, databases, and health library services in health ministries in Canada . To our knowledge, this study is the first of its kind where: a number of health service organizations i.e., primary care practices, hospitals, and regional health authorities were examined; one to three key actors in each organization were interviewed to gain a broad perspective as well as to ensure alignment of responses; the respondents were asked about a wide range of interventions that could be undertaken either by the organization or by the health system in order to facilitate evidence-informed decision-making; and the focus was on management decision-making and not clinical decision-making.
The framework for this research was built upon a scoping review that reviewed the current literature to identify infrastructural initiatives that organizations have implemented to support the use of evidence in decision-making . Our findings in this research mirror what was discovered in the scoping review, i.e., in the scoping review, we found that most studies focussed on ‘establishing a climate for research use,’ and in this research, all participants mentioned having at least one element within this domain in their organization, and most categorized at least one sub-element of climate within the top three most important elements. The scoping review also showed that: the next most addressed domains in the literature were efforts focusing on ‘facilitating pull,’ ‘linkage and exchange,’ and ‘pull activities,’ which was also highly commented on in this research; and the domain least reported on in the literature, ‘evaluation efforts,’ was also the least commented on in this research.
The responses received from the participants regarding the most essential elements to support the use of research evidence in decision-making were consistent with other literature. First, ‘establishing a climate for research use’ was classified as an integral foundation on which to ensure the use of research in decision-making. Implementing an infrastructure to support the use of EIDM is an exercise in organizational change, and research has demonstrated that a supportive climate and culture is an essential foundation element to support change in general, as well as specifically to supporting EIDM [25–27]. The second most essential element highlighted by the participants was ‘implementing facilitating pull efforts,’ which includes the implementation of a supportive technical infrastructure and access to research evidence, articles and databases. Other research and frameworks [4, 28–30] have demonstrated that the technical infrastructure needs to be in place in order to facilitate the use of research in decision-making, i.e., “Strategic goals, critical appraisal skills and enthusiasm for EIDM are of limited use if organizations lack the infrastructure to acquire research evidence”  p.9. The third and fourth most essential elements (both mentioned by many participants), ‘implementing pull efforts’ and ‘linkage and exchange efforts,’ were also supported by the literature. ‘Pull efforts’ (i.e., engaging knowledge brokers or sending staff to training programs) were mentioned frequently in the literature as integral factors in building a framework for EIDM [4, 12, 31–34]. Linkage and exchange efforts’ have been mentioned in numerous frameworks [4, 14, 27, 35, 36]. Strong links between decision-makers and researchers can enhance the transfer of research into practice , and as can be seen in this research, participants use these links to build their knowledge base and tap into when necessary.
Furthermore, strong links between decision-makers and research producers can enhance the type of research being produced, i.e., make it more relevant and highly applicable to the needs of the users, and ensure that the research addresses high priority issues [38, 39]. As is discussed in the “two communities theory,” researchers and decision-makers live in two different communities, with different values, reward systems, and languages . This needs to be addressed by increasing the linkage and exchange between the two groups to achieve a shared understanding, which can influence the agenda setting, the type of research conducted, and the transfer of research into practice [32, 41].
Findings of this study are supported and complemented by other bodies of literature that examine sustainable system changes. In order to build a strong EIDM infrastructure, it is crucial to assess the environment and include in the system the supports identified in the scoping review . There is agreement between the organizational change literature and with the findings in this study that instituting a change is multi-layered, multi-faceted, and multi-challenging [25, 31, 42]. This study provides evidence that a supportive climate is essential; however, that alone cannot ensure EIDM. Tools need to be implemented so that EIDM is supported, encouraged and utilized every time. Without the infrastructure, instruments and tools, EIDM will be difficult to achieve, and it will not occur in a consistent and repeated manner.
Strengths and limitations
There are two main strengths related to this study. First, we interviewed up to four participants in three different positions from each organization, which increased our confidence in the presented data, enabled us to cross-validate responses, and facilitated obtaining a global view of what elements were in place in the different participating organizations. Furthermore, there was limited variation in participants’ responses by position or organization type. Second, participants were from the three main types of organizations within the healthcare sector that are responsible for funding and delivering the bulk of healthcare services in Ontario and Quebec.
There are four main limitations to this study. First, some participants, notably library managers, were mostly involved with others at the clinical level and were not able to provide us with much information on evidence-based decision-making at the managerial level. However, by interviewing more than one informant from each organization, we were able to get a broad view of what infrastructures were in place and viewed as important for evidence-based decision-making at the managerial level. Second, most participants from the same organization were not interviewed at the same time. A focus group may have provided more consistent data, yet participants may have hesitated to speak openly in front of others. Focus groups were not utilized due to costs and scheduling concerns. Third, there was poor recruitment from the PCPs, which could be because of the lack of human resources allocated to KT-related duties, as is evidenced by the absence of library staff or resources on site and of knowledge broker-like personnel. Finally, while our sampling strategy was intended to be quite thorough, we do recognize that we were examining a best case scenario at a certain point in time. While we do anticipate that other organizations can learn from the high performers, no comparisons were undertaken with provinces or organizations that have not yet invested in knowledge translation initiatives.
The present study is the second phase in a broader program of research: the first phase being an environmental scan and scoping review, this second phase, and then the third phase that will consist of a large cross-sectional web survey among all RHAs and hospitals in Ontario and Quebec. This survey will provide a more in-depth and broader picture of the different supports implemented to facilitate evidence-informed decision-making in Canadian healthcare organizations. This research may serve as a springboard to cross-organization and cross-system research to better understand how to match particular supports to different contexts. The ultimate purpose of this research program is to develop context-specific interventions and then properly evaluate them to determine which interventions can facilitate the transfer of research evidence into decision-making.
Implementation research that can identify barriers and facilitators of different interventions is essential. However, research on the KT processes and potential tools that can facilitate the uptake of research into decision-making is also needed [43, 44]. One domain that was not strong in any of the participant organizations was the domain of evaluating KT efforts. A review of the current literature suggests that there are not many evaluations of KT interventions at the organizational level. Future research should examine KT interventions, infrastructural components, and tools to identify which elements are successful in which contexts.
The uptake of innovation and change in health system organizations has traditionally been a challenging process [25, 42]. The present study focused on organizations that have already demonstrated strategic structures and processes to support evidence-informed policy-making. It identified which elements these organizations currently have in place and which are held to be most important. What is clear from this research is that many infrastructural interventions exist and that organizations should benefit by building an infrastructure that not only encourages but also supports the use of research in decision-making. Those organizations that want to institute EIDM may want to explore some of the top four interventions identified by the respondents in this research and pursue those interventions to increase the prospects of the uptake of EIDM.
While some of the interventions mentioned by the participants can be quite costly and difficult to develop and implement, they may be easily transferable between organizations. The health system (i.e., hospitals, networks, provincial and federal governments) may benefit from exploring the idea of either encouraging resource and idea sharing, or investing in some of the larger up front significant investments in order to ensure widespread dissemination usage. For example, investing in a one-stop shopping website or free access to journal articles are initiatives that larger organizations can facilitate, yet smaller organizations can also reap the benefits. Such steps may help to improve the use of research evidence.