A cross-sectional survey of supports for evidence-informed decision-making in healthcare organisations: a research protocol
© Ouimet et al.; licensee BioMed Central Ltd. 2014
Received: 4 August 2014
Accepted: 19 September 2014
Published: 9 October 2014
This protocol builds on the development of a) a framework that identified the various supports (i.e. positions, activities, interventions) that a healthcare organisation or health system can implement for evidence-informed decision-making (EIDM) and b) a qualitative study that showed the current mix of supports that some Canadian healthcare organisations have in place and the ones that are perceived to facilitate the use of research evidence in decision-making. Based on these findings, we developed a web survey to collect cross-sectional data about the specific supports that regional health authorities and hospitals in two Canadian provinces (Ontario and Quebec) have in place to facilitate EIDM.
This paper describes the methods for a cross-sectional web survey among 32 regional health authorities and 253 hospitals in the provinces of Quebec and Ontario (Canada) to collect data on the current mix of organisational supports that these organisations have in place to facilitate evidence-informed decision-making. The data will be obtained through a two-step survey design: a 10-min survey among CEOs to identify key units and individuals in regard to our objectives (step 1) and a 20-min survey among managers of the key units identified in step 1 to collect information about the activities performed by their unit regarding the acquisition, assessment, adaptation and/or dissemination of research evidence in decision-making (step 2). The study will target three types of informants: CEOs, library/documentation centre managers and all other key managers whose unit is involved in the acquisition, assessment, adaptation/packaging and/or dissemination of research evidence in decision-making. We developed an innovative data collection system to increase the likelihood that only the best-informed respondent available answers each survey question. The reporting of the results will be done using descriptive statistics of supports by organisation type and by province.
This study will be the first to collect and report large-scale cross-sectional data on the current mix of supports health system organisations in the two most populous Canadian provinces have in place for evidence-informed decision-making. The study will also provide useful information to researchers on how to collect organisation-level data with reduced risk of self-reporting bias.
KeywordsHealth systems Knowledge translation Research evidence Cross-sectional study
Organisational absorptive capacity—the capacity for organisations to acquire and use external knowledge—depends on three key factors, namely, access to external sources of information (e.g. access to scientific journals), prior knowledge (e.g. employees’ skills in research methods) and social integration (e.g. ties with research producers and knowledge brokers) ,. An environmental scan in Canadian healthcare organisations and a scoping review of the literature on supports (i.e. positions, programs, interventions, instruments or tools) implemented across the health systems to support evidence-informed decision-making (EIDM) allowed us to identify four organisational-level components that can be found in an organisation’s research knowledge infrastructure: (1) climate for research use (e.g. mission, vision, values and strategic plan that reflect the value placed on the use of research evidence), (2) research production (e.g. ensuring that the appropriate research commissioning capacity is in place), (3) activities used to link research to action, which include push efforts (e.g. knowledge intelligence service that scans the research literature), facilitating pull efforts (e.g. enabling easy access to research evidence through physical tools and resources), pull efforts (e.g. training and continuing education that focus on finding and using research evidence in decision-making) and linkage and exchange efforts (e.g. meetings that highlight relevant research) and (4) evaluation efforts (e.g. monitoring and evaluation efforts on the use of research in decision-making) ,. In-depth semi-structured interviews conducted in three types of healthcare organisations (regional health authorities, hospitals and primary care practices) in two Canadian provinces (Ontario and Quebec) reveal the current mix of supports that these organisation have in place and the ones that are perceived to facilitate the use of research evidence in decision-making ,. Based on the findings of this qualitative study and of existing research syntheses -, we developed a web survey to collect cross-sectional data about the specific supports that regional health authorities and hospitals in two Canadian provinces (Ontario and Quebec) have in place to facilitate EIDM.
The types of supports that will be examined in this study are those that have the potential to overcome the key barriers to research use, such as the lack of availability and of access to research (sub-optimal dissemination channels), poor clarity/relevance/reliability of research findings, poor timing/loss of opportunity, lack of user research skills and costs . The types of supports that will be studied are also those that can be linked to the key facilitators of research use, such as availability and access to research (improved dissemination channels), improved clarity/relevance/reliability of research findings and research collaboration and ties between decision-makers, research staff and external researchers .
The chosen research design is a cross-sectional web survey. Targeted participants are all regional health authorities and all general hospitals in Quebec and Ontario as categorised and reported by the web sites of their respective ministries of health. Specialised (non-general) hospitals, such as rehabilitation hospitals, hospitals for chronic patients and hospitals for psychiatric patients or patients suffering from addiction, will not be considered in this study. In Quebec, the study population includes 18 regional health authorities and 124 hospitals; in Ontario, the population includes 14 regional health authorities, namely local health integration networks (LHINs), and 129 hospitals.
Themes covered in the survey by data collection (DC) step and type of informants
Data collection step
Manager of other units
Mention of importance of research evidence in formal mission, vision or values statements
Certified by Accreditation Canada (hospitals only)
In-house library or a documentation centre that provides access to research evidence
Other units whose roles and activities might include acquiring, assessing, adapting/packaging and/or disseminating research evidence
Direct access to a knowledge broker, i.e. someone whose role is primarily to keep the CEO informed about the latest research evidence that might be of strategic value
Presence of a chief information officer
Range of 12 services provided or not by the library to the organisation’s employees
Refraining from subscribing to academic journals or bibliographic databases due to subscription costs
Dissemination of research evidence to inform the decision-making process in one or more units
Production of systematic reviews
Production of traditional literature reviews or rapid reviews
Production of assessments of the quality and local applicability of systematic reviews
Production of summaries or abstracts of primary studies
Production of summaries or abstracts of systematic reviews
Organisation of interactive meetings or workshops to share new research evidence with staff
Formal invitations to researchers from other organisations to share research evidence with members of the organisation
Formal collaboration with researchers from other organisations in preparing primary studies
Formal collaboration with researchers from other organisations in preparing systematic reviews
Contracting out to provide training sessions within the organisation on how to acquire, assess, adapt/package, disseminate and/or use research evidence to inform decision-making
Employees whose formal role includes acquiring, assessing, adapting/packaging, and/or disseminating research evidence
Employees whose formal role includes establishing and/or maintaining relationships with researchers to inform decision-making
Employees whose formal role includes the development of training tools aimed at increasing your organisation’s internal capacity to acquire, assess, adapt/package, and/or disseminate research evidence
Resources and funding to monitor capacity to acquire, assess, adapt/package and/or disseminate research evidence
Our team developed the survey questionnaires iteratively through multiple team meetings. The survey questionnaires solely include easy-to-answer, yes/no questions about organisational attributes. They do not include any question about individual attitude, belief or behaviour. Therefore, formal psychometric validation was not needed. Conceptual definitions of key terms such as `research evidence’ or `systematic reviews’ are provided in the questionnaires.
One of the main methodological concerns and challenges of surveys that aim at collecting organisation-level data is that they often rely on one or a few key informants that might not always be the most reliable informants to answer all survey questions. The challenge is thus to design a data collection procedure that will increase the likelihood that each survey question is being answered by the most informed respondent available.
In an effort to meet this challenge, we developed an innovative data collection procedure to increase the likelihood that only the best-informed respondent available answers each survey question. Survey participants who feel that they are not the right person to answer one or more questions will have the option not to respond and to provide the contact information of one or more persons (only one per unanswered question) from the same unit who he/she feels could provide a more accurate answer. The survey was programmed in a way that the one or more questions for which an informant reported not being the right person to provide an accurate answer will automatically be sent to the one or more secondary contact persons named by the primary informant. The secondary contact person(s) will then be invited to answer only the one or more questions that were allocated to him/her. Most importantly, the secondary contact person(s) will have the same option as the primary informant, that is, to forward one or more of these questions to a third informant, etc. This process will stop automatically after the fifth informant.
The entire web survey will be administrated by the services Centre APTI operated within the Faculty of the Social Sciences (Université Laval, Québec, Canada). The data collection will last for approximately five months. Up to three reminders will be sent to all survey respondents by email.
Data analysis will take the form of simple univariate and bivariate descriptive statistics of supports for evidence-informed decision-making. These supports will be cross-tabulated by type of organisation (i.e. regional health authorities vs. hospitals) and by province (i.e. Quebec vs. Ontario). We will also compute the proportion of survey questions for which primary informants identified secondary informants to minimise reporting bias.
The research protocol was submitted to the Ethics Research Committee of the CHU de Québec. The committee discussed the project in closed session on March 31, 2014. After evaluation and discussion, the committee noted that the research did not meet the definition of research involving human participants, according to the 2nd edition of Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS2 Article 2.1), given that the survey focuses on information that respondents are allowed to communicate as employees (the information relates to their work and does not target them personally). The Committee declared that the proposed project does not fall under the competency of the Committee. The McMaster University Research Ethic Board agreed with this decision.
Participation in this study will be entirely voluntary, and there will be no compensation provided for participation. Key informants will be able to put an end to their participation at any time without negative consequences or prejudice and without having to justify their decision. Proper arrangements will be made to ensure the confidentiality of the information provided by the key informants. The survey does not aim to evaluate the performance of the recruited organisations or to collect any information at the individual level. It will be impossible to identify individual informants or organisations in the publication or presentation of the study findings.
This study will be the first to collect and report large-scale cross-sectional data on the current mix of supports that health system organisations of the two most populous Canadian provinces have in place for evidence-informed decision-making. The study will also provide useful information to researchers on how to collect organisation-level data with reduced risk of self-reporting bias. Finally, the collected organisational data will be used to develop a protocol for a cross-sectional study aimed at examining organisational and individual correlates of research mobilisation by managers (and their advisers) of a random sample of the organisations that will participate to the organisational study described in this protocol.
This project is funded through the KT Canada network, which in turn is funded by the Canadian Institutes of Health Research. JNL, JMG and MPG receive salary support from the Canada Research Chairs Program.
- Todorova G, Durisin B: Absorptive capacity: valuing a reconceptualization. Acad Manag Rev. 2007, 32: 774-786. 10.5465/AMR.2007.25275513.View ArticleGoogle Scholar
- Ouimet M, Landry R, Ziam S, Bédard P-O: The absorption of research knowledge by public civil servants. Evidence & Policy. 2009, 5: 331-350. 10.1332/174426409X478734.View ArticleGoogle Scholar
- Ellen ME, Léon G, Bouchard G, Lavis JN, Ouimet M, Grimshaw JM: What supports do health system organizations have in place to facilitate evidence-informed decision-making? A qualitative study. Implement Sci. 2013, 8: 84-10.1186/1748-5908-8-84.View ArticlePubMedPubMed CentralGoogle Scholar
- Ellen ME, Lavis JN, Ouimet M, Grimshaw J, Bédard P-O: Determining research knowledge infrastructure for healthcare systems: a qualitative study. Implement Sci. 2011, 6: 60-10.1186/1748-5908-6-60.View ArticlePubMedPubMed CentralGoogle Scholar
- Ellen ME, Léon G, Bouchard G, Ouimet M, Grimshaw J, Lavis JN: Barriers, facilitators and views about next steps to implementing supports for evidence-informed decision-making in health systems: A qualitative study. Under Rev 2014Google Scholar
- Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J: A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014, 14: 2-10.1186/1472-6963-14-2.View ArticlePubMedPubMed CentralGoogle Scholar
- Wallace J, Nwosu B, Clarke M: Barriers to the uptake of evidence from systematic reviews and meta-analyses: a systematic review of decision makers’ perceptions. BMJ Open. 2012, 2: 1-Google Scholar
- Humphries S, Stafinski T, Mumtaz Z, Menon D: Barriers and facilitators to evidence-use in program management: a systematic review of the literature. BMC Health Serv Res. 2014, 14: 171-10.1186/1472-6963-14-171.View ArticlePubMedPubMed CentralGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.