Determining research knowledge infrastructure for healthcare systems: a qualitative study
© Ellen et al; licensee BioMed Central Ltd. 2011
Received: 11 April 2011
Accepted: 6 June 2011
Published: 6 June 2011
This study examines research knowledge infrastructures (RKIs) found in health systems. An RKI is defined as any instrument (i.e., programs, interventions, tools) implemented in order to facilitate access, dissemination, exchange, and/or use of evidence in healthcare organisations. Based on an environmental scan (17 key informant interviews) and scoping review (26 studies), we found support for a framework that we developed that outlines components that a health system can have in its RKI. The broad domains are climate for research use, research production, activities used to link research to action, and evaluation.
The objective of the current study is to profile the RKI of three types of health system organisations--regional health authorities, primary care practices, and hospitals--in two Canadian provinces to determine the current mix of components these organisations have in their RKI, their experience with these components, and their views about future RKI initiatives.
This study will include semistructured telephone interviews with a purposive sample region of a senior management team member, library/resource centre manager, and a 'knowledge broker' in three regional health authorities, five or six purposively sampled hospitals, and five or six primary care practices in Ontario and Quebec, for a maximum of 71 interviewees. The interviews will explore (a) which RKI components have proven helpful, (b) barriers and facilitators in implementing RKI components, and (c) views about next steps in further development of RKIs.
This is the first qualitative examination of potential RKI efforts that can increase the use of research evidence in health system decision making. We anticipate being able to identify broadly applicable insights about important next steps in building effective RKIs. Some of the identified RKI components may increase the use of research evidence by decision makers, which may then lead to more informed decisions.
Significant worldwide investments that are made in biomedical and health research are underutilised because of challenges to knowledge translation (KT). KT is defined as 'the exchange, synthesis and ethically-sound application of researcher findings within a complex system of relationships among researchers and knowledge users to improve health' . Evidence shows that health systems frequently fail to optimally use research evidence, which leads to inefficiencies, reduced quantity and quality of life for citizens, and lost productivity .
Multiple factors determine the nature and extent of engagement in KT by different stakeholder groups. A common challenge that all decision makers face relates to the lack of knowledge management skills and infrastructure (i.e., the sheer volume of research evidence currently produced; access to research evidence; time to read; and skills to appraise, understand, and apply research evidence). Better knowledge management is necessary but probably insufficient to ensure effective KT given other challenges that may operate at different levels, including the healthcare system (e.g., financial disincentives), healthcare organisation (e.g., inappropriate skill mix, lack of facilities or equipment), healthcare teams (e.g., local standards of care not in line with desired practice), individual professionals (e.g., knowledge, attitudes, and skills), and patients (e.g., low adherence to medical advice) . Frequently, multiple challenges are present at different levels of the healthcare system. A further challenge for KT relates to the dynamism and constant evolution of healthcare. As a result, KT approaches and activities need to recognise and keep pace with the changes in the healthcare sector.
Physical and cognitive access to evidence are challenges to KT for health system managers and policy makers. Three primary KT approaches target these groups: push, pull, and exchange efforts . The push approach includes activities undertaken by researchers to package and disseminate research evidence outside the scholarly community. Pull activities focus on the efforts by health system managers and policy makers to access and use research evidence. Exchange activities focus on building and maintaining relationships between researchers and managers and policy makers. We propose to study research knowledge infrastructures (RKIs) that build on push, pull, and/or exchange components.
In this study, RKI is defined as any instrument (i.e., programs, interventions, tools, devices) implemented in key organisations and broader health systems of which they are a part (hereafter called health systems) in order to facilitate access, dissemination, exchange, and/or use of evidence. These instruments can be subdivided into two broad categories: (1) technological instruments (e.g., electronic databases, search engine) and (2) organisational instruments (e.g., documentation specialists; data analysts, such as epidemiologists; knowledge brokers who, for example, manage the collaboration between the organisation and external information and knowledge producers; and training programs to help in, for example, searching for information, appraising information, adapting and using both statistical data and academic research) . Multiple types of evidence can be used by health system managers and policy makers, including (among others) administrative information, legal information, academic research, financial data, and surveillance data. In this project, we will be focusing on access to and utilisation of the following types of evidence: academic research outputs (i.e., articles, research reports, and books) and population and health system data (i.e., surveillance data, service utilisation data, and other nonfinancial performance data).
Possible organisational level components of a research knowledge infrastructure (RKI)
I. Climate for Research Use
• Accreditation acknowledges and rates the use of research evidence in decision making
• Mission, vision, values, and strategic plan reflect the value placed on the use of research evidence in decision making
• Structures or positions exist within the organisations to ensure accountability for using research evidence in decision making
• Clear points of contacts within organisations regarding where to turn to obtain research evidence
• Formal and informal relationships with people outside the organisation who can assist in obtaining the appropriate research evidence
• Recruitment and retention strategies that reflect the value of the use of research evidence in decision making
• Recognition of employees who use research evidence within the organisation
II. Research Production
• Participating in regular priority setting processes for the research evidence needed to meet managerial and policy-making needs
• Ensuring that the appropriate research commissioning capacity is in place to commission or execute the research if it is deemed high priority yet no applicable research is available
III. Activities Used to Link Research to Action (4 parts)
IIIa. Push Efforts
• Knowledge intelligence service that scans the literature and distributes relevant research evidence throughout the organisation
• Individual(s) responsible for identifying teaching moments to profile research evidence
IIIb. Facilitating Pull Efforts
• Enabling 'easy access' to research evidence through physical tools (e.g., information technology systems)
• Enabling 'easy access' to research evidence through appropriate resources (e.g., websites with optimally packaged reviews)
IIIc. Pull Efforts
• Summarising or conducting primary research via a rapid response unit that supports the use of research in decision making
• Decision-making processes that promote the use of research in decision making
• Self-assessment tools that focus on capacity to find and use research evidence in decision making
• Training and continuing education that focus on finding and using research evidence in decision making
• Use of dedicated staff to 'pull' research into decision making
IIId. Exchange Efforts
• Regular meetings that highlight relevant research
• Interactive workshops that focus on the use of research in decision making
IV. Evaluation of Efforts
• Monitoring and evaluation efforts on the use of research in decision making
We conducted an environmental scan of current RKIs in two Canadian provinces (i.e., extraction of key information from organisations' websites and informal interviews with key informants). Our informal interviews with 17 key informants were aimed at generating examples of tools, interventions, and components of RKIs. With respect to the RKI domain of Climate for Research Use, many of the informants referred to their organisation's strategic plan, which explicitly supported the use of research evidence in decision making. Most of the informants reported having existing relationships with researchers, and formal relationships were often facilitated by the structure of the organisation. In the RKI domain of Research Production, some informants suggested the existence of priority-setting processes, and although research production appears limited within most organisations, some suggested they have the capacity to commission research. Efforts to disseminate research findings both within and across organisations and the usage of knowledge brokers were mentioned when discussing push activities.
Many of the key informants mentioned that the categories within the Facilitating Pull domain enabled them to access research evidence to be used in decision making (i.e., organisations benefited from full access to bibliographic databases through partnerships, some organisations have comprehensive library services, and some have information tools to assist management). Within the Pull Activities domain, two key informants identified the establishment of rapid response units that produced information syntheses within approximately two weeks for managers, decision makers, and policy makers. Furthermore, various forms of training programs were discussed that educated participants in the use of research evidence in decision making. No examples of the Evaluation of Efforts to Link Research to Action were provided. The environmental scan demonstrates that health system managers and policy makers in Ontario and Quebec have instituted in their organisations and in the health system as a whole varying components of an RKI; yet, the breadth, depth, usage, and effectiveness of these efforts have yet to be determined.
We also conducted a scoping review as a preliminary assessment of the size and scope of the available literature on the effectiveness of particular models, or specific components, of RKI, as well as regarding the problems they are addressing, other features of RKIs that may influence their design, and related implementation considerations. We manually reviewed (a) http://healthsystemsevidence.org, a continuously updated repository of syntheses of research evidence about governance, financial, and delivery arrangements within health systems and about implementation strategies that can support change in health systems; (b) a database of studies used in a systematic review of the factors that influence the use of research evidence in public policy making; and (c) other studies or reports that we were aware of that were applicable to our scoping review. In reviewing these databases, we were trying to identify all primary studies that address RKI either in whole or in part.
We were unable to identify any studies evaluating the effects of a full RKI on the use of evidence by health system managers and policy makers; however, the scoping review did uncover 25 qualitative studies and one randomised control trial that addressed different components of the RKI framework (for a list of included studies, please contact MEE). The majority of the studies (25 studies) addressed categories within the Climate for Research Use domain, while the next most addressed domains were Activities Used to Link Research to Action, Facilitating Pull, and Pull Activities (16 studies). Fourteen studies evaluated the Research Production domain, and 10 studies addressed exchange activities. A minority of the studies (seven studies) reported on Push Activities, and even fewer (two studies) reported on the Evaluation of Efforts to Link Research to Action. The one subdomain that was reported most often, by 20 studies, as influencing the use of research in decision making was the formal and informal relationships to people outside the organisation who can assist in obtaining the appropriate research evidence (see Additional File 1 for illustrative examples associated with each category). This scoping review illustrates that while numerous independent initiatives are in place in different healthcare systems, no one RKI framework has been developed or studied to determine overall effectiveness.
Building on the environmental scan and scoping review, we propose to conduct interviews with health system managers and policy makers in Ontario and Quebec to determine which RKI components have actually been implemented and are perceived to have added value in their organisations, the barriers and facilitators to implementing these RKI components, and views about how to further support RKI development. The RKI project will be a springboard to cross-organisation and cross-system research to better understand how to match particular RKI elements to different contexts.
The objectives of our study are as follows:
To profile the RKI of three key types of healthcare organisations in two Canadian provinces (Ontario and Quebec): regional health authorities, primary care practices, and hospitals
To identify barriers and facilitators to implementing various RKI components
To determine what these organisations view as important next steps in building effective RKIs
We will conduct in-depth, semistructured telephone interviews with a senior management team member (who will be focused more on organisational infrastructure), library/resource centre manager (who will be focused more on technical infrastructure), and (for some types of large organisations) a knowledge broker. These interviews will be conducted in three regional health authorities (three participants interviewed in each, for a total of 18 interviews), five or six purposively sampled hospitals (depending on the hospital's size, we will interview two or three participants in each hospital, for a total of 29 interviews), and five or six primary care practices (with two individuals interviewed in each organisation, for a total of 10 to 12 interviews) in Ontario and Quebec.
We are interested in examining organisations that have already participated in strategic behaviour with respect to RKI (or that are linked to such organisations). We identified a sample of these organisations by examining the publicly available list of participants that have been a part of the Canadian Health Services Research Foundation's Executive Training for Research Application program. This program is geared towards health system managers and policy makers in Canadian health system organisations and aims to develop capacity and leadership to optimise the use of research evidence. The organisations from which these participants are drawn committed to having one or more of their senior executives enroll in the program and participate in four 1- to 2-week training sessions over a two-year period, supporting their executive(s) in designing and executing an evidence-based 'intervention project', and sending their Chief Executive Officer to a final meeting where the intervention project would be discussed. We augmented the sample based on our knowledge of other organisations that have demonstrated strategic behaviour with respect to RKI.
The focus of the interviews will be to explore which RKI components have been implemented and are perceived to have added value in the participant's organisation (and which not), the barriers and facilitators encountered in implementing these RKI components, and views about next steps in supporting the further development of RKIs. Semistructured interviews were utilised because they allow the interview participants to respond freely and focus on the area of the framework that the respondents think most apply to their organisation . Furthermore, using semistructured interviews permits the interviewer to probe issues that may be of interest to the current research but that are not specifically addressed by the interview guide .
We will tape and transcribe all interviews, and they will be verified by the interviewer before analysis begins. NVivo, Version 9, (QSR International, Cambridge, MA) will be used for data management. A constant comparative method for the thematic analysis of the interview data will be used (with one very experienced qualitative researcher working with one doctoral student to identify themes). Interviews will be analysed in clusters based on their organisations, meaning all interviews from one organisation will be analysed in tandem, followed by all other institutions within that organisational context (i.e., regional health authorities, hospitals, and primary care practices). First we will read the entire interview to get a sense of the whole interview and initial impressions. Then we will read the text a second time, code units of text, and compare initial codes. Coded segments will then be reanalysed, coded into subcategories, and compared again. Two researchers will constantly compare the coding during each stage.
Upon completion of the qualitative study, we will produce a report that describes the RKI status in regional health authorities, hospitals, and primary care practices; identifies barriers and facilitators to implementing various RKI components; and provides an overview of what these health system managers and policy makers view as important next steps in building effective RKIs. Based on these qualitative findings, we will develop a protocol for a cross-sectional survey that will generate more generalisable information.
The environmental scan and scoping review have identified key elements of RKI, and this qualitative work will explore the extent of the use of these elements in different settings. We anticipate being able to identify broadly applicable insights about important next steps in building effective RKIs. This is the first qualitative examination of potential RKI efforts that can increase the use of research evidence in health system decision making. Some of the identified RKI components may increase the use of research evidence by decision makers, which may then lead to more informed decisions, and hopefully to a strengthened health system and improved health.
We thank Amanda Hammill and Katie Cao from McMaster University who assisted in the scoping review. We also acknowledge Michael Wilson and Christina Catallo who conducted a majority of the systematic review, which was used as a baseline for the scoping review.
This project is funded through the KT Canada network, which in turn is funded by the Canadian Institutes of Health Research.
JNL and JG receive salary support from the Canada Research Chairs Program.
- Canadian Institute for Health Research: CIHR Knowledge Translation Definition. 2010, Ref Type: Electronic Citation, [http://www.cihr-irsc.gc.ca/e/26574.html]Google Scholar
- Grol R: Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001, 39: II46-II54.View ArticlePubMedGoogle Scholar
- Grimshaw JM, Eccles MP, Walker AE, Thomas RE: Changing physicians' behavior: what works and thoughts on getting more things to work. J Contin Educ Health Prof. 2002, 22: 237-243. 10.1002/chp.1340220408.View ArticlePubMedGoogle Scholar
- Lavis JN, Lomas J, Hamid M, Sewankambo NK: Assessing country-level efforts to link research to action. Bulletin of the World Health Organization. 2006, 84: 620-628. 10.2471/BLT.06.030312.View ArticlePubMedPubMed CentralGoogle Scholar
- Meso P, Smith R: A resource-based view of organizational knowledge management systems. Journal of Knowledge Management. 2000, 4: 224-234. 10.1108/13673270010350020.View ArticleGoogle Scholar
- Innvaer S, Vist G, Trommald M, Oxman AD: Health policy-makers' perceptions of their use of evidence: A systematic review. Journal of Health Services Research and Policy. 2002, 7: 239-244. 10.1258/135581902320432778.View ArticlePubMedGoogle Scholar
- Lavis JN, Davies HTO, Oxman AD, Denis J-L, Golden-Biddle K, Ferlie E: Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research and Policy. 2005, 10: S1:35-S1:48.View ArticleGoogle Scholar
- Lomas J: The in-between world of knowledge brokering. BMJ. 2007, 334: 129-132. 10.1136/bmj.39038.593380.AE.View ArticlePubMedPubMed CentralGoogle Scholar
- Lavis JN: Research, public policymaking, and knowledge-translation processes: Canadian efforts to build bridges. The Journal of Continuing Education in the Health Professions. 2006, 26: 37-45. 10.1002/chp.49.View ArticlePubMedGoogle Scholar
- Morse J, Field P: Qualitative Research Methods for Health Professionals. 1995, Thousand Oaks, SageGoogle Scholar
- Marshall C, Rossman G: Designing Qualitative Research. 1989, Newbury Park: SageGoogle Scholar
- Armstrong R, Waters E, Crockett B, Keleher H: The nature of evidence resources and knowledge translation for health promotion practitioners. HEALTH PROMOT INT. 2007, 22: 254-260. 10.1093/heapro/dam017.View ArticlePubMedGoogle Scholar
- Baklien B: The use of social science in a Norwegian Ministry: As a tool of policy or mode of thinking?. Acta Sociologica. 1983, 26: 33-47. 10.1177/000169938302600103.View ArticleGoogle Scholar
- Madden L, King L, Shiell A: How do government health departments in Australia access health economics advice to inform decisions for health? A survey. Australia and New Zealand Health Policy. 2009, 6: 6-10.1186/1743-8462-6-6.View ArticlePubMedPubMed CentralGoogle Scholar
- Campbell DM, Redman S, Jorm S, Cooke M, Zwi AB, Rychetnik L: Increasing the use of evidence in health policy: practice and views of policy makers and researchers. Australia and New Zealand Health Policy. 2009, 6: 21-10.1186/1743-8462-6-21.View ArticlePubMedPubMed CentralGoogle Scholar
- Coburn AF: The role of health services research in developing state health policy. HEALTH AFF. 1998, 17: 139-151. 10.1377/hlthaff.17.1.139.View ArticleGoogle Scholar
- Jewell CJ, Bero LA: "Developing good taste in evidence": Facilitators of and hindrances to evidence-informed health policymaking in state government. The Milbank Quarterly. 2008, 86: 177-208. 10.1111/j.1468-0009.2008.00519.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Lavis JN, Oxman AD, Moynihan R, Paulsen E: Evidence-informed health policy: 1. Synthesis of findings from a multi-method study of organizations that support the use of research evidence. Implementation Science. 2008,Google Scholar
- Dobbins M, Hanna S, Ciliska D, Manske S, Cameron R, Mercer S: A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implementation Science. 2009, 4:Google Scholar
- Wilkinson D, Michie S, McCarthy M: The use and perceptions of routine health data: a qualitative study of four cancer network teams in England. Health Services Management Research. 2007, 20: 211-219. 10.1258/095148407782219058.View ArticlePubMedGoogle Scholar
- Hoffmann C, Stoykova BA, Nixon J, Glanville JM, Misso K, Drummond MF: Do health-care decision makers find economic evaluations useful? The findings of focus group research in UK health authorities. Value Health. 2002, 5: 71-78. 10.1046/j.1524-4733.2002.52109.x.View ArticlePubMedGoogle Scholar
- Iglesias CP, Drummond MF, Rovira J: Health-care decision-making processes in Latin America: problems and prospects for the use of economic evaluation. International Journal of Technology Assessment in Health Care. 2005, 21: 1-14.View ArticlePubMedGoogle Scholar
- Lavis J, Oxman A, Moynihan R, Paulsen E: Evidence-informed health policy 3 - Interviews with the directors of organizations that support the use of research evidence. Implementation Science. 2008, 3:Google Scholar
- Hoffmann C, von der Schulenburg J-MG: The influence of economic evaluation studies on decision making. A European survey. Health Policy. 2000, 52: 179-192. 10.1016/S0168-8510(00)00076-2.View ArticlePubMedGoogle Scholar
- Lavis J, Paulsen E, Oxman A, Moynihan R: Evidence-informed health policy 2 - Survey of organizations that support the use of research evidence. Implementation Science. 2008, 3:Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.