Assessing availability of scientific journals, databases, and health library services in Canadian health ministries: a cross-sectional study
Implementation Science volume 8, Article number: 34 (2013)
Evidence-informed health policymaking logically depends on timely access to research evidence. To our knowledge, despite the substantial political and societal pressure to enhance the use of the best available research evidence in public health policy and program decision making, there is no study addressing availability of peer-reviewed research in Canadian health ministries.
To assess availability of (1) a purposive sample of high-ranking scientific journals, (2) bibliographic databases, and (3) health library services in the fourteen Canadian health ministries.
From May to October 2011, we conducted a cross-sectional survey among librarians employed by Canadian health ministries to collect information relative to availability of scientific journals, bibliographic databases, and health library services. Availability of scientific journals in each ministry was determined using a sample of 48 journals selected from the 2009 Journal Citation Reports (Sciences and Social Sciences Editions). Selection criteria were: relevance for health policy based on scope note information about subject categories and journal popularity based on impact factors.
We found that the majority of Canadian health ministries did not have subscription access to key journals and relied heavily on interlibrary loans. Overall, based on a sample of high-ranking scientific journals, availability of journals through interlibrary loans, online and print-only subscriptions was estimated at 63%, 28% and 3%, respectively. Health Canada had a 2.3-fold higher number of journal subscriptions than that of the provincial ministries’ average. Most of the organisations provided access to numerous discipline-specific and multidisciplinary databases. Many organisations provided access to the library resources described through library partnerships or consortia. No professionally led health library environment was found in four out of fourteen Canadian health ministries (i.e. Manitoba Health, Northwest Territories Department of Health and Social Services, Nunavut Department of Health and Social Services and Yukon Department of Health and Social Services).
There is inequity in availability of peer-reviewed research in the fourteen Canadian health ministries. This inequity could present a problem, as each province and territory is responsible for formulating and implementing evidence-informed health policies and services for the benefit of its population.
In Canada, the provincial and territorial health ministries are responsible for developing legislation, regulations, policies and directives to support strategic directions, and in some cases, operational decision-making for their health care systems (see  for more information about Canada’s health care system). Cohesion within this decentralised health system is maintained by national standards through the Canada Health Act . Since the end of the 1990s, leaders of governments, such as in Canada [3, 4], the United States, the United Kingdom and Australia, have emphasised the importance of developing an evidence-informed health system, with decisions made on the basis of appropriate, balanced and high-quality evidence.
Evidence-informed health policymaking is described as an approach to policy decisions that aims to ensure that decision making is well-informed by the best available research evidence among other factors (e.g. institutional constraints, interest group pressures, and citizen values) . This approach was formally adapted to public health by scholars and public health practitioners about 14 years ago on the principles of evidence-based medicine [6, 7]. Public health surveillance, evidence-based guidelines, review-derived products (e.g. summaries and overviews of systematic reviews, policy briefs based on systematic reviews), primary research studies and the gray literature are all relevant sources of information for public health and health system policy decision makers . Public health and health promotion decision makers generally support the development of a readily accessible and easy-to-use source of research information [9–12]. However, developing information resources for health policy is a challenging process because of the range of decision-making settings and the tremendous diversity in the nature of information needed in the field of medicine, epidemiology, nursing, sociology, political science, administration, economics, law, statistics, public administration, engineering, and other disciplines .
Over the past ten years, studies have been conducted around the world on the relation between research evidence and policymaking to identify some of the key issues and potential solutions to promote the development and application of evidence in health care decision making [8, 11, 14–17]. Public health decision makers strongly indicated a need to receive research evidence that is specific to their decision-making environments and appears user-friendly in a concise format [18–20]. Several initiatives to retrieve research evidence, such as the development of web-based resources (e.g. Medline Plus, MORE EBN, Health Systems Evidence, Health-evidence.ca), have tended to focus on making information available and accessible (in a timely fashion) to decision makers in order to increase their capacity to use research [8, 21–24]. Moreover, intense efforts have been expended on developing knowledge management tools and services in order to enhance the use of systematic reviews (SRs) by health policy makers and managers [25–28]. Over the past years, researchers and decision makers participated in knowledge transfer and exchange activities to increase the use of research in health policy. However, recent reviews of the literature showed that few studies pertain to the implementation and evaluation of the effectiveness of these activities [15, 16].
Several factors contribute to the limited application of research evidence within program decision making and policy development [29–32]. Major barriers to evidence-informed decision making include lack of time, limited availability and access to research evidence, and limited user capacity [18, 31, 33–35]. A recent systematic review of the literature showed that little information is available on the extent to which research evidence is used in public health decision-making processes. Although a range of different types of research evidence was accessed, the authors of this review found that research evidence impact on policy was often indirect and competed with other influences .
Physical access to research evidence is a prerequisite of research utilisation by policy makers [11, 17, 18]. For the past four years, the Canadian Virtual Health Library has worked towards the pursuit of equal access to health information for all Canadian health professionals, including public health workers and health care system administrators . However, Canada still lacks a national health information service that promotes the convergence of initiatives and resources across health system organisations . National online library initiatives that enhance equal access to quality-assured health information are found in many countries, such as the United States (the National Network of Libraries of Medicine ), the United Kingdom (NHS Evidence  and the Knowledge Network ), Norway (the Electronic Health Library ), Iceland (the National and University Library ) and Australia (the National Library of Australia ).
Among the requirements for research evidence to further inform decision making, public health decision makers indicate that i) “evidence should arise from sources which are seen as unbiased (such as peer-reviewed research), authoritative and credible; and provide methodological details so rigour can be assessed” and ii) “research evidence should be made more widely available to decision makers through the use of email bulletins, public health professional organisations or clearinghouses” . To our knowledge, despite the substantial political and societal pressure to enhance the use of the best available research evidence in public health policy and program decision-making, there is no study addressing availability of peer-reviewed research in Canadian health ministries. The aim of this study was three-fold: to assess availability of (1) a purposive sample of high-ranking scientific journals, (2) bibliographic databases, and (3) health library services in the fourteen Canadian health ministries.
The Laval University Institutional Review Board (IRB) established that this research project did not fall within the second edition of the “Tri-Council Policy Statement. For that reason, the Laval University IRB did not require further assessment and approval. However, proper arrangements were made to assure the participants’ (informants’) anonymity in analysis, presentation and publication of the data.
Study design, setting and participants
We used a cross-sectional survey to collect information relating to availability of a purposive sample of scholarly journals, bibliographic databases, and health library services in the following fourteen Canadian health ministries: Health Canada, British Columbia Ministry of Health Services, Alberta Health and Wellness, Saskatchewan Ministry of Health, Manitoba Health, Ontario Ministry of Health and Long-Term Care, Quebec Ministry of Health and Social Services, New Brunswick Department of Health, Nova Scotia Department of Health and Wellness, Prince Edward Island Department of Health and Wellness, Newfoundland and Labrador Department of Health and Community Services, Yukon Department of Health and Social Services, Northwest Territories Department of Health and Social Services, and Nunavut Department of Health and Social Services. Here, assessment of availability of journals, databases and library services in these organisations included the access provided through library partnerships or consortia.
Informants were: one library employee (e.g. manager, librarian, technician or records clerk) from each ministry, except for the territorial health ministries because these organisations had no department library at the time of this study. Instead, the most relevant resource persons available were two legislative librarians for the Northwest Territories Department of Health and Social Services and the Nunavut Department of Health and Social Services, and a senior policy advisor for the Yukon Department of Health and Social Services. The number of informants was purposely limited to one per organisation (n = 14), because the survey questionnaire aimed at collecting basic information at the organisational level, as opposed to the individual level (i.e., perceptions/views about health library resource and service). Also, some organisations had limited (e.g., Manitoba Health) or no (e.g., the territorial health ministries) staff assigned to health library services. The limited number of participants allowed timely response to the participants’ information request. Reminder e-mails were sent every two weeks until the survey closed.
The contact information of potential informants was obtained through the ministries’ websites (listed on the Health Canada website at ), by contacting their communication divisions or by other contacts, when necessary. Then, the informants were contacted by GL by email and informed about the project. On accepting to participate, the informants were invited to complete a survey (available as a fillable PDF form) and to send it back by email to GL. Participants were asked to take as much time as needed to answer the survey and, when necessary, to go look at the resources available and/or to contact their colleagues to verify the accuracy of their responses before sending their responses. When a health library infrastructure was not present, or when the contacted informant declined to participate, we asked for the contact information of the most relevant resource persons available until one participant was found per organisation. The only inclusion criterion was being able to answer basic organisational questions relative to the current availability of journals, databases and library services. If more than one informant per organisation was eligible, the first respondent that agreed to participate was chosen and the other(s) was (were) thanked. The survey was composed of four questions: “Q1: Does your department library have a print and/or an online subscription to the following journals?”, “Q2: Does your department library have an online catalogue? If yes, please provide the URL address”, “Q3: Does your department library have a partnership and/or is a member of a resource-sharing network? If yes, please provide a description of the network and its services”, and “Q4: Does your department library subscribe to any health or science databases? If yes, please list them all”. The first question focuses on the type of access that a health ministry might provide for a purposive sample of scientific journals (see below for information about the method we used for selecting the journals). More specifically, we collected data for the following types of access: online subscription, print-only subscription, interlibrary loan, and ‘no access’ using one of the previous means.
For the first question, a list of scientific journals sorted by subject category and then by impact factor was determined in March 2011 using the 2009 Journal Citation Reports (JCR) published online by Thomson Reuters. Based on scope note information about subject categories, eight categories from the 2009 Science Edition (i.e. “Health Care Sciences & Services”, “Medical Ethics”, “Medical Informatics”, “Medicine, General & Internal”, “Medicine, Legal”, “Medicine, Research & Experimental”, “Nursing” and “Public, Environmental & Occupational Health”) and three categories from the 2009 Social Sciences Edition (i.e. “Health Policy & Services”, “Public, Environmental & Occupational Health” and “Social Sciences, Biomedical”) of the JCR were chosen because of their relevance for health policy. Then, in order to narrow our journal sample, we used the journal impact factors as an indicator of journal popularity and selected the five top-ranked journals for each journal category. Six journal duplicates found in more than one category were removed from the sample, whereas three Canadian journals (i.e. Canadian Medical Association Journal, Canadian Journal of Public Health, and Healthcare Policy) and one of the most exclusive journals of systematic literature reviews for health care decision making (i.e. Cochrane Database of Systematic Reviews) were added to the sample. Healthcare Policy was the only journal in our sample that was not evaluated in the 2009 JCR. Finally, journals that publish open access (OA) articles (i.e. direct OA, hybrid OA and delayed OA) were identified. Table 1 shows the list of 53 scientific journals that were selected and included in the survey. The availability indicator taken into account in this study is the number of journals (from our sample) that are accessible using one of the following types of access: 1) online subscription (with or without an additional print subscription), 2) print-only subscription, 3) interlibrary loan, and 4) ‘no access’. The detailed results of availability by journal and ministry are presented in Additional file 1.
Data collected in the survey were used to generate five tables of data that describe availability of scientific journals (Tables 2 and 3, and Additional file 1), bibliographic databases (Table 4) and health library services (Table 5) in the provincial, territorial and federal health ministries in Canada.
Data collected for the five OA journals of our sample, (i.e. Health Technology Assessment, Journal of Medical Internet Research, British Medical Journal, WHO Technical Report Series, and Environmental Health Perspectives) were excluded from the analysis in Tables 2 and 3, because they provided online access to all research articles without the need of a subscription, which resulted in unexpected data (i.e. ‘no provision’ or ‘interlibrary loan’) for some ministries.
Availability of scientific journals
Our first objective was to assess availability of the current issue of a purposive sample of high impact factor journals for Canadian health ministries. We found that the majority of Canadian health ministries did not have subscription access to key journals and relied heavily on interlibrary loans (Table 2). Overall, based on a sample of high-ranking scientific journals (n = 48), availability of journals through interlibrary loans, online and print-only subscriptions was estimated at 63%, 28% and 3%, respectively.
The provincial health ministries provided access to our journal sample primarily by using interlibrary loans (30 ± 13 out of 48 journals; mean for 10 ministries ± standard deviation). Seven provincial health ministries had a similar or higher number of journal subscriptions than that of the provincial ministries average (15.0 ± 7.8 out of 48 journals), whereas three provincial health ministries (i.e. Manitoba Health, New Brunswick Department of Health and Prince Edward Island Department of Health and Wellness) had a ≥2-fold lower number of journal subscriptions than that of the provincial ministries average. For unknown reasons that were not investigated in this study, Manitoba Health was the only provincial ministry that had no department library and relied exclusively on interlibrary loans to provide access to scientific journals, while the Ontario Ministry of Health and Long-Term Care had no interlibrary loan services.
The territorial health ministries of the Northwest Territories and Nunavut had few journal subscriptions (i.e. ≤4, online and print-only subscriptions combined) and relied almost exclusively on interlibrary loan services to provide access to scientific journals. No data were collected for the Yukon Department of Health and Social Services, because of the absence of a department library or a central place that coordinates journal subscriptions. The federal health ministry, Health Canada, used exclusively online subscriptions and provided access to most of our journal sample (34 out of 48 journals). Health Canada had no interlibrary loan services.
To determine whether the provincial health ministries have more subscriptions in one category of journals over another, we analysed the combined results of online and print-only subscriptions by subject category (Table 3). Overall, the number of provincial ministries with a journal subscription by subject category was 3.0 ± 0.8 out of 10 ministries (mean for eleven categories, three to five journals per category ± standard deviation). Four subject categories in the JCR Science Edition (i.e. “Health Care Sciences & Services”, “Medical Ethics”, “Medicine, General & Internal” and “Public, Environmental & Occupational Health”) and one subject category in the JCR Social Science Edition (i.e. “Public, Environmental & Occupational Health”) were above the provincial average. Journals from the “Medicine, General & Internal” subject category had the highest number of provincial ministries with a journal subscription (7.5 ± 0.6 out of 10 ministries), whereas journals from the “Medicine, Legal” subject category had the lowest number of ministries with a journal subscription (0.2 ± 0.4 out of 10 provincial ministries).
To determine whether Health Canada and the provincial ministries have similar subscription patterns for our sample of journals, we analysed the combined number of online and print-only subscriptions by subject category (Table 3). Overall, the number of journal subscriptions by subject category in Health Canada was 2.3-fold higher than that of the provincial ministries average (3.0 ± 1.5 and 1.3 ± 0.4 journals, respectively. Mean for eleven categories, three to five journals per category ± standard deviation). High number of journal subscriptions in Health Canada (i.e. >2.3-fold that of the provincial ministries average) was found for the “Medical Informatics”, “Medicine, Legal”, “Medicine, Research & Experimental”, “Public, Environmental & Occupational Health” (i.e. JCR Science Edition) and “Social Sciences, Biomedical” subject categories. For the federal and provincial ministries, the number of journal subscriptions was highest for the “Medicine, General & Internal” subject category (≥3.0 journals out of 4) and lowest for “Nursing” subject category (<1 journal out of 5).
Availability of bibliographic databases
Our second objective was to assess availability of bibliographic databases for Canadian health ministries. Indeed, to conduct searches on a regular basis, health ministry staff need full access to bibliographic databases. Free databases can be accessed on the Internet (e.g. MEDLINE, DARE or HSE), however the major discipline-specific (e.g. CINAHL, EMBASE, UpToDate, Cochrane Library or PsycINFO) and multidisciplinary (e.g. Academic Search or Scopus) databases require a paid subscription for full access. Table 4 shows that the federal and provincial health ministries provided access to numerous discipline-specific and multidisciplinary databases, except for four ministries (i.e. Prince Edward Island Department of Health and Wellness, Manitoba Health, New Brunswick Department of Health and Quebec Ministry of Health and Social Services). These latter ministries provided limited or no access to any databases that required a paid subscription. However, Manitoba Health had access to health research support provided by Manitoba’s Health Information and Knowledge Network. The Northwest Territories and Nunavut health ministries did not provide access to databases that required a paid subscription. However, we found that employees of the Nunavut Department of Health and Social Services had access to health research support provided by the Neil John McLean Health Sciences Library (University of Manitoba). No data were available for the Yukon Department of Health and Social Services.
Health library services
Our last objective was to assess health library services available in the Canadian health ministries. A search of the organisation websites followed by informal email and/or telephonic conversations with government library employees revealed that most (9 out of 10) provincial health ministries had a professionally led library environment, which provides reference services and client support, within their organisation infrastructures (Table 5). Manitoba Health relied exclusively on external health library services. A virtual-only library environment was found within the Ontario Ministry of Health and Long-Term Care and the Saskatchewan Ministry of Health infrastructures. Table 5 shows that provincial health ministries with a department library infrastructure also had access to external health library services available through various library partnerships or consortia, except for the Ontario Ministry of Health and Long-Term Care.
The three territorial health ministries had no department library. Access to scientific journals in the Northwest Territories and the Nunavut health ministries was provided by the legislative library. In the Yukon Department of Health and Social Services, we were not able to collect data concerning availability of peer-reviewed research, because there was no central place that coordinates the journal subscriptions at the time of the study. It is however possible that individual branches in this ministry managed a subscription for a particular journal.
Finally, as for most provincial health ministries, access to a professionally led library environment was found within Health Canada. However, the following library services were provided by the National Research Council, Canada Institute for Scientific and Technical Information (NRC-CISTI): licensing and acquisitions, catalogue, library web site, help desk, cataloguing, and information delivery.
Summary of research findings
In this study, we investigated availability of scientific journals, bibliographic databases and health library services in the fourteen Canadian health ministries. On the one hand, we found several similarities in availability of peer-reviewed research among the ministries. Firstly, using a sample of 48 scientific journals for analysis, we found that interlibrary loans were primarily used by most health ministries to provide access to scientific journals when compared to online and print-only subscriptions (i.e. 63%, 28% and 3%, respectively). Moreover, online subscriptions (with or without an additional print subscription) were primarily used to provide access to scientific journals when compared to print-only subscriptions (except for the Northwest Territories Department of Health and Social Services; no data were available for the Yukon Department of Health and Social Services). Secondly, most of the organisations provided access to numerous discipline-specific and multidisciplinary databases (except the Prince Edward Island, Manitoba, New Brunswick and Quebec health ministries). Thirdly, a professionally led health library environment was generally found on site (except for the Manitoba, Northwest Territories, Nunavut and Yukon health ministries). Lastly, external library services were generally available together with the department library service through various partnerships or consortia (except for the Ontario Ministry of Health and Long-Term Care).
On the other hand, we found several differences in availability of peer-reviewed research among the ministries. Firstly, five out of the fourteen Canadian health ministries (i.e. the Manitoba, New Brunswick, Prince Edward Island, Northwest Territories and Nunavut health ministries) had a ≥2-fold lower number of journal subscriptions than that of the provincial ministries’ average (15.0 ± 7.8). Secondly, among the provincial ministries, the health ministries in the Maritimes (i.e. New Brunswick, Nova Scotia and Prince Edward Island) had the lowest number of journal subscriptions. Thirdly, Health Canada had a 2.3-fold higher number of journal subscriptions than that of the provincial ministries average in a context where responsibilities for the formulation and implementation of most health care policies and services are almost exclusively in the hands of the provinces and territories. Lastly, for unknown reasons that were not investigated in this study, Health Canada and the Ontario Ministry of Health and Long-Term Care are the only organisations that had no interlibrary loan services (no data were available for the Yukon Department of Health and Social Services).
Availability of peer-reviewed research evidence in Canadian health ministries
Overall, these data suggest that immediate access (i.e. availability of the current issue through online or print-only subscription) to popular scientific journals in the field of health is limited for six out of fourteen health ministries in Canada (i.e. the Manitoba, New Brunswick, Prince Edward Island, Northwest Territories, Nunavut and Yukon health ministries) when compared to the average number of journal subscriptions available for the provincial health ministries. Therefore, we suggest that the current availability of peer-reviewed research is not optimal and potentially a barrier to health-system policy makers’ use of evidence in several Canadian health ministries. Also, these findings suggest that the use of interlibrary loan as the primary type of provision of scientific journals could be a direct consequence of budget constraints and/or local use. It is therefore not surprising that several health ministries are members of library consortia (Table 5). More research is needed to determine whether lack of subscriptions to the high-ranked journals tested in this study is mediated by provision of lower-ranked journals based on cost-effectiveness analysis (i.e. analyses that include local access, usage - locally collected citation score - and subscription rates). Also, plausible explanations for the discrepancies observed between these organisations may be provided by the assessment of the current supports that Canadian health ministries have to facilitate evidence-informed decision making (e.g. funding, programs, interventions, tools, etc.).
Timely access to high quality research evidence is of the essence in evidence-informed health policymaking [29, 45]. Previous studies have shown that policymakers and stakeholders need systematic reviews (SRs) of research evidence to inform policymaking processes [10, 11, 46]. Here, 33 out of 48 journals (69%) included in data analysis contain SRs in current and/or past issues (Table 1). Journals that had a high number of subscriptions in the provincial and federal health ministries (i.e. ≥8 out of 11 ministries) were high impact factor and/or Canadian journals that address health policy issues, among other topics. Most notable were the journals from the “Medicine, General & Internal” subject category (i.e. New England Journal of Medicine, Lancet, Journal of the American Medical Association, Annals of Internal Medicine, Canadian Medical Association Journal and Cochrane Database of Systematic Reviews). Moreover, using data gathered from Health Systems Evidence (an online repository of syntheses of research evidence about governance, financial and delivery arrangements within health systems, and about implementation strategies that can support change in health systems) , we found that Cochrane Database of Systematic Reviews, Journal of the American Medical Association, Annals of Internal Medicine, Lancet and Canadian Medical Association Journal were among the top 50 journals that frequently contain SRs of effects or of other questions (i.e. 412, 19, 11, 10 and 5 SRs, respectively) (data not shown). Other journals that match up with our journal sample were also found among the top 50 journals, but had either a low number of subscriptions in the provincial and federal health ministries (i.e., ≤4 out of 11 ministries, for International Journal of Nursing Studies - 22 SRs, Medical Care - 13 SRs, Journal of the American Medical Informatics Association - 8 SRs, Social Science & Medicine - 8 SRs, and Psychiatric Services - 7 SRs) or were OA (British Medical Journal - 40 SRs, and Health Technology Assessment - 22 SRs). These findings suggest that, although not optimal, the provincial and federal health ministries provide access to high-quality SRs to inform the policy-making process.
The lack of journal subscriptions by the Canadian health ministries could be mediated by OA journals. Since 2008, Canadian Institutes of Health Research (CIHR) grant recipients have been strongly required to give access to their research outputs in an OA journal and/or in an open Web repository . The importance of OA has also been recognised by other research funding agencies, including the European Research Council , the United Kingdom Medical Research Council , the United States National Institutes of Health , and The Wellcome Trust . However, many journal publishers do not offer OA options for authors yet, or allow only OA to certain articles in an issue for advertising or promotional matters. Indeed, the share of direct (i.e. full) OA articles of all peer reviewed articles indexed by Ulrich’s, Scopus, and Web of Science (ISI) bibliographic databases in 2009 was estimated at 7.7%, 6.8% and 5.9%, respectively . In a recent study, it was shown that the overall access to all scholarly articles through OA publishers and open web repositories was estimated at 8.5% and 11.9%, respectively . Therefore, OA peer-reviewed research remain limited and unlikely to meet the needs of Canadian health ministries within the near future.
Analysis of journal availability in the provincial health ministries showed that online subscription (with or without an additional print subscription) was primarily used to provide access when compared to print-only subscription. A decrease of print journal use with the introduction of online journals was shown in health science libraries during the past decade , as the cost per use of print journals can be five times that of online articles accessible by site licenses . Online subscriptions also provide desk access to literature that is likely to be more accessible than in paper-based journals. Although budget constraints for licensing contents and the local use of scientific journals are good arguments in favour of interlibrary loan use in remote locations (such as the territorial health ministries), the loss of productivity resulting from excessive time spent in pursuit of information may be significant for the provincial health ministries. Another impact associated with interlibrary loans use might be the delay for the policymakers requesting the evidence, with the potential for decisions to be made without the available evidence. Extensive cost-benefit analysis would ultimately show why interlibrary loan should be used primarily to provide access when compared to online subscription in the provincial health ministries. Few quantitative studies on the actual use of research evidence by Canadian public health decision makers are available [19, 56–58]. The higher (i.e. 2.3-fold) number of journal subscriptions for Health Canada than that of the provincial ministries average could be due to the multi-million dollar initiative of the Government of Canada to provide improved access to information at the desktop for Canadian federal scientists, policy analysts and decision makers [59, 60]. To date, all researchers, policy analysts and decision makers in the departments of the Federal Science eLibrary cluster (which includes Health Canada) have access to the entire e-journal collection of Springer, Canadian Science Publishing and the Web Editions collection of the American Chemical Society. Alternatively, adoption of a country-wide approach to provide access to health evidence, such as the Canadian Virtual Health Library , would optimise resource expenditures across Canadian health system organisations.
To the best of our knowledge, this is the first study to systematically address availability of peer-reviewed research in the Canadian health ministries. However, this study has multiple limitations that should be made explicit. Firstly, the cross-sectional nature of the data means that it was not possible to track changes longitudinally. Secondly, collected data are self-reported. Thirdly, our data may not reflect the actual scientific journal availability for all ministry employees, as some of them are pursuing their education and may have privileged access to scientific journals through their learning institutions. Also, availability by pay-per-view was not considered in our study. Fourthly, Thomson Reuters journal impact factors does not reflect a journal’s quality , or relevance and applicability to decision-making. Lastly, this study did not require participants to systematically validate their access to any research articles in the current issue of a journal using their Internet access.
This study provides new insight into availability of scientific journals, databases and library services in the fourteen Canadian health ministries. We found that the majority of ministries did not have subscription access to key journals and relied heavily on interlibrary loans. Study findings have implications for researchers, decision makers and library managers who might wish to improve the availability of peer-reviewed journals within Canadian health ministries, notably those publishing systematic reviews. More research is needed to describe the current access and use of research evidence by policy analysts and decision makers in the Canadian health ministries. A next step could include investigating the correlation between measures of research use and availability of scientific journals, the cost-effectiveness of a different business model for ministerial libraries, librarians’ views about their role (e.g., knowledge brokers and/or custodians of journal collections), and the barriers to availability of scientific journals and databases in health ministries.
Health Canada, Canada’s Health Care System.http://www.hc-sc.gc.ca/hcs-sss/pubs/system-regime/2011-hcs-sss/index-eng.php,
Department of Justice Canada, Canada Health Act, Bill C-3, Statutes of Canada, 32–33 Elizabeth II (RSC 1985, c 6; RSC 1989, c C-6). 1984,http://laws-lois.justice.gc.ca/eng/acts/C-6/index.html,
Health Canada, National Forum on Health, Canada Health Action: Building on the Legacy, Vol. I: Final Report. 1997,http://www.hc-sc.gc.ca/hcs-sss/com/fed/nfh-fns-eng.php,
Health Canada, Advisory Council on Health Infostructure, Canada Health Infoway: Paths to Better Health Final Report. 1999,http://www.hc-sc.gc.ca/hcs-sss/pubs/ehealth-esante/1999-paths-voies-fin/index-eng.php,
Oxman AD, Lavis JN, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking?. Health Res Policy Syst. 2009, 7 (Suppl 1): S1-10.1186/1478-4505-7-S1-S1.
Brownson RC, Gurney JG, Land GH: Evidence-based decision making in public health. J Public Health Manag Pract. 1999, 5: 86-97.
Glasziou P, Longbottom H: Evidence-based public health practice. Aust Nz J Publ Heal. 1999, 23: 436-440. 10.1111/j.1467-842X.1999.tb01291.x.
Chambers D, Wilson PM, Thompson CA, Hanbury A, Farley K, Light K: Maximizing the impact of systematic reviews in health care decision making: a systematic scoping review of knowledge-translation resources. Milbank Q. 2011, 89: 131-156. 10.1111/j.1468-0009.2011.00622.x.
Dobbins M, DeCorby K, Twiddy T: A knowledge transfer strategy for public health decision makers. Worldviews Evid Based Nurs. 2004, 1: 120-128. 10.1111/j.1741-6787.2004.t01-1-04009.x.
Dobbins M, Jack S, Thomas H, Kothari A: Public health decision-makers’ informational needs and preferences for receiving research evidence. Worldviews Evid Based Nurs. 2007, 4: 156-163. 10.1111/j.1741-6787.2007.00089.x.
Lavis J, Davies H, Oxman A, Denis J-L, Golden-Biddle K, Ferlie E: Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy. 2005, 10 (Suppl 1): 35-48.
Whitehead M, Petticrew M, Graham H, Macintyre SJ, Bambra C, Egan M: Evidence for public health policy on inequalities: 2: assembling the evidence jigsaw. J Epidemiol Community Health. 2004, 58: 817-821. 10.1136/jech.2003.015297.
Kemm J: The limitations of ‘evidence-based’ public health. J Eval Clin Pract. 2006, 12: 319-324. 10.1111/j.1365-2753.2006.00600.x.
Brownson RC, Fielding JE, Maylahn CM: Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009, 30: 175-201. 10.1146/annurev.publhealth.031308.100134.
Moore G, Redman S, Haines M, Todd A: What works to increase the use of research in population health policy and programmes: a review. Evid Policy: J Res Debate Practice. 2011, 7: 277-305. 10.1332/174426411X579199.
Mitton C, Adair CE, McKenzie E, Patten SB, Waye Perry B: Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q. 2007, 85: 729-768. 10.1111/j.1468-0009.2007.00506.x.
Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002, 7: 239-244. 10.1258/135581902320432778.
Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S: The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011, 6: e21704-10.1371/journal.pone.0021704.
Dobbins M, Thomas H, O’Brien MA, Duggan M: Use of systematic reviews in the development of new provincial public health policies in Ontario. Int J Technol Assess Health Care. 2004, 20: 399-404.
Golden-Biddle K, Reay T, Petz S, Witt C, Casebeer A, Pablo A, Hinings CRB: Toward a communicative perspective of collaborating in research: the case of the researcher-decision-maker partnership. J Health Serv Res Policy. 2003, 8 (Suppl 2): 20-25.
Bowen S, Zwi AB: Pathways to “evidence-informed” policy and practice: a framework for action. PLoS Med. 2005, 2: e166-10.1371/journal.pmed.0020166.
Dobbins M, De Corby K: Health-evidence.ca: an innovative knowledge source for health services. Worldviews Evid Based Nurs. 2004, 1: 76-10.1111/j.1741-6787.2004.04014.x.
Tso P, Culyer AJ, Brouwers M, Dobrow MJ: Developing a decision aid to guide public sector health policy decisions: a study protocol. Implement Sci. 2011, 6: 46-10.1186/1748-5908-6-46.
Waters E, Armstrong R, Swinburn B, Moore L, Dobbins M, Anderson L, Petticrew M, Clark R, Conning R, Moodie M, Carter R: An exploratory cluster randomised controlled trial of knowledge translation strategies to support evidence-informed decision-making in local governments (The KT4LG study). BMC Public Health. 2011, 11: 34-10.1186/1471-2458-11-34.
McMaster University, Health systems evidence.http://www.healthsystemsevidence.org,
Cahn MA, Auston I, Selden CR, Cogdill K, Baker S, Cavanaugh D, Elliott S, Foster AJ, Leep CJ, Perez DJ, Pomietto BR: The partners in information access for the public health workforce: a collaboration to improve and protect the public’s health, 1995–2006. J Med Libr Assoc. 2007, 95: 301-309. 10.3163/1536-5050.95.3.301.
Dobbins M, DeCorby K, Robeson P, Husson H, Tirilis D, Greco L: A knowledge management tool for public health: health-evidence.ca. BMC Public Health. 2010, 10: 496-10.1186/1471-2458-10-496.
Weir MC, Ryan R, Mayhew A, Worswick J, Santesso N, Lowe D, Leslie B, Stevens A, Hill S, Grimshaw JM: The Rx for Change database: a first-in-class tool for optimal prescribing and medicines use. Implement Sci. 2010, 5: 89-10.1186/1748-5908-5-89.
Bowen S, Erickson T, Martens PJ, Crocket S: More than ‘Using Research’: the real challenges in promoting evidence-informed decision-making. Healthc Policy. 2009, 4: 87-102.
Brownson RC, Royer C, Ewing R, McBride TD: Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006, 30: 164-172. 10.1016/j.amepre.2005.10.004.
Ciliska D, Hayward S, Dobbins M, Brunton G, Underwood J: Transferring public-health nursing research to health-system planning: assessing the relevance and accessibility of systematic reviews. Can J Nurs Res. 1999, 31: 23-36.
Paluck EC, Williamson DL, Milligan CD, Frankish CJ: The use of population health and health promotion research by health regions in Canada. Can J Public Health-Revue Canadienne De Sante Publique. 2001, 92: 19-23.
Dobbins M, Rosenbaum P, Plews N, Law M, Fysh A: Information transfer: what do decision makers want and need from researchers?. Implement Sci. 2007, 2: 20-10.1186/1748-5908-2-20.
LaPelle NR, Luckmann R, Simpson EH, Martin ER: Identifying strategies to improve access to credible and relevant information for public health professionals: a qualitative study. BMC Public Health. 2006, 6: 89-10.1186/1471-2458-6-89.
Revere D, Turner AM, Madhavan A, Rambo N, Bugni PF, Kimball A, Fuller SS: Understanding the information needs of public health practitioners: a literature review to inform design of an interactive digital knowledge management system. J Biomed Inform. 2007, 40: 410-421. 10.1016/j.jbi.2006.12.008.
The Canadian Virtual Health Library.http://cvhl.ca/en,
Canada Health Library Association/Association des Bibliothèques de la Santé du Canada: Canadian Virtual Health Library - Environmental Scan. Canada Health Infoway Phase 0 Report. 2008,http://www.chla-absc.ca/nnlh/cvhl-enviro.pdf,
National Network of Libraries of Medicine.http://nnlm.gov/,
The Knowledge Network Scotland.http://www.knowledge.scot.nhs.uk/,
The Norwegian Electronic Health Library.http://www.Helsebiblioteket.no,
National and University Library of Iceland.http://www.landsbokasafn.is/,
National Library of Australia.http://www.nla.gov.au/,
Health Canada, Provincial/Territorial Role in Health.http://www.hc-sc.gc.ca/hcs-sss/delivery-prestation/ptrole/index-eng.php,
Kiefer L, Frank J, Di Ruggiero E, Dobbins M, Manuel D, Gully PR, Mowat D: Fostering evidence-based decision-making in Canada: examining the need for a Canadian population and public health evidence centre and research network. Can J Public Health. 2005, 96: I1-I40. following 200
Lavis JN: How can we support the use of systematic reviews in policymaking?. PLoS Med. 2009, 6: e1000141-10.1371/journal.pmed.1000141.
Canadian Institutes of Health Research, Policy on Access to Research Outputs. 2007,http://www.cihr-irsc.gc.ca/e/46068.html,
European Research Council Scientific Council Statement on Open Access (December 2006).http://erc.europa.eu/node/1245,
Medical Research Council position statement in support of open and unrestricted access to published research (Last updated April 2008).http://www.mrc.ac.uk/Ourresearch/Ethicsresearchguidance/Openaccesspublishing/Positionstatement/index.htm,
The NIH Public Access Policy implements Division G, Title II, Section 218 of PL 110–161 (Consolidated Appropriations Act, 2008).http://publicaccess.nih.gov/policy.htm,
Wellcome Trust position statement in support of open and unrestricted access to published research. Open access policy (accessed May 2012).http://www.wellcome.ac.uk/About-us/Policy/Spotlight-issues/Open-access/Policy/index.htm,
Laakso M, Welling P, Bukvova H, Nyman L, Bjork BC, Hedlund T: The development of open access journal publishing from 1993 to 2009. PLoS One. 2011, 6: e20961-10.1371/journal.pone.0020961.
Bjork BC, Welling P, Laakso M, Majlender P, Hedlund T, Gudnason G: Open access to the scientific journal literature: situation 2009. PLoS One. 2010, 5: e11273-10.1371/journal.pone.0011273.
Burrows S: A review of electronic journal acquisition, management, and use in health sciences libraries. J Med Libr Assoc. 2006, 94: 67-74.
Obst O: Patterns and costs of printed and online journal usage. Health Info Libr J. 2003, 20: 22-32.
Ouimet M, Bedard PO, Turgeon J, Lavis JN, Gelineau F, Gagnon F, Dallaire C: Correlates of consulting research evidence among policy analysts in government ministries: a cross-sectional survey. Evid Policy. 2010, 6: 433-460. 10.1332/174426410X535846.
Dobbins M, Cockerill R, Barnsley J, Ciliska D: Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions. Int J Technol Assess Health Care. 2001, 17: 467-478.
Jetha N, Robinson K, Wilkerson T, Dubois N, Turgeon V, DesMeules M: Supporting knowledge into action: the Canadian Best Practices Initiative for Health Promotion and Chronic Disease Prevention. Can J Public Health. 2008, 99: I1-I8.
Government of Canada, Federal Science eLibrary.http://safstl-asbstf.scitech.gc.ca/eng/index.html,
Brown B, Found C, McConnell M: Federal Science eLibrary pilot - Seamless, equitable desktop access for Canadian government researchers. Electron Libr. 2007, 25: 8-17. 10.1108/02640470710729083.
Vanclay JK: Impact factor: outdated artefact or stepping-stone to journal certification?. Scientometrics. 2012, 92: 211-238. 10.1007/s11192-011-0561-0.
This project was funded through the KT Canada network, which in turn is funded by the Canadian Institutes of Health Research and Canadian Foundation for Innovation. MO receives financial support from the Fonds de la recherche en santé du Québec (FRSQ) as a research scholar. JG and MPG receive salary support from the Canada Research Chairs Program. MPG holds a New Investigator career grant from the Canadian Institutes of Health Research. The authors would like to thank all participants who dedicated some of their professional time to contributing to this study.
The authors declare that they have no competing interests.
GL and MO conceived the project and contributed equally to the design of the study, interpretation of the data and writing of the manuscript. All authors read, commented and approved the final manuscript.
Grégory Léon, Mathieu Ouimet contributed equally to this work.
Electronic supplementary material
Additional file 1: Availability of scientific journals (n = 53) for the provincial, territorial and federal health ministries in Canada. Summary of raw data from the survey (Question 1). (DOC 160 KB)
About this article
Cite this article
Léon, G., Ouimet, M., Lavis, J.N. et al. Assessing availability of scientific journals, databases, and health library services in Canadian health ministries: a cross-sectional study. Implementation Sci 8, 34 (2013). https://doi.org/10.1186/1748-5908-8-34