- Open Access
- Open Peer Review
A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014
Implementation Science volume 11, Article number: 1 (2016)
Policy has a tremendous potential to improve population health when informed by research evidence. Such evidence, however, typically plays a suboptimal role in policymaking processes. The field of policy dissemination and implementation research (policy D&I) exists to address this challenge. The purpose of this study was to: (1) determine the extent to which policy D&I was funded by the National Institutes of Health (NIH), (2) identify trends in NIH-funded policy D&I, and (3) describe characteristics of NIH-funded policy D&I projects.
The NIH Research Portfolio Online Reporting Tool was used to identify all projects funded through D&I-focused funding announcements. We screened for policy D&I projects by searching project title, abstract, and term fields for mentions of “policy,” “policies,” “law,” “legal,” “legislation,” “ordinance,” “statute,” “regulation,” “regulatory,” “code,” or “rule.” A project was classified as policy D&I if it explicitly proposed to conduct research about the content of a policy, the process through which it was developed, or outcomes it produced. A coding guide was iteratively developed, and all projects were independently coded by two researchers. ClinicalTrials.gov and PubMed were used to obtain additional project information and validate coding decisions. Descriptive statistics—stratified by funding mechanism, Institute, and project characteristics—were produced.
Between 2007 and 2014, 146 projects were funded through the D&I funding announcements, 12 (8.2 %) of which were policy D&I. Policy D&I funding totaled $16,177,250, equivalent to 10.5 % of all funding through the D&I funding announcements. The proportion of funding for policy D&I projects ranged from 14.6 % in 2007 to 8.0 % in 2012. Policy D&I projects were primarily focused on policy outcomes (66.7 %), implementation (41.7 %), state-level policies (41.7 %), and policies within the USA (83.3 %). Tobacco (33.3 %) and cancer (25.0 %) control were the primary topics of focus. Many projects combined survey (58.3 %) and interview (33.3 %) methods with analysis of archival data sources.
NIH has made an initial investment in policy D&I research, but the level of support has varied between Institutes. Policy D&I researchers have utilized a variety of designs, methods, and data sources to investigate the development processes, content, and outcomes of public and private policies.
Policy has a tremendous potential to improve population health [1, 2]. The extent to which policy produces such outcomes, however, often depends upon the degree to which it is informed by scientific knowledge and aligned with evidence-based interventions [3–5]. Unfortunately, research typically plays a minor role in policymaking processes [6, 7]. For decades, scholars have documented myriad barriers to translating research into policy—including political influences, economic constraints, scientific uncertainty, and institutional culture [8–20].
The field of health policy dissemination and implementation research (policy D&I) exists to address these challenges. Policy D&I is focused on generating knowledge to effectively spread research evidence among policymakers and integrate evidence-based interventions into policy designs . Policy D&I, known as policy “knowledge translation and exchange” outside of the USA, has evolved over the past 40 years. The 1970s and 1980s were marked by interest in how social science research was used in public policymaking and saw the development of key contributions to the field—such as Weiss’ typology of research evidence  and Caplan’s two communities theory of knowledge utilization . In the 1990s and 2000s, frameworks and instruments for policy D&I emerged [21–25] and the field was embraced within the discipline of public health [9, 26, 27]. Today, policy D&I is a priority of the World Health Organization [28, 29] and a major focus of journals such as Health Research Policy and Systems and Evidence & Policy.
The history of policy D&I has demonstrated that government funding is essential for a county to develop and maintain a robust policy D&I research infrastructure [22, 30]. Outside of the USA, governments have made sustained investments in policy D&I research and initiatives to promote the use of research evidence in health policymaking. Examples include the Canadian Institutes of Health Research’s integrated knowledge translation programs, the National Health and Medical Research Council of Australia’s Partnership Projects and Centres, and the UK National Institute for Health Research’s Collaboration for Leadership in Applied Health Research and Care. While national governments have invested in policy D&I abroad, policy D&I research has largely existed outside of the mainstream, government-funded health research enterprise in the USA . This is potentially changing, however, as the National Institutes of Health (NIH)—the largest government funder of health research in the USA—has recently invested in the broader field of D&I research and identified policy D&I as a priority .
In 2002, the National Institute of Mental Health issued a Program Announcement (PAR) entitled “Dissemination and Implementation Research in Mental Health” that highlighted the importance of developing effective strategies to disseminate research findings to the policymakers . In 2005, the PAR was re-issued under the broader name “Dissemination and Implementation Research in Health” (D&IRH) with more participating Institutes (e.g., the National Cancer Institute, National Institute on Drug Abuse) . In 2007, the NIH Center for Scientific Review created a permanent D&IRH study section to evaluate D&I proposals, re-affirming NIH’s investment in the field . In 2013, the “D&IRH” PAR was re-issued, this time with a greater emphasis on policy D&I research . For example, the PAR encouraged applications that “address[ed] the complexity of bridging research, policy and practice.”
NIH has expressed interest in policy D&I research, but the extent to which it has funded research in this area is unclear. Two recent studies reviewed NIH D&I funding related to specific topics; one study focused on nursing  and the other focused on cancer control . A similar review of policy D&I research is needed to assess growth of the field and identify funding gaps. Such a study would support growth in the field of policy D&I in the USA by providing investigators with a menu of approaches to policy D&I research that can be adopted and adapted when developing proposals. This article reports the results of a study that was conducted to address these knowledge gaps. The aims of the study were to: (1) determine the extent to which policy D&I research has been funded by NIH between 2007 and 2014, (2) identify trends in NIH-funded policy D&I research, and (3) describe the characteristics of NIH-funded policy D&I research projects.
We used the NIH Research Portfolio Online Reporting Tool  to identify all projects that received funding during any US fiscal year between 2007 and 2014 through the D&I funding opportunity announcements (FOAs) Dissemination and Implementation Research in Health (PARs: 06-039, 06-071, 06-072, 06-520, 06-521, 07-086, 10-038, 10-039, 10-040, 13-054, 13-055, 13-056) or Dissemination and Implementation Research in Mental Health (PA 02-131). We selected 2007 as the starting point because it was the first year that projects supported by the Dissemination and Implementation Research in Health FOAs received funding. For each project, we extracted the title, abstract, project terms, award amount per fiscal year, funding mechanism, and Institute.
Our review was guided by Bogenschneider’s definition of policy as: “the development, enactment, and implementation of a plan or course of action carried out through law, rule, code, or other mechanism in the public or private sector” . Accordingly, we identified potential policy D&I projects by searching the project title, abstract, and term fields for mentions of “policy,” “policies,” “law,” “legal,” “legislation,” “ordinance,” “statute,” “regulation,” “regulatory,” “code,” or “rule.” Two coders then independently reviewed project abstracts and developed preliminary coding categories to capture project characteristics. These categories reflected themes in that data, categories used in previous NIH D&I funding reviews [36, 37], and policy D&I scholarship [3–6, 8, 9]. The coders then jointly developed a coding guide and independently re-reviewed and coded the projects. Because many projects were not exclusively focused on policy D&I, projects were coded according to their policy D&I features. Incongruent coding decisions were identified in <10 % of projects and resolved through discussions.
We classified a project as policy D&I if it explicitly proposed to conduct empirical research about the “content” of a policy (e.g., analysis of the text of clean indoor air laws), the “process” through which it was developed (e.g., assessment of how state legislators use research evidence when developing clean indoor air laws), or the “outcomes” it produced (e.g., evaluation of the impacts of clean indoor air laws on cardiovascular health outcomes). These inclusion criteria were informed by Bogenschneider’s definition of policy  and domains of health policy research proposed by Brownson and colleagues [3–5]. For projects classified as policy D&I, we searched for its NIH project number in ClinicalTrials.gov  and PubMed  to obtain additional information and validate coding decisions.
We calculated the total dollar amount awarded through all NIH research grants and through the D&I FOAs, stratified by funding mechanism and Institute. We also calculated the amount awarded for policy D&I projects, and the percentage of total D&I FOA funding they comprised, within strata. Data were managed and analyzed in Microsoft Excel.
Results and discussion
Between 2007 and 2014, 146 projects were funded through the D&I FOAs, 12 (8.2 %) of which were classified as policy D&I research (Fig. 1). A total of $16,177,250 was awarded for these projects, equivalent to 10.5 % of all funding through the D&I FOAs (Table 1). The NIH Office of the Director allocated 69.7 % of its D&I FOA funding to a policy D&I project, indicating agency-level support for the field. The National Cancer Institute (NCI) was the primary funder of policy D&I research, supporting six projects which comprised 13.0 % of all NCI funding through the D&I FOAs. Some Institutes that were engaged in D&I funding—such as the National Institute of Nursing Research which funded seven D&I projects comprising over 1.0 % of the Institute’s total research grant spending, a proportion larger than any other Institute—did not fund any policy D&I projects. Differences in policy D&I funding between Institutes could potentially be a reflection of varying levels of knowledge about policy D&I research among investigators within different health science disciplines or differences in the degree to which the disciplines are clinically, as opposed to policy, oriented.
NIH funding for policy D&I increased between 2007 and 2014 (Fig. 2). Annual policy D&I funding increased by 98.9 % within this period, from $1,584,327 in 2007 to $3,151,286 in 2014. The proportion of funding awarded through the D&I FOAs that was for policy D&I projects ranged from 14.6 % in 2007 to 8.0 % in 2011. Between 2012 and 2013, funding decreased by 6.3 % for all NIH research grants and increased by 7.4 % for projects funded through the D&I FOAs and by 51.8 % for policy D&I projects.
NIH-funded policy D&I projects utilized a range of methodologies and data sources across different domains of policy research (Tables 2, 3, Additional file 1). These projects were primarily focused on policy outcomes (66.7 %), implementation research (41.7 %), state-level policies (41.7 %), and the USA (83.3 %). Tobacco (33.3 %) and cancer control (25.0 %), for which evidence-based policy strategies exist [41, 42], were the most common topics of focus. Many projects combined survey (58.3 %) and interview (33.3 %) methods with analysis of archival data sources, such as medical records (25.0 %) and policy documents (25.0 %). It was unclear; however, the extent to which projects used true mixed-method designs (e.g., the results of one method informed data collection of the other, or the results of both methods were presented together).
We limited our review to projects funded through FOAs explicitly focused on D&I and may not have identified all NIH-funded policy D&I projects. Our review also did not capture policy D&I projects funded by other US government agencies or philanthropies. We did not attempt to obtain the full text of proposals and thus did not have complete information on project characteristics, such as information on how the projects planned to disseminate their findings. We also did not have data on the characteristics of policy D&I research proposals that were unsuccessful in obtaining funding through NIH. Thus, we were unable to identify specific characteristics that increased the likelihood of a policy D&I project being funded. We dichotomously categorized projects as policy D&I (yes/no) and did not differentiate between projects in which policy D&I was the primary focus or subcomponent. Because only 12 policy D&I projects were identified, subgroup differences and comparisons should be interpreted with caution.
NIH has made an initial investment in policy D&I research, signaling that the field might be entering the mainstream US health research enterprise. The level of support for policy D&I research, however, has varied between Institutes and is probably not commensurate with the potential of evidence-based policy to positively impact human health. Policy D&I researchers have utilized a variety of designs, methods, and data sources to investigate the development processes, content, and outcomes of public and private policies. By mapping the characteristics of NIH funded policy D&I research projects, the current study provides investigators with guidance on approaches to policy D&I research that they can consider when conceptualizing research ideas and developing NIH proposals in this area.
Institute of Medicine. For the public’s health: revitalizing law and policy to meet new challenges. Washington, DC: The National Academies Press; 2011.
Burris S, Wagenaar AC, Swanson J, Ibrahim JK, Wood J, Mello MM. Making the case for laws that improve health: a framework for public health law research. Milbank Q. 2010;88(2):169–210.
Brownson RC, Chriqui JF, Stamatakis KA. Understanding evidence-based public health policy. Am J Public Health. 2009;99(9):1576–83.
Brownson RC, Seiler R, Eyler AA. Measuring the impact of public health policy. Prev Chronic Dis. 2010;7(4):A77.
Dodson E, Brownson RC, Weiss S. Policy dissemination research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York, NY: Oxford University Press; 2012. p. 437–58.
Bogenschneider K, Corbett TJ. Evidence-based policymaking: insights from policy-minded researchers and research-minded policymakers. New York, NY: Routledge; 2011.
Weiss CH. The many meanings of research utilization. Pub Admin Rev. 1979;39(5):426–31.
Innvær S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002;7(4):239–44.
Brownson RC, Royer C, Ewing R, McBride TD. Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006;30(2):164–72.
De Leeuw E, Clavier C, Breton E. Health policy—why research it and how: health political science. Health Res Policy Syst. 2014;12(1):55.
Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011;6(7):e21704.
Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14(1):2.
Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst. 2003;1(1):2.
Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy. A systematic review. PLoS One. 2013;8(10):e77404.
Oliver TR. The politics of public health policy. Annu Rev Public Health. 2006;27(1):195–233.
Jewell CJ, Bero LA. “Developing good taste in evidence”: facilitators of and hindrances to evidence‐informed health policymaking in state government. Milbank Q. 2008;86(2):177–208.
Caplan N. The two-communities theory and knowledge utilization. Am Behav Sci. 1979;22(3):459.
Ellen ME, Lavis JN, Sharon A, Shemer J. Health systems and policy research evidence in health policy making in Israel: what are researchers’ practices in transferring knowledge to policy makers? Health Res Policy Syst. 2014;12(1):67.
Lindblom CE, Cohen DK. Usable knowledge: social science and social problem solving, vol. 21. New Haven, CT: Yale University Press; 1979.
Lomas J. Diffusion, dissemination, and implementation: who should do what? Ann N Y Acad Sci. 1993;703(1):226–37.
Lester JP, Wilds LJ. The utilization of public policy analysis: a conceptual framework. Eval Program Plann. 1990;13(3):313–9.
Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ. 2006;84(8):620–8.
Boyko JA, Lavis JN, Dobbins M, Souza NM. Reliability of a tool for measuring theory of planned behaviour constructs for use in evaluating research use in policymaking. Health Res Policy Syst. 2011;9(1):29.
Dobbins M, Ciliska D, Cockerill R, Barnsley J, DiCenso A. A framework for the dissemination and utilization of research for health‐care policy and practice. Worldviews Evid Based Nurs. 2002;E9(1):149–60.
Glasgow RE, Green LW, Taylor MV, Stange KC. An evidence integration triangle for aligning science with policy and practice. Am J Prev Med. 2012;42(6):646–54.
Samet JM. Epidemiology and policy: the pump handle meets the new millennium. Epidemiol Rev. 2000;22(1):145–54.
Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M. Evidence for public health policy on inequalities: 1: the reality according to policymakers. J Epidemiol Community Health. 2004;58(10):811–6.
Panisset U, Campbell S, Lavis J. 2012–2015 Strategic plan: towards a world in which the best available research evidence informs health policy-making. World Health Organization. 2012. http://www.who.int/evidence/OMSEVIPNetStratPlan.pdf?ua=1. Accessed 1 Dec 2015.
World Health Organization. World report on knowledge for better health: strengthening health systems. 2004. http://www.who.int/rpc/meetings/en/world_report_on_knowledge_for_better_health2.pdf. Accessed 1 Dec 2015.
Sibbald SL, Tetroe J, Graham ID. Research funder required research partnerships: a qualitative inquiry. Implement Sci. 2014;9(1):176.
Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J of Public Health. 2012;102(7):1274–81.
National Institutes of Health. Dissemination and implementation research in mental health. 2002. http://grants.nih.gov/grants/guide/pa-files/PA-02-131.html. Accessed 1 Dec 2015.
National Institutes of Health. Dissemination and implementation research in mental health. 2005. https://grants.nih.gov/grants/guide/pa-files/PAR-06-039.html. Accessed 1 Dec 2015.
Glasgow RE, Chambers DA, Cynkin L. News from the NIH: highlights in implementation science from the National Cancer Institute and the National Institute of Mental Health. Transl Behav Med. 2013;3(4):335–7.
National Institutes of Health. Dissemination and implementation research in health (R01). 2013. http://grants.nih.gov/grants/guide/pa-files/PAR-13-055.html. Accessed 20 Aug 2015.
Tinkle M, Kimball R, Haozous EA, Shuster G, Meize-Grochowski R. Dissemination and implementation research funded by the US National Institutes of Health, 2005–2012. Nurs Res Pract. 2013;2013:1–15.
Neta G, Sanchez MA, Chambers DA, Phillips SM, Leyva B, Cynkin L, et al. Implementation science in cancer prevention and control: a decade of grant funding by the National Cancer Institute and future directions. Implement Sci. 2015;10(1):4.
Research Portfolio Online Reporting Tools (RePORT). http://report.nih.gov/. Accessed 20 Aug 2015.
ClinicalTrials.gov. https://clinicaltrials.gov/. Accessed 20 Aug 2015.
PubMed. http://www.ncbi.nlm.nih.gov/pubmed. Accessed 20 Aug 2015.
Community Preventive Services Taskforce. Guide to community preventive services. Preventing skin cancer: education and policy approaches. http://www.thecommunityguide.org/cancer/skin/education-policy/index.html. Accessed 20 Aug 2015.
Community Preventive Services Taskforce. Guide to community preventive services. Reducing tobacco use and secondhand smoke exposure: smoke-free policies. http://www.thecommunityguide.org/tobacco/smokefreepolicies.html. Accessed 20 Aug 2015.
The authors declare that they have no competing interests.
JP conceptualized and designed the study. JP and RP developed the search strategy, reviewed, and coded projects. RB provided guidance and direction throughout the research process. JP drafted the manuscript and was responsible for overseeing revisions. RP and RB provided reviews of the manuscript and suggested edits that were addressed. All authors approved the final version of the manuscript.
About this article
Cite this article
Purtle, J., Peters, R. & Brownson, R.C. A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014. Implementation Sci 11, 1 (2015). https://doi.org/10.1186/s13012-015-0367-1
- Knowledge Translation
- Implementation Research
- Improve Population Health
- Policy Dissemination
- Integrate Knowledge Translation