- Open Access
- Open Peer Review
Bringing it home: expanding the local reach of dissemination and implementation training via a university-based workshop
© Morrato et al. 2015
- Received: 19 October 2014
- Accepted: 15 June 2015
- Published: 4 July 2015
Currently, national training programs do not have the capacity to meet the growing demand for dissemination and implementation (D&I) workforce education and development. The Colorado Research in Implementation Science Program (CRISP) developed and delivered an introductory D&I workshop adapted from national programs to extend training reach and foster a local learning community for D&I.
To gauge interest and assess learning needs, a pre-registration survey was administered. Based on feedback, a 1.5-day workshop was designed. Day 1 introduced D&I frameworks, strategies, and evaluation principles. Local and national D&I experts provided ignite-style talks on key lessons followed by panel discussion. Breakout sessions discussed community engagement and applying for D&I grants. A workbook was developed to enhance the training and provided exercises for application to an individual’s projects. Day 2 offered expert-led mentoring sessions with selected participants who desired advanced instruction. Two follow-up surveys (immediate post-workshop, 6 months) assessed knowledge gained from participation and utilization of workshop content.
Ninety-three workshop registrants completed an assessment survey to inform workshop objectives and curriculum design; 43 % were new and 54 % reported a basic understanding of the D&I field. Pre-registrants intended to use the training to “apply for a D&I grant” (73 %); “incorporate D&I into existing projects” (76 %), and for quality improvement (51 %). Sixty-eight individuals attended Day 1; 11 also attended Day 2 mentoring sessions. In the 1-week post-workshop survey (n = 34), 100 % strongly agreed they were satisfied with the training; 97 % strongly agreed the workshop workbook was a valuable resource. All Day 2 participants strongly agreed that working closely with faculty and experts increased their overall confidence. In the 6-month follow-up evaluation (n = 23), evidence of new D&I-related manuscripts and grant proposals was found. Training materials were published online (www.ucdenver.edu/implementation/workshops) and disseminated via the National Institutes of Health (NIH) Clinical and Translational Science Awards Consortium. To sustain reach, CRISP adapted the materials into an interactive e-book (www.CRISPebooks.org) and launched a new graduate course.
Local D&I training workshops can extend the reach of national training programs.
- Dissemination and implementation science
- Online education
“. . . the stark reality [is] that we invest billions in research to find appropriate treatments, we spend more than $1 trillion on health care annually, we have extraordinary knowledge and capacity to deliver the best care in the world, but we repeatedly fail to translate that knowledge and capacity into clinical practice” .
Compounding the problem has been historically limited the funding supporting implementation science; for every dollar spent in discovery, mere pennies have been spent learning how effective interventions can be better disseminated .
The good news is that D&I research is an emerging national priority . The Department of Veterans Health Administration (VHA) has been an early leader in the field establishing the Quality Enhancement Research Initiative (QUERI) and publishing an implementation guide . The Agency for Healthcare Research and Quality (AHRQ) has also funded research networks to accelerate the diffusion of health care research into practice and to find solutions to the problems of underuse, overuse, and disparities in implementation of preventive services [6, 7]. The National Institutes of Health (NIH) maintains an active portfolio of D&I research [8, 9] and with the NIH Common Fund supporting a new Healthcare Systems Research Collaboratory to conduct implementation-ready pragmatic clinical trials. This latter program is being conducted in partnership with health care systems to strengthen the relevance of the research results and to expedite translation into health practice . Lastly, the Patient-Centered Outcomes Research Institute (PCORI), an independent, nonprofit organization created under the Affordable Care Act, has the strategic goal to speed the implementation and use of patient-centered outcomes research evidence. Its emphasis on D&I is reflected in its merit review criteria for project proposals  and recent publication of a dissemination and implementation framework . PCORI is mandated to invest 20 % of its funding into dissemination and research capacity building , approximately $400 million annually, which is also stimulating the area of D&I research.
A key challenge for this emerging field is that training programs have not kept up with the capacity needed to meet the growing demand for D&I workforce education and development. D&I research in health comprises paradigms from diverse disciplines, including psychology, public health, communication, and marketing. Therefore, because it is not situated within one specific discipline, there are few university-based D&I training programs [14, 15]. Existing D&I learning opportunities and resources include the following: formal training programs for early-stage investigators [16, 17], university-specific graduate programs , an annual dissemination and implementation science conference now hosted by NIH and Academy Health , and publication of a definitive textbook for the field .
Of particular note, the NIH and VHA established the Training Institute for Dissemination and Implementation Research in Health (TIDIRH) in 2011 . In its first year, 30 trainees were selected from a pool of 266 applicants indicating a greater demand for training opportunities than TIDIRH could meet. To enhance program sustainability and reach, the programs follow a “train the trainer” model and aimed to generate a network of faculty who would share TIDIRH learning at their home institutions in the form of educational events, mentoring, and collaborations. Other national and international training programs were established by colleagues in the School of Social Work at Washington University in St. Louis focusing on research in D&I in the context of mental health  and cancer  and one focusing on knowledge translation in Canada . In addition to training institutes, other educational resources were also created such as regular cyber webinars [23, 24].
Despite these efforts, leaders from the National Institutes of Health Clinical and Translational Science Award (CTSA) programs found deficiencies in D&I knowledge and practice, particularly at the local institutional level . For example, one-third of respondents to a needs assessment survey conducted by the Colorado Clinical and Translational Science Institute was involved in translation to practice and population; however, half reported they “needed but either did not know this resource existed or how to access it” for D&I research training and consultation .
Therefore to bring D&I training to more local investigators, the Colorado Research in Implementation Science and Prevention Program (CRISP) at the University of Colorado Anschutz Medical Campus developed and delivered an introductory D&I training workshop involving leaders from the national TIDIRH program. This paper describes the context of the local training environment, findings from a pre-workshop needs assessment survey, training design and structure, and post-workshop evaluation. Lessons learned may inform others intending to develop local D&I training workshops.
Local institutional context
CRISP is an Agency for Healthcare Research and Quality (AHRQ) Research Center for Excellence in Clinical Preventive Services. It is a program that brings together expertise from across the CU Anschutz Medical Campus in implementation science [27–31], development of interventions to improve use of clinical preventive services [32–34], pragmatic research involving practice-based research networks (PBRNs) [35, 36], and health information technology (HIT) [37, 38]. CRISP’s overall aim is to expand understanding of how to effectively implement interventions that work in order to improve the appropriate use of clinical preventive care for children and adults in the US. CRISP focuses its implementation efforts on strengthening partnerships between primary care providers, public health entities, and communities.
CRISP is a program within the Adult and Child Center for Health Outcomes Research and Delivery Science (ACCORDS). ACCORDS serves as a local incubator for research ideas and fosters interdisciplinary collaboration between clinicians, social scientists, and health services researchers. ACCORDS functions as both an actual and virtual center, with a group of investigators from multiple disciplines who have their primary office on-site and a much larger group affiliating with ACCORDS personnel, programs, and cores while maintaining an off-site research home. Currently, 46 investigators, 25 research assistants, 12 biostatisticians/analysts, and 9 administrative and information technology personnel are co-located together at the ACCORDS center. Over a hundred additional investigators interface with the program, primarily for consultation or to attend educational offerings. Collaborating investigators represent all School of Medicine departments, as well the School of Public Health, the School of Pharmacy, and the College of Nursing. Clinical affiliations include the Colorado Children’s Hospital, University of Colorado Health System, Kaiser Permanente, Denver Health Authority, and the Veterans Health Administration.
One of AHRQ’s charges to its Research Centers for Excellence was to train researchers in implementation science . CRISP organized its educational efforts into four domains by adapting knowledge frameworks used in implementation science [18, 20]: (1) theory and strategies, (2) community and stakeholder engagement, (3) implementation tools and approaches, and (4) evaluation, design, and analysis. A new seminar series on implementation science were initiated to foster a collaborative learning community. Based on key informant discussions with senior leadership on the Anschutz Medical Campus, it was determined that an introductory training workshop on implementation science would be a valuable next step in kick-starting knowledge quickly for a broad group of interested investigators, students, and research staff.
To gauge workshop interest and assess D&I skill level and learning priorities, an online “save-the-date” registration survey was emailed 6 months ahead of the planned workshop to local university and health care organizations affiliated with CRISP and the Colorado Clinical and Translational Sciences Institute (CCTSI). The CCTSI is a collaborative enterprise between the Anschutz Medical Campus, University of Colorado Boulder, Colorado State University, six affiliated hospitals and health care organizations, and multiple community organizations.
Registration survey results
Selected characteristics of workshop registrants
Anschutz Medical Campus
School of Medicine
Colorado School of Public Health
Other schools and colleges (e.g., pharmacy, nursing, dental medicine)
Kaiser Permanente Colorado, National Jewish Health, Denver Health
Children’s Hospital of Colorado
Type of affiliation
University research staff
Practicing clinician, health provider, or public health professional
Highest role in research
Led quality improvement projects
Level of D&I science knowledge
New to D&I science
Intended application of workshop learnings
Apply for a D&I-related grant
Incorporate D&I principles into existing projects
Quality improvement practice
Most registrants intended to use the knowledge gained from the workshop to apply for grants, to develop D&I interventions, and to incorporate D&I practices into existing research projects. The top five topics of greatest interest were the following: D&I study design, D&I strategies, measurement for D&I, available tools and resources for D&I, and the theoretical basis for D&I science and practice. The registration survey also highlighted the training needs for two other local audiences:  clinical and public health practitioners who wanted to incorporate D&I principles into quality improvement initiatives and  research staff, who were not principal investigators, but wanted to better understand the field and expand their competencies. Half of the pre-registered participants wanted a 1-day workshop format.
Information from the registration survey was used to identify the learning objectives and delivery method for the workshop. As a result, the workshop was designed to be relevant to both researchers and practitioners interested in D&I for the improvement of public health and health care. The primary learning objective was to introduce participants to D&I concepts, strategies, and design principles. The goal was to provide the resources necessary for participants to formulate their own D&I research questions and to incorporate D&I principles into their grant proposals. The secondary learning objective was to build a stronger network of local D&I researchers and encourage cross-disciplinary mentorship and collaborations across the institution.
Dissemination: the targeted distribution of information and intervention materials to a specific public health or clinical practice audience. The intent is to spread (“scale up”) and sustain knowledge and the associated evidence-based interventions.
Implementation: the use of strategies to adopt and integrate evidence-based health interventions and change practice patterns within specific settings.
Dissemination and implementation research: research to produce a generalizable knowledge base about how health information, interventions, and new clinical practices and policies are transmitted and translated for public health and health care service use in specific settings.
Definitions, backgrounds, and rationale. Describe the importance of D&I research and practice in achieving a healthy America. Define common D&I terminology.
Educational content for this domain was obtained from the following sources: the National Institutes of Health on their priorities and approaches to D&I science , the QUERI organizational framework for system-level change  and implementation guide , PCORI’s national priorities and research agenda , and the chapter on D&I terminology by Rabin and Brownson in Dissemination and Implementation Research in Health and content from the www.makeresearchmatter.org website.
Theories and approaches. Demonstrate the use of common D&I frameworks. Demonstrate how to design for D&I. Identify existing D&I resources and toolkits.
Information on D&I models and frameworks was obtained from work published by investigators from the CDC Prevention Research Center at Washington University in St. Louis [44, 45]. Key models from diffusion theory [14, 46, 47] and referenced by the Implementation Science Division of the National Cancer Institute [48–53] were presented. Methods and approaches in stakeholder engagement were adapted from work supported through the NIH CTSA program [35, 54, 55]. Resources on D&I planning tools included the following: www.makeresearchmatter.org, the AHRQ dissemination planning tool in patient safety , and the Academy Health decision guide for researchers navigating the translation and dissemination of findings .
Design and analysis. Compare and contrast study evaluation approaches commonly used in D&I. Identify key metrics in D&I.
Resources on pragmatic trials and the PRECIS indicator to dimension more pragmatic trials [58–61] were used to describe common study design approaches. Resources on pragmatic measures , a conceptual framework for outcomes for implementation research , and cost measures to enhance translation  were discussed in the workshop.
Practice-based considerations. Share tips for success in writing a D&I research proposal and incorporation of stakeholder engagement.
Educational content from this domain was derived from the personal experiences of the expert faculty involved in the workshop (see Table 3).
Workshop structure and educational delivery
Based on the information from the needs assessment survey, a 1½-day introductory training workshop was developed. Day 1 provided an introduction to D&I principles for both beginner and intermediate learners. Day 2 was an optional half-day for learners who were more advanced and desired consultation with faculty and experts on a specific D&I project or proposal.
Day 1 (full day): an introduction to D&I science
Workshop agenda for Day 1: an introduction to D&I Science
To review workshop objectives
Why are we here?
To describe the importance of D&I research and practice in achieving a healthy America
The promise and challenge of D&I in health
What are we talking about?
To define common terminology
D&I definitions, frameworks
To demonstrate the use of common frameworks
To network with D&I colleagues
What approaches should I take?
To identify existing D&I resources and toolkits
Strategies and toolkits
To demonstrate how to design for D&I
To network with D&I colleagues
How do I know if I am successful?
To compare and contrast study evaluation approaches commonly used in D&I
Evaluation and measurement
To identify key metrics in D&I
To network with D&I colleagues
Application: RE-AIM activity
To apply knowledge of RE-AIM framework
Tips for success
To share tips for success in writing a research proposal, implementing a D&I program, or managing a D&I project
To recap key learning points
To network with D&I colleagues
Day 2 (half day): small-group consultation with D&I experts
The objective of Day 2 was to meet the needs of more advanced learners and provide opportunity for in-depth feedback on specific D&I proposals or projects. Participants were asked to submit a one- to two-page concept paper for one of the three D&I topic areas: (1) research, (2) practice, or (3) project management. This approach was modeled after the application process for NIH training institutes. Day 2 was a half-day session and participation was limited to allow for more in-depth small-group discussions with workshop faculty and invited experts. Groups were formed based on related research interests and study populations to facilitate peer-to-peer learning and mentorship. At the end of the session, the group facilitators met collectively to review common themes and design issues across projects.
A workshop planning committee (comprised of the authors) was established to develop educational objectives and course content. Committee members represented a diverse cross section of local academic and health care system researchers in implementation science from the University of Colorado Schools of Medicine and Public Health; the Veterans Health Administration, Kaiser Permanente Colorado, and National Jewish Health.
Guest faculty: D&I experts
Juliana Barnard, MA
Senior Professional Research Associate, University of Colorado Denver School of Medicine
Arne Beck, PhD
Senior investigator and Director of Quality Improvement and Strategic Research, Kaiser Permanente Colorado Institute for Health Research
Ross Brownson, PhD
Professor and Co-Director, CDC Prevention Research Center in St. Louis, Washington University in St. Louis
Russell Glasgow, PhD
Deputy Director Emeritus of Implementation Science, Division of Cancer Control and Population Sciences at the U.S. National Cancer Institute
David Goff, MD PhD
Dean, Colorado School of Public Health
Allison Kempe, MD, MPH
Director, Adult and Child Clinical Outcomes Research and Service Delivery Science Program, University of Colorado Denver
Executive Director, The Center for African American Health
Julie Lowery, PhD
Associate Director, VA Ann Arbor Health Services Research & Development Center of Innovation
Spero Manson, PhD
Director, Center for American Indian and Alaska Native Diabetes Translational Research, Colorado School of Public Health,
Wilson Pace, MD
Chief Executive Officer, Distributed Ambulatory Research in Therapeutics Network (DARTNet) Institute
Debra Ritzwoller, PhD
Senior Investigator and health economist, Kaiser Permanente Colorado Institute for Health Research
Evaluation and discussion
Figure 1 shows the number of workshop participants who provided feedback in the 1-week and 6-month follow-up evaluation surveys. The intent of these surveys was to provide formative feedback and guide changes for subsequent workshops. The CCTSI Evaluation Core conducted an independent evaluation to ascertain participant feedback and workshop outcomes. The survey questions elicited feedback about the workshop overall, knowledge gained, intended application of learnings, and suggestions for future workshops using closed and open-ended questions.
1-week post-workshop evaluation
Day 1 feedback
Respondents most often commented on the value of presentations by experts in the field. One individual, new to the field, shared that he/she “found it to be a great overview of the material.” More specifically, “It was great to have leading experts presenting and giving insight.” Additionally, respondents provided a great deal of positive feedback regarding the usefulness of the materials and resources provided. Nearly all (97 %) of the respondents agreed or strongly agreed that the workshop workbook would be a valuable resource for them. One respondent stated that “the synthesis of literature and resources condensed into one source document [was] very helpful.”
Day 2 feedback
All respondents agreed or strongly agreed that Day 2 was beneficial. Specifically, 100 % of respondents agreed or strongly agreed that they received helpful feedback during the second day of the workshop. One individual stated that “the group discussion and feedback session was very helpful, both to learn from hearing about other D&I research and to get feedback on my own [research].” Also, respondents found it beneficial to discuss their projects with one another. For example, one individual stated he/she benefited from “meeting with others who are interested in similar types of research across multiple disciplines and hearing about problems and solutions others are currently facing.”
Suggestions for future D&I training workshops
“A dissemination type forum so that others can view applied D&I methods/findings and where you can also obtain feedback on your study.” Across items, various respondents suggested providing breakout sessions that are “geared more towards advanced users”
When designing D&I workshops in the future, one respondent suggested “having a ‘new to D&I day’ and an intermediate-advanced day, or have breakout sessions that cover the range of experience.”
6-month post-workshop evaluation
This formative evaluation assessed short-term outcomes from participating in the workshop. Survey response rates among those who attended Day 1 only was 18 % (n = 12 out of 68 participants) and 79 % (n = 12 out of 14 participants) for those who also attended Day 2. Note that the response rate was higher among those respondents who participated in the more in-depth, hands-on Day 2 session. This would be expected as they were more invested in the program.
Respondents reported that the most impactful/useful component of the workshop were “the new ideas/insights gained”. Approximately one-third of respondents reported that they had developed or were currently developing a proposal to obtain funding for a D&I relevant research project (n = 7). One-third reported they developed or were developing a D&I scientific paper (n = 7). Several mentioned (n = 4) they had modified a currently funded research project based on information from the workshop. One-third (n = 8) mentioned they had developed new D&I relevant collaborations and had attended multiple CRISP-sponsored seminars (n = 7). A majority of respondents (n = 10) had accessed D&I-related online resources and local institutional support since the workshop to expand upon their learning.
Future training suggestions included the following: (a) pragmatic clinical trial design, (b) advanced workshops on specific D&I topics, and (c) a refresher D&I workshop. Finally, survey respondents, who predominantly described themselves as having moderate D&I understanding, reported low to moderate interest in enrolling in a D&I research in health graduate course. This formative information led the CRISP education team to focus its next workshop on pragmatic trials because the majority of respondents (n = 12) reported being the most interested in this learning topic.
The purpose of the workshop was to engage both new and intermediate learners. An advantage of this approach was that it rapidly jump-started D&I training across the campus making it broadly visible. It provided efficiencies of scale for the workshop faculty allowing them to pool resources and attract D&I experts for a singular training event. Including learners at different levels also supported the idea of a learning community. A disadvantage of this approach is that it is challenging to provide differentiated learning in a condensed large-group setting. The use of expert panel discussion, luncheon breakout sessions, provision of a synthesizing workbook (then e-book) resource, and offering a second day for small-group discussion and expert feedback helped to address this challenge.
The evaluation is limited by a low response rate, particularly at the 6-month post-workshop time point, among participants who only participated in the first day of the workshop. It should be noted that response was high (79–86 %) for the group who also attended the more intensive second-day workshop. This may reflect their greater engagement in the learning objectives. On the positive side, evidence of new D&I grants and publications as a result of the workshop was found. However, the findings should be interpreted cautiously as their generalizability to all workshop participants is not known. Future evaluations should examine training outcomes by initial participant level of D&I knowledge (e.g., beginner vs. more advanced) and discipline or field.
Another limitation of an introductory workshop format is the inability to provide in-depth training and sustained research mentorship. To address this need, participants were directed to additional local resources, such as the monthly CRISP seminar series on D&I theory, methods, application and evaluation, and a new D&I graduate course offered through the Clinical Science program of the CCTSI (entitled Dissemination and Implementation Research in Health). The graduate course is targeted for beginning D&I learners and its educational objectives are similar to the workshop. It differs from the introductory workshop in that it allowed for more in-depth class discussion and reflection on the theory and measures and provided the forum for personal application of these concepts via a multi-step, graded assignment on the development of a dissemination and implementation plan. The course was first taught in the Fall 2015 semester to a group of students with diverse educational and background and research interests (e.g., clinical and public health). It received positive feedback and several faculty and students interested in D&I research are now writing this course into their education plans for their degree program, post-doctoral training, or career development grant proposals.
Implementation science educational resources at the University of Colorado Anschutz Medical Campus
Year of offering
Mode of delivery
Methods and application.
In-person (1x/mo). Archived webinars.
Fostering a learning community.
Intro to theory, strategies, and evaluation. Case application. (Optional: expert consultation).
In-person (1 + 1/2 day). Open-access online learning system (Canvas).
Self-guided navigation of theory, strategies, and evaluation resources.
Interactive, web-based (www.CRISPebooks.org)
Intro to theory, strategies, and evaluation. Proposal development.
In-person (2-credit hours). Graded assessment.
Intro to PRECIS-2  and patient engagement. Case application. (Optional: expert consultation).
In-person (1 + 1/2 day). Open-access online learning system (Canvas).
Self-guided navigation of PRECIS-2 and patient engagement resources.
Interactive, web-based (www.CRISPebooks.org).
Implementing and creating evidence for mHealth interventions. Developing mobile applications in the private sector.
In-person (1 day). Open-access online learning system (Canvas).
PCORI research and engagement
Tips-for-success in PCORI research and engagement applications.
In-person (2 h). Open-access online learning system (Canvas).
Facilitating community-based participatory research. Building capacity in community-academic partnerships.
Consultation. Five-day community immersion training and directed reading.
A structured, collaborative roundtable discussion with relevant research experts 
A new needs assessment survey is being fielded among the CRISP learning community to identify further training opportunities for differentiated D&I learning for beginner, intermediate, and advanced learners.
A short D&I training workshop delivered in a local academic setting can extend the reach of national D&I training programs and quickly engage new and intermediary learners. Workshop success was strengthened by the use of a pre-workshop registration survey to guide curriculum development and the inclusion of national and local D&I experts. For the more experienced learners, the Day 2 intensive small-group feedback sessions were well received and a good complement to Day 1’s introductory nature. Convening the workshop also served as a catalyst for building a shared educational vision among diverse faculty across the organizations with D&I expertise at our local institution (i.e., schools of medicine, public health, VHA, and Kaiser Permanente). One of the educational products from this collaboration was launching a new graduate course in the multi-disciplinary clinical sciences graduate program. The training materials and workbook resources were published online and disseminated via the NIH CTSA network to increase spread in hope that they might serve as a resource for other local D&I training workshops. To achieve reach among those who did not attend the workshop, an interactive e-book format for the workbook was also launched Fall 2014.
The D&I workshop was supported by grant number P01HS021138 from the Agency for Healthcare Research and Quality, by NIH/NCATS Colorado CTSI Grant Number UL1 TR001082, and by VA grant number HX 13-001. Contents are the authors’ sole responsibility and do not necessarily represent the official views of the Agency for Healthcare Research and Quality, NIH, or the VHA. BR and BL received support from the Institute for Health Research Strategic Initiatives Committee at Kaiser Permanente, CO, through the Center for Health Education Dissemination and Implementation Research. Content will be presented in part at the 7th Annual Conference on the Science of Dissemination and Implementation, Bethesda, MD, 8–9 December 2014.
The authors are grateful to the 2013 D&I workshop attendees and guest faculty for their feedback and evaluation of the program. We thank Laura Seewald who provided writing services on behalf of the University of Colorado Denver.
- Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, D.C.: National Academy of Sciences; 2001.Google Scholar
- Adams K, Corrigan JM, editors. Institute of Medicine of the National Academies. Priority areas for national action: transforming health care quality. Washington, D.C: The National Academies Press; 2003.Google Scholar
- Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J Public Health. 2012;102(7):1274–81. doi:10.2105/AJPH.2012.300755. PubMed.PubMedPubMed CentralGoogle Scholar
- Colditz GA. The promise and challenge of dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. New York, NY: Oxford University Press; 2012.Google Scholar
- Implementation Guide. Department of Veterans Health Administration, Health Services Research & Development, Quality Enhancement Research Initiative. 2013.Google Scholar
- Agency for Healthcare Research and Quality. Accelerating Change and Transformation in Organizations and Networks (ACTION): fact sheet: field partnerships for applied research Rockville, MD: agency for healthcare research and quality. 2009. [updated November 2009]. Available from: http://www.ahrq.gov/research/findings/factsheets/translating/action/index.html.Google Scholar
- Research Centers for Excellence in Clinical Preventive Services Rockville, MD: agency for healthcare research and quality. 2014. [updated March 2014]. Available from: http://www.ahrq.gov/professionals/prevention-chronic-care/decision/research-centers/index.html
- Dissemination and implementation research in health (R01). In: Department of Health and Human Services, National Institutes of Health, editors. 2013Google Scholar
- Tinkle M, Kimball R, Haozous EA, Shuster G, Meize-Grochowski R. Dissemination and implementation research funded by the US National Institutes of Health, 2005–2012. Nurs Res Pract. 2013;2013:909606. doi:10.1155/2013/909606. PubMed PMID: 23606958; PMCID: 3625600.PubMedPubMed CentralGoogle Scholar
- National Institutes of Health. NIH HCS Research Collaboratory. 2014. Available from: http://commonfund.nih.gov/hcscollaboratory/index.Google Scholar
- Patient-Centered Outcomes Research Institute. Merit review criteria 2014. 2014. Available from: http://www.pcori.org/content/merit-review-criteria.Google Scholar
- Esposito D, Heeringa J, Bradley K, Croake S, Kimmey L. PCORI Dissemination and Implementation Framework. Washington, D.C.: Patient-Centered Outcomes Research Institute; 2015.Google Scholar
- Patient-Centered Outcomes Research Insitute. 2013 annual report. 2014.Google Scholar
- Dearing JW, Keith RE. Historical roots of dissemination and implementation science. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in health: Translating Science to Practice. New York, NY: Oxford University Press; 2012.Google Scholar
- Meissner HI, Glasgow RE, Vinson CA, Chambers D, Brownson RC, Green LW, et al. The U.S. training institute for dissemination and implementation research in health. Implement Sci. 2013;8:12. doi:10.1186/1748-5908-8-12. PubMed PMID: 23347882; PMCID: 3564839.PubMedPubMed CentralGoogle Scholar
- Stamatakis KA, Norton WE, Stirman SW, Melvin C, Brownson RC. Developing the next generation of dissemination and implementation researchers: insights from initial trainees. Implement Sci. 2013;8:29. doi:10.1186/1748-5908-8-29. PubMed PMID: 23497462; PMCID: 3626831.PubMedPubMed CentralGoogle Scholar
- Proctor EK, Landsverk J, Baumann AA, Mittman BS, Aarons GA, Brownson RC, et al. The implementation research institute: training mental health implementation researchers in the United States. Implement Sci. 2013;8:105. doi:10.1186/1748-5908-8-105. PubMed PMID: 24007290; PMCID: 3844451.PubMedPubMed CentralGoogle Scholar
- Gonzales R, Handley MA, Ackerman S, O’Sullivan PS. A framework for training health professionals in implementation and dissemination science. Academic medicine: journal of the Association of American Medical Colleges. 2012;87(3):271–8. doi:10.1097/ACM.0b013e3182449d33. PubMed PMID: 22373617; PMCID: 3307591.Google Scholar
- Academy Health, editor. 7th annual conference on the science of dissemination and implementation: transforming health systems to optimize individual and population health. Bethesda, MD: Academy Health; 2014.Google Scholar
- Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York, NY: Oxford University Press; 2012.Google Scholar
- Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) Prevention Research Center in St. Louis, Washington University in St. Louis [October 17, 2014]. Available from: http://prcstl.wustl.edu/training/Pages/MTDIRC.aspx
- Leung BM, Catallo C, Riediger ND, Cahill NE, Kastner M. The trainees’ perspective on developing an end-of-grant knowledge translation plan. Implement Sci. 2010;5:78. doi:10.1186/1748-5908-5-78. PubMed PMID: 20946679; PMCID: 2967498.PubMedPubMed CentralGoogle Scholar
- Health Services Research and Development. QUERI Implementation Seminar Series Cyberseminars Series: U.S. Department of Veterans Affairs. 2015.Google Scholar
- Implementation Science Research Webinar Series. National Cancer Institute (NCI) Division of Cancer Control & Population Sciences Implementation Science Team. 2014.Google Scholar
- Morrato EH, Concannon TW, Meissner P, Shah ND, Turner BJ. Dissemination and implementation of comparative effectiveness evidence: key informant interviews with Clinical and Translational Science Award institutions. Journal of Comparative Effectiveness Research. 2013;2(2):185–94. doi:10.2217/cer.13.10. PubMed PMID: 24236560; PMCID: 3961460.PubMedPubMed CentralGoogle Scholar
- The Evaluation Center. Colorado Clinical and Translational Sciences Institute (CCTSI) needs assessment. University of Colorado Denver. 2014.Google Scholar
- Glasgow RE, Kessler RS, Ory MG, Roby D, Gorin SS, Krist A. Conducting rapid, relevant research: lessons learned from the My Own Health Report project. American Journal of Preventive Medicine. 2014;47(2):212–9. doi:10.1016/j.amepre.2014.03.007. Epub 2014/06/24.PubMedPubMed CentralGoogle Scholar
- Dickinson LM, Dickinson WP, Nutting PA, Fisher L, Harbrecht M, Crabtree BF, et al. Practice context affects efforts to improve diabetes care for primary care patients: a pragmatic cluster randomized trial. J Gen Intern Med. 2015;30(4):476–82. doi:10.1007/s11606-014-3131-3. Epub 2014/12/05. PubMed PMID: 25472509; PMCID: PMC4370994.PubMedGoogle Scholar
- Neta G, Glasgow RE, Carpenter CR, Grimshaw JM, Rabin BA, Fernandez ME, et al. A framework for enhancing the value of research for dissemination and implementation. Am J Public Health. 2015;105(1):49–57. doi:10.2105/ajph.2014.302206. Epub 2014/11/14. PubMed PMID: 25393182; PMCID: PMC4265905.PubMedPubMed CentralGoogle Scholar
- Albright K, Gechter K, Kempe A. Importance of mixed methods in pragmatic trials and dissemination and implementation research. Acad Pediatr. 2013;13(5):400–7. doi:10.1016/j.acap.2013.06.010. PubMed.PubMedGoogle Scholar
- Smith MY, Morrato E. Advancing the field of pharmaceutical risk minimization through application of implementation science best practices. Drug safety: an International Journal of Medical Toxicology and Drug Experience. 2014;37(8):569–80. doi:10.1007/s40264-014-0197-0. PubMed PMID: 25005707; PMCID: 4134476.Google Scholar
- Bender BG, Dickinson P, Rankin A, Wamboldt FS, Zittleman L, Westfall JM. The Colorado Asthma Toolkit Program: a practice coaching intervention from the High Plains Research Network. J Am Board Fam Med. 2011;24(3):240–8. doi:10.3122/jabfm.2011.03.100171. PubMed. Epub 2011/05/10.PubMedGoogle Scholar
- Westfall JM, Zittleman L, Sutter C, Emsermann CB, Staton EW, Van Vorst R, et al. Testing to prevent colon cancer: results from a rural community intervention. Ann Fam Med. 2013;11(6):500–7. doi:10.1370/afm.1582. Epub 2013/11/13. PubMed PMID: 24218373; PMCID: PMC3823720.PubMedPubMed CentralGoogle Scholar
- Kempe A, Saville A, Dickinson LM, Eisert S, Reynolds J, Herrero D, et al. Population-based versus practice-based recall for childhood immunizations: a randomized controlled comparative effectiveness trial. Am J Public Health. 2013;103(6):1116–23. doi:10.2105/AJPH.2012.301035. PubMed PMID: 23237154; PMCID: 3619016.PubMedPubMed CentralGoogle Scholar
- Westfall JM, Ingram B, Navarro D, Magee D, Niebauer L, Zittleman L, et al. Engaging communities in education and research: PBRNs, AHEC, and CTSA. Clinical and Translational Science. 2012;5(3):250–8. doi:10.1111/j.1752-8062.2011.00389.x. Epub 2012/06/13. PubMed PMID: 22686202; PMCID: PMC3375864.PubMedPubMed CentralGoogle Scholar
- Westfall JM, Dolor RJ, Mold JW, Hedgecock J. PBRNS engaging the community to integrate primary care and public health. Ann Fam Med. 2013;11(3):284–5. doi:10.1370/afm.1537. Epub 2013/05/22. PubMed PMID: 23690332; PMCID: PMC3659149.PubMedPubMed CentralGoogle Scholar
- Schilling LM, Kwan BM, Drolshagen CT, Hosokawa PW, Brandt E, Pace WD, et al. Scalable Architecture for Federated Translational Inquiries Network (SAFTINet) Technology Infrastructure for a Distributed Data Network. EGEMS (Wash DC). 2013;1(1):1027. doi:10.13063/2327-9214. Epub 2013/01/01. 1027. PubMed PMID: 25848567; PMCID: PMC4371513.Google Scholar
- Pace WD, Fox CH, White T, Graham D, Schilling LM, West DR. The DARTNet Institute: seeking a sustainable support mechanism for electronic data enabled research networks. EGEMS (Wash DC). 2014;2(2):1063. doi:10.13063/2327-9214. Epub 2014/01/01.1063. PubMed PMID: 25848603; PMCID: PMC4371434.Google Scholar
- Training the next generation of leaders in clinical preventive services: profiles from the AHRQ Research Centers for Excellence Rockville, MD: Agency for Healthcare Research and Quality; 2013 [5/1/2014]. Available from: http://www.ahrq.gov/professionals/prevention-chronic-care/decision/research-centers/rce_profile_morrato.html.
- Thacker SB, Brownson RC. Practicing epidemiology: how competent are we? Public Health Reports. 2008;123 Suppl 1:4–5. PubMed.PubMedPubMed CentralGoogle Scholar
- Padek M, Brownson RC, Proctor EK, Colditz GA, Kreuter MW, Dobbins M, et al. Developing dissemination and implementation competencies for training programs. In: The National Institutes for Health (NIH), The Agency for Healthcare Research and Quality (AHRQ), Patient-Centered Outcomes Research Institute (PCORI), Robert Wood Johnson Foundation, William T. Grant Foundation, (VA) VA, editors. 7th Annual Conference on the Science of Dissemination and Implementation. Bethesda, MD: Academy Health; 2014.Google Scholar
- Stetler CB, McQueen L, Demakis J, Mittman BS. An organizational framework and strategic implementation for system-level change to enhance research-based practice: QUERI Series. Implement Sci. 2008;3:30. doi:10.1186/1748-5908-3-30. Epub 2008/05/31. PubMed PMID: 18510750; PMCID: PMC2430586.PubMedPubMed CentralGoogle Scholar
- National Priorities for Research and Research Agenda. Washington, D.C.: Patient-Centered Outcomes Research Institute. 2012.Google Scholar
- Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. American Journal of Preventive Medicine. 2012;43(3):337–50. doi:10.1016/j.amepre.2012.05.024. PubMed PMID: 22898128; PMCID: 3592983.PubMedPubMed CentralGoogle Scholar
- Tabak RG, Khoong EC, Chambers D, Brownson RC. Models in dissemination and implementation research: useful tools in public health services and systems research. Frontiers in PHSSR. 2013;2(1):8.Google Scholar
- Rogers EM, editor. Diffusion of innovations. 5th ed. New York: Free Press; 2003.Google Scholar
- Dearing JW, Smith DK, Larson RS, Estabrooks CA. Designing for diffusion of a biomedical intervention. American Journal of Preventive Medicine. 2013;44(1 Suppl 2):S70–6. doi:10.1016/j.amepre.2012.09.038. PubMed.PubMedGoogle Scholar
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi:10.1186/1748-5908-4-50. PubMed PMID: 19664226; PMCID: 2736161.PubMedPubMed CentralGoogle Scholar
- Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41(3-4):171–81. doi:10.1007/s10464-008-9174-z. Epub 2008/02/28. PubMed.PubMedGoogle Scholar
- Flaspohler P, Lesesne CA, Puddy RW, Smith E, Wandersman A. Advances in bridging research and practice: introduction to the second special issue on the interactive system framework for dissemination and implementation. Am J Community Psychol. 2012;50(3-4):271–81. doi:10.1007/s10464-012-9545-3. Epub 2012/08/10. PubMed PMID: 22875685; PMCID: PMC3766740.PubMedPubMed CentralGoogle Scholar
- Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient safety / Joint Commission Resources. 2008;34(4):228–43. Epub 2008/05/13. PubMed.PubMedGoogle Scholar
- Green LW, Kreuter MW. Health program planning: an educational and ecological approach. 4th ed. New York: McGraw-Hill; 2001.Google Scholar
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7. Epub 1999/09/04. PubMed PMID: 10474547; PMCID: PMC1508772.PubMedPubMed CentralGoogle Scholar
- Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27(8):985–91. doi:10.1007/s11606-012-2037-1. Epub 2012/04/25. PubMed PMID: 22528615; PMCID: PMC3403141.PubMedPubMed CentralGoogle Scholar
- Norman N, Bennett C, Cowart S, Felzien M, Flores M, Flores R, et al. Boot camp translation: a method for building a community of solution. J Am Board Fam Med. 2013;26(3):254–63. doi:10.3122/jabfm.2013.03.120253. Epub 2013/05/10. PubMed PMID: 23657693; PMCID: PMC3828080.PubMedPubMed CentralGoogle Scholar
- Agency for Healthcare Research and Quality, U.S. Department of Defense. Advances in patient safety: from research to implementation. 2005. Contract No.: AHRQ 05-0021-CD.Google Scholar
- Sofaer S, Talis N, Edmunds M, Papa K. Navigating the translation and dissemination of PHSSR findings: a decision guide for researchers. Washington, D.C: AcademyHealth; 2013.Google Scholar
- Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care. 2005;43(6):551–7. Epub 2005/05/24. PubMed.PubMedGoogle Scholar
- Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet. 2005;365(9453):82–93. doi:10.1016/s0140-6736(04)17670-8. Epub 2005/01/11. PubMed.PubMedGoogle Scholar
- Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Journal of clinical epidemiology. 2009;62(5):464–75. doi:10.1016/j.jclinepi.2008.12.011. Epub 2009/04/08. PubMed.PubMedGoogle Scholar
- Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290(12):1624–32. doi:10.1001/jama.290.12.1624. Epub 2003/09/25. PubMed.PubMedGoogle Scholar
- Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. American Journal of Preventive Medicine. 2013;45(2):237–43. doi:10.1016/j.amepre.2013.03.010. PubMed.PubMedGoogle Scholar
- Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. doi:10.1007/s10488-010-0319-7. Epub 2010/10/20. PubMed PMID: 20957426; PMCID: PMC3068522.PubMedGoogle Scholar
- Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing behavioral interventions: a practical guide to enhance translation. Ann Behav Med. 2009;37(2):218–27. doi:10.1007/s12160-009-9088-5. Epub 2009/03/18. PubMed.PubMedGoogle Scholar
- Byrne DW, Biaggioni I, Bernard GR, Helmer TT, Boone LR, Pulley JM, et al. Clinical and translational research studios: a multidisciplinary internal support program. Academic Medicine: Journal of the Association of American Medical Colleges. 2012;87(8):1052–9. doi:10.1097/ACM.0b013e31825d29d4. PubMed PMID: 22722360; PMCID: 3406254.Google Scholar
- Loudon K, Zwarenstein M, Sullivan F, Donnan P, Treweek S. PRECIS-2: a tool to improve the applicability of randomised controlled trials. Trials. 2013;14 Suppl 1:O28. doi:10.1186/1745-6215-14-S1-O28.PubMed CentralGoogle Scholar
- Byrne DW, Biaggioni I, Bernard GR, Helmer TT, Boone LR, Pulley JM, et al. Clinical and translational research studios: a multidisciplinary internal support program. Academic Medicine. 2012;87(8):1052–9. doi:10.1097/ACM.0b013e31825d29d4. PubMed.PubMedPubMed CentralGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.