Skip to main content

Table 3 Additional characteristics of the D&I CBIs included in the review

From: Building capacity in dissemination and implementation science: a systematic review of the academic literature on teaching and training initiatives

First author/date Context/profession Information provided on CBI content and structurea Numbers attending the CBI and evaluation of key findingsb
Ackerman, 2016 [88] Cardiology/medical students Description in text and table on programme curriculum 1. Action Research Program (ARP)
Six students were selected, with an additional 2 joining in month 7 of the programme (for these 2students training focussed on the experiential clinic-based learning component). The CBI was evaluated through interviews with students and clinicians. Students reported increased understanding of how care delivery systems work, improved clinical skills and confidence in interactions with patients. Clinicians reported increased efficiency at the clinic level and improved job satisfaction as a result of their mentoring role. With regards to the improvement projects, although some ideas were implemented, most did not move from conceptualisation because students did not have enough time dedicated to conduct the project (reported by clinicians) or were not given enough guidance by faculty (reported by students).
Baumann, 2019 [57] (e-print ahead of 2020 publication) Mental health/multiple Description in text and reference to another publication on the CBI 2. Implementation Research Institute (IRI)
Article evaluated the CBI across 4 different cohorts from 2010 to 2013—the first 3 years had 10 delegates and the 4thyear had 12 (43 in total). Applicants selected for the IRI training, versus those that were not, were 6 times more likely to be awarded a D&I grant after attending the IRI, even when controlling for other variables. Applicants’ odds of publishing in the journal ‘Implementation Science’ were higher for earlier alumni, starting at 12% after 1 year out of training to 94% for those 4 years from training (versus non-selected applicants which remained relatively stable). CBI also described in Brownson, 2017A, Luke, 2016 and Proctor, 2013.
Brownson, 2017A [18] Mental health/multiple Brief description (1 paragraph) and references to other publications on the CBI and a website on the CBI 2. Implementation Research Institute (IRI)
Article states that IRI has trained 43 fellows at the time of publication—the breakdown of numbers of each year is referred to in Baumann, 2019 (above). No evaluative data is included in this article. CBI also described in Baumann, 2019, Luke, 2016 and Proctor, 2013.
Brownson, 2017B [18] Cancer/multiple Brief description (1 paragraph) and references to other publications on the CBI and a website on the CBI 3. Mentored Training for Dissemination & Implementation Research in Cancer (MT-DIRC)
Article states that 14 fellows per year attend the training. CBI not evaluated in this article. CBI also described in Padek, 2018.
Brownson, 2017C [18] NR/NR Mentioned briefly in the text (< 1 paragraph) 4. Introduction to D&I Science
No evaluative data reported.
Brownson, 2017D [18] NR/NR Mentioned briefly in the text (< 1 paragraph) 5 Implementing and Evaluating Evidence-based Practice
No evaluative data reported.
Burton, 2016 [89] Child and adolescent behavioural health/multiple Description in text, figure on programme plan by semester, and a table of the benefits of a blended theoretical approach 6. The Institute for Translational Research in Adolescent Behavioural Health
In total 28 scholars were recruited in the first 2 years. Preliminary results from surveys revealed that gaining research experience through real-world service-learning opportunities was a key factor in the decision to apply for the graduate certificate. The online method for presenting coursework proved difficult and required additional time and effort from faculty to help navigate technology. Academic mentors felt the design of the program was beneficial but that they needed more guidance on their role as mentors and the scope of the projects.
Carlfjord, 2017 [47] Non-specific/multiple Description in text and tables on the topics/lectures in the curriculum, group discussions and seminars 7. Implementation Theory and Practice
This CBI occurs once a year and was evaluated over a 5-year period (2011–2015). In total, 101 completed the course, with numbers ranging from 20 to 25 over the years (this is now capped at 20 for practical reasons). Students rated their overall perception of the course and its contents highly. The majority reported the course had contributed to their current knowledge in implementation science and two-thirds felt that the knowledge gained would be very relevant to their work. Additional data collected a few months after course completion revealed that most individuals felt they had used the knowledge gained in their work and that this had been valuable.
Farrell, 2014A [90] Cancer/multiple Brief description (1 paragraph) and link to website relating to CBI 8. Research to Reality Cyber Seminars
Article reports that registrants and participants on the seminars have significantly grown. The first web-seminar was conducted in January 2010 with over 1100 registrants and 700 participants. Since the first webinar through December 2013, nearly 20,000 people have registered for the yearly, 10-month seminar schedule. On average there are 675 registrants and 260 participants each month.
Farrell, 2014B [90] Cancer/multiple Brief description (1 paragraph) and link to website relating to CBI 9. Research to Reality Mentorship Program
No evaluative data reported.
Farrell, 2014C [90] Cancer/multiple Brief description (1 paragraph) and link to website relating to CBI 10. Advanced Topics in Implementation Science
No evaluative data reported.
Gonzalez, 2012A [104] Non-specific/multiple Description in text, and online supplementary file (case study for the CBI) 11. Translating Evidence into Practice – Implementation & Dissemination courses
No evaluative data reported but article states that approximately twenty scholars participated substantively in the IDS
curriculum (completing multiple IDS-specific courses and initiating IDS research projects).
Gonzalez, 2012B [104] Non-specific/multiple Brief description (1 paragraph) in text 12. Implementation & Dissemination Training
No evaluative data reported.
Gonzalez, 2012C [104] Non-specific/multiple Highlighted as a relevant CBI relating to D&I in a table 13. IDS Grant Writing Course
No evaluative data reported.
Goodenough, 2013 [49] Dementia/multiple Description in text, table of content and learning objectives and link to a website 14. Knowledge Translation Workshop
This was a one-day workshop on KT in dementia but also included a seminar on KT methods and practices (which the article focusses on). Article states the response rate for the evaluation survey but does not state the number of delegates that attended the KT seminar. Delegates were emailed a survey 6-months post-workshop. Results were compared between those that did and did not opt for the KT seminar as part of the workshop. The KT group reported the highest median number of overall uses of workshop information in daily practice when compared to those that only participated in the clinical seminars - 7.5 vs. 6 (p > 0.05). There was a correlation (p < 0.05) between the total number of ‘kinds of research use’ (e.g. changed a practice, changed your beliefs) and individual mean scores (average across 5 items) for conceptual research uses (e.g. ‘gave you knowledge on how to care for residents’), and this was stronger for those that attended the KT seminar. Three items stood out—changing a practice, changing a procedure and creating a new policy/guideline. Six separate one-day workshops were held in total.
Greenhalgh, 2006 [105] Primary health/multiple Description in text 15. Getting Research into Practice and Policy
Article briefly summarises students’ evaluations—students highlighted that the online environment provided the opportunity to rehearse and modify potential implementation scenarios of knowledge into practice, the asynchronous nature of the virtual discussions (vs. synchronous) provides more opportunity for reflection and the record of text messages means they have a permanent record of information to refer back to.
Jones, 2015 [91] Public health/multiple Brief description in text (1 paragraph) 16. Knowledge Translation for Researchers
A pilot half-day course was delivered and evaluated prior to the 1-day course being developed (the 1-day course has been delivered twice). No data is provided but the article states that the course has been well received and is relevant and useful to a range of researchers.
Kho, 2009 [106] Non-specific/multiple Description in text, table summary of the curriculum, appendix on the small group task, online supplementary files on daily programme and curriculum and small group project and slide deck and responses to essay questions 17. Knowledge Translation Summer Institute
In total, 30 applicants were offered a place on the training. Article focusses more on ‘lessons learnt’ than evaluative data per se but states the CBI provided many invaluable opportunities for attendees, in that all attendees expressed an interest in maintaining relationships, being updated with each other’s work and participating in future training opportunities. This CBI is also described in Leung, 2010 and is linked to Straus, 2011C.
Leung, 2010 [107] Non-specific/multiple Description in text, reference to a publication, appendices on an overview of the CBI, description of the case study, end-of-grant KT plan and guiding questions for group discussions 17. End-of Grant KT Plan (part of KTSI above)
Article focussses mainly on challenges and recommendations of end-of-grant KT plans. Feedback from attendees and KT experts was that the session was too complex for what would be a small component of the grant proposal. However, the KT experts also emphasised the importance of including a KT component in the grant to increase the likelihood of a successful grant application (see Kho, 2009 and Straus, 2011C).
Luke, 2016 [92] Mental health/multiple Description in text and reference to another publication 2. Implementation Research Institute
Article reports that 43 fellows in four cohorts have been trained (the breakdown of numbers in each cohort is reported in Baumann, 2019). This article focusses on the mentoring component of the CBI. Mentoring was positively and significantly related to having scientific collaboration 2 years later, including new research, grant submissions and publications. For every additional mentoring relationship that was established, the likelihood of scientific collaboration increased by nearly 7%. CBI also discussed in Brownson, 2017A, Luke, 2016 and Proctor, 2013.
Marriott, 2016 [93] Non-specific/NR Description in text 18. Implementation Development Workshops
Between 2011 and 2015, 72 members participated in at least one workshop (number of attendees in each workshop is not clear). 40 participated in face-to-face only, 16 in virtual only, and 16 in both formats. The focus of the article was to compare F-F vs. virtual format for implementation science training. Both were found to be equally acceptable and were effective for collaboration and growth and success in obtaining grants. A third of presenters received funding for their proposals and more than 80% of presenters said they would present again.
Means, 2016 [94] Global health/NR Brief description (1 paragraph) 19. Implementation Science and Health Metrics
No evaluative data provided.
Meissner, 2013 [48] Non-specific/multiple Description in text and list of faculty and daily curriculum 20. Training in Dissemination and Implementation Research in Health (TIDIRH)
Thirty-five applicants were accepted on the course. Attendees rated CBI as ‘very helpful’. A 6-month follow-up survey (97% response rate) revealed 72% had initiated a new grant proposal, 28% had received funding and 77% had used skills from TIDIRH to influence peers about dissemination & implementation research, build research networks, organise presentations and teaching and lead interdisciplinary teams to conduct D&I research. CBI also discussed in Vinson, 2019.
Moore, 2018 [58] Non-specific/
Multiple
Description in text, link to a website, table of course structure and delivery and online supplementary file on core competencies 21. Practising Knowledge Translation
Seventeen participants were enrolled on the course. Data were collected at 3, 6 and 12 months. Attendees reported significant positive effects in terms of—increased use of implementing theories, models and frameworks and increased knowledge of developing evidence-informed programmes, evidence implementation, evidence evaluation, sustainability scale and spread and context assessment (with self-efficacy increasing across these measures too).
Moore, 2013 [108] Nursing/nurses Description in text and table of competencies 22. EBP 11: Evaluating and Applying Evidence
Numbers on the course have increased steadily from 2009 (32) to 2013 (64). No specific evaluation data relating to the EBP 11 module was provided. Students rated the overall CBI highly and identified several strengths, including – exposure to different research article critique instruments and group interactions.
Morrato, 2015 [95] Non-specific/multiple Description in text, link to website, tables on agenda, faculty and D&I CBI resources 23. Introduction to Dissemination and Implementation
Sixty-eight delegates attended day one and 11 also attended the half-day on day two (which was optional). Data collected 1 week after the CBI (from 34/68 responses) revealed that: 100% ‘strongly agreed’ they were satisfied with the training and 97% felt the workbook was a valuable resource. Delegates that attended the day 2 mentoring session ‘strongly agreed’ that working closely with faculty/experts increased their confidence. At 6-month follow-up, evidence of 23 new manuscripts and grant proposals were found.
Norton, 2014 [96] Public health/multiple Description in text and a table of weekly topics 24. Dissemination and Implementation in Health
A total of 24 students enrolled in the course and 19 faculty researchers participated. Students strongly agreed that they would recommend the course to other students, they enjoyed it and were able to apply what they learnt to their D&I project. Faculty rated it highly too and strongly agreed that they would recommend participation in the course to other faculty and were interested in learning about D&I from students. The collaborative learning projects were rated by both as one of the most valuable aspects. Suggestions for improvement centred on (for students) course logistics, more meetings to discuss collaborative project, more time from start of course to when they meet faculty partners. Faculty reported the need for clearer expectations for the collaborative learning project and the opportunity to attend lectures.
Osanjo, 2016 [97] Non-specific/multiple Description in text and a table on curriculum 25. Implementation Science Fellowship Program
There were 5 trainees in the two cohorts that undertook the course. A survey (in years 1 and 2) revealed a high degree of satisfaction with most aspect of the CBI including content, duration and attachment sites. Fellows expressed high satisfaction with the mentorship program and would prefer the existing mentorship arrangement to be extended. Some fellows indicated they were already applying the skills gained at their home institutions. Fellows have embarked on PhD programmes in dissemination and implementation (N = 4), secured funding (N = 3) and most (85%) identify implementation science as a component of their work activity.
Padek, 2018 [98] Cancer/multiple Description in text, tables on faculty and mentoring, weblink to training, and an online supplementary file on the agenda 3. The Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC)
On average 14 fellows attend the training each year and from 2014 to2017, 56 fellows have participated. Forty-three dissemination and implementation science competencies were assessed—all improved from baseline to 6 months and 18 months. The effect was apparent across beginner, intermediate, and advanced fellows. Mentoring was rated very highly by fellows (and more highly than by the mentors). The importance of different mentoring activities was linked to fellows’ satisfaction with the mentoring activities. CBI also discussed in Brownson, 2017B.
Park, 2018 [99] Non-specific/multiple Description in text, and an online supplementary file on the agenda 26. Foundations in Knowledge Translation
A total of 46 participants across two cohorts have completed the training (16 teams ranging in size from 2–4 people). Surveys (at 6, 12, 18, 24 months) revealed attendees’ self-efficacy in evidence-based practices, KT activities, and using evidence to inform practice increased over time. Focus groups and interviews indicated that confidence in using KT increased from baseline to 24 months and that training helped to achieve attendees’ KT objectives, plan their projects and solve problems. Teams with high self-reported capacity and commitment to implement projects and ‘buy-in’ from upper management that resulted in securing funding and resources were stated as important to achieve goals. Sustained spread of KT practice was observed with 5 teams at 24 months.
Proctor, 2013 [109] Mental health/multiple Description in text 2. Implementation Research Institute (IRI)
Article states that 10 fellows are selected each year. Fellows were very satisfied with the program and would recommend it to colleagues. Fellows and faculty rated the calibre of their counterparts as ‘excellent’. Fellows from the first 3 cohorts have submitted 74 proposals (52 funded) and are beginning to serve as mentors for more junior investigators. A total of 208 publications have been submitted/published (7.64 per fellow) as well as conference presentations and teaching. CBI also discussed in Baumann, 2019, Brownson, 2017A and Luke, 2016.
Proctor, 2019 [100] Behavioural health/multiple Description in text and table on curriculum 27. Training in Implementation Practice Leadership (TRIPLE)
Sixteen mid-level leaders were enrolled in the training. Most attendees reported increased implementation leadership skills (86%) and implementation climate (79%) after the training (p < 0.05). Implementation leadership skills improved most on the proactive and knowledgeable subscales. For implementation climate, educational support and recognition for using evidence-based practice revealed the greatest increase (post training). Attendees found the training highly acceptable and appropriate and qualitative results indicated that training led to increased organisational implementation as well as leadership skills for attendees.
Ramaswamy, 2019 [62] Public health/multiple Description in text, table on courses and descriptions and an online supplementary files on course syllabi, alignment of CBI with competencies 28. Applied Implementation Science
As of April 2018, a total of 11 sections of the course have been offered, with a total enrolment of 142 (127 of whom were MPH students). Taking the 4 courses collectively, students’ qualitative feedback was positive (e.g. ‘useful tools for the application of implementation science’, ‘practical and allows you to build real skills’). The degree to which students had applied what they had learned was supported by 8 students embarking on practicums, masters papers and other implementation science-related learning projects.
Riner, 2015 [101] Nursing/nurses Description in text and table of competencies 29. Evidence-based Research and Translational Science: Inquiry 11
No evaluative data provided.
Straus, 2011A [110] Non-specific/multiple Brief description (1 paragraph), table of topics and a link to a website 30. Knowledge Translation Seminars (stream 1)
No evaluative data provided.
Straus, 2011B [110] Non-specific/multiple Brief description (1 paragraph) and a link to a website 31. Research Operations (stream 1)
No evaluative data provided.
Straus, 2011C [110] Non-specific/multiple Description in text and a table of themes for CBI 17. Knowledge Translation Summer Institute (stream 1)
To date (2011) three summer institutes have been held with 90 trainees in total. No specific evaluative data from trainees but article reports that trainees have been involved in 3 publications, the preparation of collaborative multi-site grants and projects and have worked together on education modules and presentations. CBI also described in Kho, 2009 and linked to Leung, 2010.
Straus, 2011D [110] Non-specific/multiple Brief description (< a paragraph) 32. Introduction to Evidence-based Medicine (stream 1)
No evaluative data provided.
Straus, 2011E [110] Non-specific/multiple Brief description (< a paragraph) 33. Introduction to Systematic Reviews (stream 1)
No evaluative data provided.
Straus, 2011F [110] Non-specific/multiple Brief description (< a paragraph) ) 34. Pragmatic KT Trials (stream 1)
No evaluative data provided.
Straus, 2011G [110] Non-specific/multiple Brief description (1 paragraph) and a link to website 35. End of Grant KT (Stream 1)
No evaluative data provided. This CBI is linked to Kho, 2009 and Straus 2011C,
Straus, 2011H [110] Non-specific/multiple Brief description (1 paragraph) and a link to website 36. Integrated KT (stream 2)
No evaluative data provided.
Straus, 2011I [110] Non-specific/multiple Brief description (1 paragraph) 37. End of Grant KT (Stream 2)
No evaluative data provided.
Straus, 2011J [110] Non-specific/multiple Brief description (1 paragraph) 38. Introduction to KT (Stream 3)
No evaluative data provided.
Straus, 2011K [110] Non-specific/multiple Brief description (1 paragraph) 39. Basics of KT (stream 3)
Article does not provide evaluative data but does state that this CBI has been held on two occasions including colleagues from 16 teams.
Ulrich, 2017 [102] Non-specific/multiple Description in text, table of curriculum, website link to MSc module manual 40. Master of Science in Health Service Research and Implementation Science
The first cohort of students had 13 students, and the second cohort had 23 students. Article provides data on expectations (from the perspective of students, experts and teaching staff) of what should be included in the course, rather than their evaluations of the course per se. 27/42 of the competencies were felt to be crucial or very important by more than 80% of participants. 6/8 items that individuals rated as very important specifically related to implementation in practice were in this category, e.g. knowledge of implementation strategies and barriers and enablers to implementation.
Vinson, 2019 [103] Non-specific/multiple Description in text, table of course content, reference to another publication on the CBI 20. Training in Dissemination and Implementation Research in Health (TIDIRH)
In total 197 trainees have undertaken the training between 2011-2015. Article evaluated long-term impact on trainees that attended one of the TIDIRH’s over a 5-year period (TIDIRH held once annually). Selected applicants were compared to unselected applicants for applications for NIH peer-reviewed funding. A survey of trainees and unselected applicants as well as a faculty survey was conducted. Thirty-eight per cent of trainees stated they had extensive contact with faculty following training and a further 38% indicated they had at least limited contact. Twenty-four per cent had extensive collaborations with other fellows after the training and 43% had at least limited contact. Overall trainees submitted more funding applications than unselected applicants and had better funding outcomes (25% vs. 19%). CBI also discussed in Meissner, 2013.
Wahabi, 2011 [111] Family medicine/medics Description in text, table of CBI contents, online supplementary files on CBI format and objectives and project objectives 41. Advanced Evidence-Based Healthcare
Twenty-one participants attended the workshop. Participants indicated that the ‘debate approach’ added a new dimension to their evidence-based medicine skills by adding purpose and motivation but that their performance would have been better if they had been offered a practical demonstration of how to conduct a debate. The KT project enhanced understanding of the relationship between evidence and implementation, however, some maintained this fell out the scope of the role of the doctor.
  1. Text in bold denotes relevant information about the CBI that was not described in the article but was described in another article in the review that focussed on the same CBI—the linked article(s) are highlighted in bold in the ‘description’ column.
  2. Where articles are numbered ‘A’, ‘B’ (e.g. Brownson, 2017A, 2017B), these are CBIs that are discussed in the same article which are distinct from one another so are included as separate entries in the table. Each CBI is given a unique identifier to show which findings refer to each CBI.
  3. Whenever possible, we have provided information on the CBIs—the level of detail in Table 1 varies depending on what was reported in the article. Cells that are coded as ‘NR’ = when information is not explicitly reported.
  4. aThe types of information provided on the CBI are listed in the table—this is only a high-level summary and should not be used as an indicator of article quality (the content and structure of these CBIs will be examined in follow-up work)
  5. bA high-level summary of evaluative findings on the CBI (where reported) is provided—a more detailed analysis will be conducted in follow-up work.