Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Assessing citation networks for dissemination and implementation research frameworks

  • Ted A. Skolarus1, 2, 3Email author,
  • Todd Lehmann4,
  • Rachel G. Tabak4,
  • Jenine Harris5,
  • Jesse Lecy6 and
  • Anne E. Sales1, 7
Implementation Science201712:97

https://doi.org/10.1186/s13012-017-0628-2

Received: 17 February 2017

Accepted: 13 July 2017

Published: 28 July 2017

Abstract

Background

A recent review of frameworks used in dissemination and implementation (D&I) science described 61 judged to be related either to dissemination, implementation, or both. The current use of these frameworks and their contributions to D&I science more broadly has yet to be reviewed. For these reasons, our objective was to determine the role of these frameworks in the development of D&I science.

Methods

We used the Web of Science™ Core Collection and Google Scholar™ to conduct a citation network analysis for the key frameworks described in a recent systematic review of D&I frameworks (Am J Prev Med 43(3):337–350, 2012). From January to August 2016, we collected framework data including title, reference, publication year, and citations per year and conducted descriptive and main path network analyses to identify those most important in holding the current citation network for D&I frameworks together.

Results

The source article contained 119 cited references, with 50 published articles and 11 documents identified as a primary framework reference. The average citations per year for the 61 frameworks reviewed ranged from 0.7 to 103.3 among articles published from 1985 to 2012. Citation rates from all frameworks are reported with citation network analyses for the framework review article and ten highly cited framework seed articles. The main path for the D&I framework citation network is presented.

Conclusions

We examined citation rates and the main paths through the citation network to delineate the current landscape of D&I framework research, and opportunities for advancing framework development and use. Dissemination and implementation researchers and practitioners may consider frequency of framework citation and our network findings when planning implementation efforts to build upon this foundation and promote systematic advances in D&I science.

Keywords

Network analysis Knowledge translation Management science Model Implementation science Bibliometrics Quality improvement Behavioral theory

Background

The field of dissemination and implementation (D&I) science continues to evolve with contributions from a variety of disciplines, researchers, and institutions across the globe [1]. Significant advances in our understanding of how to conceptualize D&I research and practice were facilitated by a recent comprehensive review of relevant models, theories, and frameworks [2]. The review identified 61 frameworks to guide D&I researchers and practitioners in their research-to-practice activities at different socio-ecologic levels within the health care system (individual, organization, community, healthcare system, policy). The goal was to develop a D&I framework inventory to inform selection efforts for researchers and practitioners based on a given framework’s construct flexibility, its predilection for dissemination and/or implementation activities, as well as its socio-ecologic level targeting.

However, better understanding the most frequently cited D&I frameworks and the citation networks surrounding these frameworks can also provide useful information for selection, conceptualization, and resources for operationalization. For example, in cases where several different frameworks might be applicable to a given implementation intervention, identifying the most prominent and commonly applied frameworks in the field could have several advantages. First, it could provide researchers and practitioners with the most supporting literature to inform their effort. Second, accessing this information may increase the chances of intervention success and therefore help the best frameworks emerge. Third, as the framework literature evolves, there will be increasing opportunities to advance D&I science with respect to fidelity of framework use, core framework components, standardized measurement, advantages and disadvantages of a given framework, and ultimately implementation outcomes [3]. More broadly, mapping D&I framework networks can build upon this foundation to promote systematic advances in D&I science through identifying the common set of assumptions and knowledge that constitutes consensus in the field.

Bibliometric (or citation) analysis is one method to investigate the scholarly landscape surrounding D&I frameworks from the review. This quantitative technique is increasingly applied to measure the impact of academic research and examine relationships using tools such as citation network analysis [46]. In general, citation network analysis provides a map of the most highly cited publications within a given research domain, much like the way Google™ uses page rank to identify the most relevant websites [7]. This approach to understanding the state of scientific advancement has been used across a range of fields, including public administration, public health service systems, physical activity environments, and analytic method development, to discern the degree to which information flows through a scholarly network and identify opportunities for transdisciplinary collaboration and crosstalk [814]. Using citation analysis to examine the rapidly evolving D&I field could not only indicate the most frequently cited D&I frameworks but also determine their relationships across time and discipline, and map the emerging knowledge network constituting the D&I framework field.

For these reasons, we conducted a citation network analysis of D&I research frameworks. We created a snapshot of the scientific development of D&I framework research based on carefully selected framework articles followed forward in time as they integrated into the growing body of D&I knowledge. We examined citation rates and the main paths through the citation network to delineate the current landscape of D&I framework research, and opportunities for advancing framework development and use.

Methods

Citation network analysis

We used a citation data network collection tool, the Citation Network Analyzer (CNA), to generate the data and conduct our study [15, 16]. This tool uses a constrained snowball sampling approach to identify a network of documents (i.e., journal and conference papers, theses and dissertations, academic books, pre-prints, abstracts, technical reports) in Google Scholar™ that can be used for descriptive, main path, and other network analyses via an R software package. In general, a constrained snowball sample of academic publications is created by identifying seed articles, determining the levels of data (articles that cite the seeds, articles that cite those, and so on), and selecting the sampling rate at each level. This vetted, efficient and inclusive networking approach to following citations forward in time is uniquely suited to advance our current understanding of the literature surrounding D&I framework development and use. In addition, the output from the CNA tool can be used to graphically represent the citation network and assign weights to the articles based on their importance in maintaining the network architecture as described below.

Our approach of using citation network analysis to conduct structured literature reviews was based on prior work using the CNA tool [9, 10, 13, 15, 17]. This approach can lead to a less biased assessment of the academic literature than traditional narrative reviews for at least two reasons. First, a citation analysis approach can avoid the cognitive bias associated with traditional literature searches using keyword searches which may be limited by the researcher expertise, training, and preferences. Second, the use of Google Scholar™ and a snowball sampling technique based on selected seed articles, rather than Web of Science™ citation tools based on keywords for instance, is able to survey a broader scope of publications that may be relevant to D&I frameworks especially given their expansive roots in fields ranging from agriculture, business, and political science to public health and medicine [18, 19]. In addition, the CNA tool allows for a constrained approach to snowball sampling, rather than traditional snowball sampling where the sample grows exponentially, in order to limit the articles at each level from the seed article to arrive at empirical findings using a fraction of the data [15].

As detailed in Additional file 1 , we conducted two analyses using this novel approach. First, we synthesized the literature covered in the framework review article by Tabak et al. [2] with respect to recent citations and performed a structured literature review of the article itself. Next, we applied a structured literature review to a snowball sample of ten framework articles identified as the most important by the study team, largely based on the Tabak review. Overall, this work allowed us to understand the relevance of the framework review article as a D&I resource and to identify those frameworks forming the current backbone of the D&I framework field (i.e., framework articles in the network’s main path).

Characterizing the Tabak et al. framework review article and its citation network

The Tabak systematic review contained 119 references, with 50 published articles and 11 documents (reports/chapters/books) identified as a primary D&I framework reference (n = 61) [2]. These D&I frameworks were identified first through selecting commonly cited frameworks, then through snowball sampling and expert consultation including with U.S. National Institute of Health offıcials who process and review D&I grants. Frameworks were excluded from the review according to the following criteria: (1) focused on practitioner rather than D&I researcher; (2) applied to individual behavior change only (i.e., without ties to local, organizational or community dissemination); (3) intended only for national level use versus local, community, or organizational level; (4) frameworks focused only on dissemination after research study completion; and (5) articles not written in the English language. The frameworks were then judged by the authors to be related either more to dissemination, implementation, or both equally. Each framework’s construct flexibility was rated as broad and flexible versus operational and defined for a given context and activity. Last, the socio-ecologic level (individual, organization, community, healthcare system, policy) targeted by the framework was categorized, with most operating at more than one level.

We extracted the primary citation for each framework. In cases where more than one primary reference was used (n = 21), we selected the most relevant reference, usually the oldest, as the primary reference. The primary references for 11 frameworks were reports, chapters, or books. Because peer-reviewed articles were the most common documents cited in this study, we use the term article to denote all documents throughout the remainder of the manuscript.

To better understand the framework articles discussed in the Tabak review, we conducted descriptive analyses to identify the most common journals, authors, and countries of origin for the 61 models. We also examined the citation rates for each framework. We defined a citation rate as the number of citations/year(s) since publication. We used the Web of Science™ Core Collection in January 2016 to conduct these descriptive citation analyses and inform our subsequent network analysis described in the Additional file 1 .

Citation network analysis of selected D&I frameworks

Next, we conducted a citation network analysis of ten carefully selected D&I framework articles we felt reflected the current state of the field. Eight of these were based on citation rates and the Tabak review. However, we also included two additional frameworks given their relevance to implementation science and relatively high citation rates: (1) Theoretical Domains Framework (TDF) [20, 21] and the (2) Knowledge to Action Framework (KTA) [22], for a total of ten seed articles for our next citation network analysis. Both of these models were developed by researchers outside the USA and were not included in the Tabak review. The details of the D&I framework citation analysis are included in the Additional file 1 .

Last, we performed a main path analysis to identify the connectedness and links among the articles considered to be the backbone of the D&I framework citation network. This approach identifies the key articles influencing D&I models based on the selected seed articles. We determined the traversal weights indicating the proportion of network paths that included a given article node in the network [23]. For instance, a traversal weight of 0.25 for framework X indicates that its article exists in 25% of the citation paths in the network. This traversal weight indicates the importance of any particular node (i.e., article) in the network. We constructed the main path by removing all ties in the network scoring below the 95% percentile for traversal weight value. We normalized the traversal weights according to flow using the Search Path Count method [24]. All computations were accomplished with Pajek [23].

All analyses were conducted between January 2016 and August 2016. This study was deemed not regulated by the Institutional Review Board at the University of Michigan.

Results

Tabak framework review article and its citation network

As illustrated in Fig. 1, the Tabak framework review article is an increasingly cited resource. As of January 2016, it had been cumulatively cited 456 times across 388 articles and other source items indexed within Web of Science™ Core Collection. As shown in Table 1, there was a broad distribution of citation numbers and annual citation rates across the 61 framework articles within the Tabak review and our two selected framework articles (KTA and TDF). The average number of citations per year ranged from 0 to 1949 among articles published from 1962 to 2012. The outlier with the highest citation rate was a book reference for Rogers’ Diffusion of Innovations.
Fig. 1

Citation report through 2015 for ‘Bridging Research and Practice Models for Dissemination and Implementation Research’ by Tabak et al. [2]

Table 1

Citations for D&I frameworks in published articles as of January 2016

No.

Framework—manuscripts

Web of Science™ Core Collection Citations

Citations/year

Year

Country

Most relevant framework reference, usually oldest

-

a Knowledge to Action Framework b

1445

160.6

2006

Canada

Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006 Winter;26(1):13–24 [22].

1

aA Conceptual Model for the Diffusion of Innovations in Service Organizations

1136

103.3

2004

UK

Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82(4):581–629 [27].

2

aSticky Knowledge

1949

102.6

1996

USA

Szulanski G. Exploring internal stickiness: impediments to the transfer of best practice within the fırm. Strat Manag J 1996;17:27–43 [37].

-

a Theoretical Domains Framework b

613

61.3

2005

UK

Michie S, Johnston M, Abraham C, et al. “Psychological Theory” Group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005 Feb;14(1):26–33 [38].

3

aThe RE-AIM Framework

731

45.7

1999

USA

Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999;89(9):1322–7 [30].

4

aConsolidated Framework for Implementation Research

257

42.8

2009

USA

Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research fındings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50 [28].

5

aConceptual Model of Evidence-Based Practice Implementation in Public Service Sectors

118

29.5

2011

USA

Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 2011;38(1):4–23 [31].

6

aConceptual Model of Implementation Research

147

24.5

2009

USA

Proctor EK, Landsverk J, Aarons G, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health 2009;36(1):24–34 [29].

7

aImplementation Effectiveness Model

426

22.4

1996

USA

Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manag Rev. 1996:1055–80 [26]

8

aPromoting Action on Research Implementation in Health Services

379

22.3

1998

UK

Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care 1998;7(3):149 [25].

9

Research Knowledge Infrastructure

263

21.9

2003

Canada

Lavis JN, Robertson D, Woodside JM, et al. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003;81(2):221–48 [39].

10

Interactive Systems Framework

143

20.4

2008

USA

Wandersman A, Duffy J, Flaspohler P, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Comm Psych 2008;41(3–4):171–81 [40].

11

Utilization-Focused Surveillance Framework

110

18.3

2009

USA

Green LW, Ottoson JM, Garcia C, et al. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev. Public Health 2009;30:151–74 [41].

12

Normalization Process Theory

98

16.3

2009

UK

May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Soc J Br Soc Assoc 2009;43(3):535–54 [42].

13

Multi-level Conceptual Framework of Organizational Innovation Adoption

177

13.6

2002

Netherlands

Frambach RT, Schillewaert N. Organizational innovation adoption: a multi-level framework of determinants and opportunities for future research. J Business Res 2002;55(2):163–76 [43].

14

Davis’ Pathman-PRECEED Model

211

11.1

1996

USA

Pathman DE, Konrad TR, Freed GL, et al. The awareness-to-adherence model of the steps to clinical guideline compliance: the case of pediatric vaccine recommendations. Med Care 1996;34(9):873 [44].

15

Pronovost’s 4E’s Process Theory

76

10.9

2008

USA

Pronovost PJ, Berenholtz SM, Needham DM. Translating evidence into practice: a model for large scale knowledge translation. BMJ 2008;337:a1714 [45].

16

Knowledge Exchange Framework

64

10.7

2009

UK

Ward V, House A, Hamer S. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. J Health Serv Res Policy 2009;14(3):156–64 [46].

17

Framework of Dissemination in Health Services Intervention Research

73

10.4

2008

USA

Mendel P, Meredith LS, Schoenbaum M, et al. Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in hlth srvcs rsrch. Adm Pol Ment Hlth 2008;35(1–2):21–37 [47].

18

A Framework for Analyzing Adoption of Complex Health Innovations

52

10.4

2010

UK

Atun, Rifat, de Jongh, Thyra, Secci, Federica, et al. Integration of targeted health interventions into health systems: a conceptual framework for analysis. HEALTH POLICY AND PLANNING Vol: 25, Iss: 2 Pgs: 104–111, MAR 2010 [48]

19

Pathways to Evidence Informed Policy

102

10.2

2005

Australia

Bowen S, Zwi AB. Pathways to “evidence-informed” policy and practice: a framework for action. PLoS Med 2005;2(7):e166 [49].

20

Availability, Responsiveness & Continuity (ARC): An Organizational & Community

92

9.2

2005

USA

Glisson C, Schoenwald SK. The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Ment Health Serv Res 2005;7(4):243–59 [50].

21

Practical, Robust Implementation and Sustainability Model (PRISM)

54

7.7

2008

USA

Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research fındings into practice. Jt Comm J Qual Patient Saf 2008;34(4):228–43 [51].

22

An Organizational Theory of Innovation Implementation

43

7.2

2009

USA

Weiner BJ, Lewis MA, Linnan LA. Using organization theory to understand the determinants of effective implementation of worksite health promotion programs. Health Educ Res 2009;24(2):292–305 [52].

23

Ottawa Model of Research Use

103

6.1

1998

Canada

Logan J, Graham ID. Toward a comprehensive interdisciplinary model of health care research use. Sci Commun 1998;20(2):227 [53].

24

Policy Framework for Increasing Diffusion of Evidence-Based Physical Activity Interventions

54

6.0

2006

Australia

Owen N, Glanz K, Sallis JF, Kelder SH. Evidence-based approaches to dissemination and diffusion of physical activity interventions. Am J Prev Med 2006;31(4S):S35–S44 [54].

25

Replicating Effective Programs Plus Framework

43

5.4

2007

USA

Kilbourne AM, Neumann MS, Pincus HA, et al. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci 2007;2:42 [55].

26

Framework for Knowledge Translation

62

5.2

2003

Canada

Jacobson N, Butterill D, Goering P. Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy 2003;8(2):94–9 [56].

27

Collaborative Model for Knowledge Translation Between Research and Practice Settings

33

4.7

2008

Canada

Baumbusch JL, Kirkham SR, Khan KB, et al. Pursuing common agendas: a collaborative model for knowledge translation between research and practice in clinical settings. Res Nurs Health 2008; 31(2):130–40 [57].

28

A Convergent Diffusion and Social Marketing Approach for Dissemination

34

3.8

2006

USA

Dearing JW, Maibach EW, Buller DB. A convergent diffusion and social marketing approach for disseminating proven approaches to physical activity promotion. Am J Prev Med 2006;31(4S):S11–S23 [58].

29

Framework for the Dissemination & Utilization of Research for Health-Care Policy & Practice

47

3.6

2002

Canada

Dobbins M, Ciliska D, Cockerill R, et al. A framework for the dissemination and utilization of research for health-care policy and practice. Online J Knowl Synth Nurs 2002;9:7 [59].

30

Push–Pull Capacity Model

32

3.6

2006

USA

Green LW, Orleans CT, Ottoson JM, et al. Inferring strategies for disseminating physical activity policies, programs, and practices from the successes of tobacco control. Am J Prev Med 2006;31(4S):S66 –S81 [60].

31

Critical Realism & the Arts Research Utilization Model (CRARUM)

17

2.8

2009

Canada

Kontos PC, Poland BD. Mapping new theoretical and methodological terrain for knowledge translation: contributions from critical realism and the arts. Implement Sci 2009;4:1 [61].

32

Coordinated Implementation Model

62

2.8

1993

Canada

Lomas J. Retailing research: increasing the role of evidence in clinical services for childbirth. Milbank Q 1993;71(3):439–75 [62].

33

Knowledge Translation Model of Tehran University of Medical Sciences

17

2.4

2008

Iran

Majdzadeh R, Sadighi J, Nejat S, Mahani AS, Gholami J. Knowledge translation for research utilization: design of a knowledge translation model at Tehran University of Medical Sciences. J Cont Educ Health Prof 2008;28(4):270–7 [63].

34

Dissemination of Evidence-based Interventions to Prevent Obesity

6

2.0

2012

USA

Dreisinger ML, Boland EM, Filler CD, Baker EA, Hessel AS, Brownson RC. Contextual factors influencing readiness for dissemination of obesity prevention programs and policies. Health Educ Res 2012;27(2):292–306 [64].

35

OPTIONS Model

33

1.9

1998

Canada

Martin GW, Herie MA, Turner BJ, Cunningham JA. A social marketing model for disseminating research-based treatments to addictions treatment providers. Addiction 1998;93(11):1703–15 [65].

36

Conceptualizing Dissemination Research and Activity: Canadian Heart Health Initiative

23

1.9

2003

Canada

Elliott SJ, O’Loughlin J, Robinson K, et al. Conceptualizing dissemination research and activity: the case of the Canadian Heart Health Initiative. Health Educ Behav 2003;30(3):267–82; discussion 283–6 [66].

38

Conceptual Framework for Research Knowledge Transfer and Utilization

22

1.8

2003

Canada

Kramer DM, Cole DC. Sustained, intensive engagement to promote health and safety knowledge transfer to and utilization by workplaces. Sci Commun 2003;25(1):56 [67].

37

“4E” Framework for Knowledge Dissemination and Utilization

22

1.8

2003

USA

Farkas M, Jette AM, Tennstedt S, Haley SM, Quinn V. Knowledge dissemination and utilization in gerontology: an organizing framework. Gerontologist 2003;43(S1):47 [68].

39

Linking Systems Framework

18

1.8

2005

Canada

Robinson K, Elliott SJ, Driedger SM, et al. Using linking systems to build capacity and enhance dissemination in heart health promotion: a Canadian multiple-case study. Health Educ Res 2005;20(5): 499–513 [69].

40

Blueprint for Dissemination

9

1.8

2010

USA

Yuan CT, Nembhard IM, Stern AF, Brush JE Jr., Krumholz HM, Bradley EH. Blueprint for the dissemination of evidence-based practices in health care. Issue Brief (Commonw Fund) 2010;86:1–16 [70].

41

Health Promotion Research Center Framework

5

1.7

2012

USA

Harris JR, Cheadle A, Hannon PA, et al. A framework for disseminating evidence-based health promotion practices. Prev Chronic Dis 2012;9:E22 [71].

42

A Framework for Spread

16

1.6

2005

USA

Nolan K, Schall MW, Erb F, et al. Using a framework for spread: The case of patient access in the Veterans Health Administration. Jt Comm J Qual Patient Saf 2005;31(6):339–47 [72].

43

Model for Locally Based Research Transfer Development

25

1.6

1999

Canada

Anderson M, Cosby J, Swan B, et al. The use of research in local health service agencies. Soc Sci Med 1999; 49(8):1007–19 [73].

44

A Six-Step Framework For International Physical Activity Dissemination

14

1.6

2006

Australia

Bauman AE, Nelson DE, Pratt M, et al. Dissemination of physical activity evidence, programs, policies, and surveillance in the international public health arena. Am J Prev Med 2006;31(4S):S57–S65 [74].

45

CDC DHAP’s Research-to-Practice Framework

23

1.5

2000

USA

Sogolow ED, Kay LS, Doll LS, et al. Strengthening HIV prevention: application of a research-to-practice framework. AIDS Educ Prev 2000;12(5S):21–32 [75].

46

Health Promotion Technology Transfer Process

28

1.5

1996

USA

Orlandi MA. Health promotion technology transfer: organizational perspectives. Can J Public Health 1996;87(S2):S28 –S33 [76].

47

RAND Model of Persuasive Communication and Diffusion of Communication and Medical Innovation

31

1.0

1985

USA

Winkler JD, Lohr KN, Brook RH. Persuasive communication and medical technology assessment. Arch Intern Med 1985;145(2):314–7 [77].

48

A Conceptual Model of Knowledge Utilization

21

1.0

1993

USA

Lester JP. The utilization of policy analysis by state agency offıcials. Sci Commun 1993;14(3):267 [78].

49

Model for Improving the Dissemination of Nursing Research

19

0.7

1989

USA

Funk SG, Tornquist EM, Champagne MT. A model for improving the dissemination of nursing research. West J Nurs Res 1989;11(3): 361–72 [79].

50

Effective Dissemination Strategies

9

0.7

2002

UK

Scullion PA. Effective dissemination strategies. Nurs Res 2002; 10(1):65–77 [80].

 

Frameworks – Other documents

51

Diffusion of Innovation

 

1144.2

1962

USA

Rogers, Everett M. (1962). Diffusion of Innovations. Glencoe: Free Press. ISBN 0–612–62,843-4 [81]

52

Streams of Policy Process

 

395.8

1984

USA

Kingdon JW. Agendas, alternatives, and public policies. Boston: Little, Brown, 1984 [82]

53

Active Implementation Framework

 

213.9

2005

USA

Fixsen DL, Naoom SF, Blasé KA, et al. Implementation research: a synthesis of the literature. Tampa FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network, 2005 [83].

54

The Precede–Proceed Model

 

73.8

2005

USA

Green LW, Kreuter MW. Health program planning: an educational and ecological approach. 4th ed. New York: McGraw-Hill, 2005 [84].

55

Research Development Dissemination and Utilization Framework

 

22.7

1969

USA

Havelock RG. Planning for innovation through dissemination and utilization of knowledge. Centre for Research on Utilization of Scientifıc Knowledge, Institute for Social Research, University of Michigan, 1969 [85].

56

Real-World Dissemination

 

14.7

1992

UK

Pettigrew AM, Ferlie E, McKee L. Shaping strategic change: making change in large organizations: the case of the National Health Service. Thousand Oaks CA: Sage Publications, 1992 [86].

57

A Framework for the Transfer of Patient Safety Research into Practice

 

3.5

2005

USA

Nieva VF, Murphy R, Ridley N, et al. From science to service: a framework for the transfer of patient safety. 2005. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in patient safety: from research to implementation (Vol. 2: Concepts and methodology). Rockville MD: Agency for Healthcare Research and Quality, 2005 [87].

58

Framework for Dissemination of Evidence-Based Policy

 

1.0

2012

USA

Dodson EA, Brownson RC, Weiss SW. Policy dissemination research. In: Brownson R, Colditz G, Proctor EK, eds. Dissemination and implementation research in health: translating science to practice. Oxford: Oxford University Press, 2012 [88].

59

Marketing and Distribution System for Public Health

 

1.0

2012

USA

Kreuter MW, Casey CM, Bernhardt JM. Enhancing dissemination though marketing and distribution systems: a vision for public health. In: Brownson RC, Colditz G, Proctor EK, eds. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press, 2012 [89].

60

Facilitating Adoption of Best Practices (FAB) Model

 

0.1

2008

USA

Damush T, Bravata DM, Plue L, et al. Facilitation of Best Practices (FAB) Framework. Stroke QUERI Center annual report. 2008 [90].

61

Interacting Elements of Integrating Science, Policy, and Practice

 

0.0

2011

USA

TIDIRH Working Group. Interacting elements of integrating science, policy, and practice. In: Training institute for dissemination and implementation research in health. Conference proceedings. Chapel Hill NC, 2011 [91].

aIncluded as one of ten seed articles for citation network analysis

b Two additional frameworks were included along with the Tabak framework review articles given their relevance to implementation science - Theoretical Domains Framework (TDF) [20] and the Knowledge to Action Framework (KTA) [22]

Based on the structured literature review of the Tabak article using the CNA tool, we identified 239 articles across the network and its three levels of ‘distance.’ This included 17 level-one articles directly referencing the Tabak article, with the remainder of articles residing two and three levels from the Tabak source article. The majority of the documents were journal articles (84%), followed by books (16%). The articles in the Tabak network were published between 2002 and 2016, with 51 articles published prior to the source article year of 2012. The majority (86%) of these were three levels from the Tabak seed article and (35%) were book references. We identified 202 unique first authors contributing to this network. Each author contributed 1.18 articles (standard deviation (SD) = 0.58), on average. Most first authors contributed only one article to the network (one = 177; two = 19, three = 3, four = 2, six = 1). We identified 123 unique journals (books excluded) contributing to the Tabak network, each providing an average of 1.62 articles (SD = 2.63). Most journals contributed one article (n = 95). The top three journals producing the most articles were: Implementation Science (n = 29), Annual Review of Public Health (n = 6), and BMC Public Health (n = 5). All other journals had four or fewer articles each. The articles in the Tabak network were cited between 0 and 4410 times. The top ten cited articles in the Tabak network are shown in Table 2, and none of which served as a primary framework reference. As illustrated in Fig. 2, there were prominent ties in the Tabak network to social care and the law by Aveyard; normalization process and general implementation theory by May; implementation work by Glasgow, Proctor, Neta, and Chambers; a gateway to broader literature via a movement science article by Peters; a Karlin article which ties in psychotherapy; and a 2013 contribution by Straus that was an introduction to knowledge translation in healthcare.
Table 2

Ten most cited articles within the Tabak framework review citation network

Title

First author

Year

Journal

Google Scholar Citations

Network Level

Doing your research project: a guide for first-time researchers [92]

Bell

2014

Book

4410

3

Research methods for sports studies [93]

Gratton

2010

Book

734

3

The utilization of health research in policy-making: concepts, examples and methods of assessment [94]

Hanney

2003

Health Research Policy and Systems

558

3

Information retrieval: a health and biomedical perspective [95]

Hersh

2008

Book

446

3

Anti-oppressive practice: social care and the law [96]

Dalrymple

2006

Book

377

3

Doing a literature review in health and social care: A practical guide [97]

Aveyard

2014

Book

341

2

Social work skills: a practice handbook [98]

Trevithick

2005

Book

318

3

Understanding social work: preparing for practice [99]

Thompson

2015

Book

236

2

Reflexivity, its meanings and relevance for social work: a critical review of the literature [100]

D’Cruz

2007

British Journal of Social Work

220

3

Synthesis of recommendations for the assessment and management of low back pain from recent clinical practice guidelines [101]

Dagenais

2010

The Spine Journal

213

3

Fig. 2

Citation network for ‘Bridging Research and Practice Models for Dissemination and Implementation Research’ by Tabak et al. [2]. Most first authors contributed only one article (one = 177). Those authors with two articles—Aarons, G; Archambault, P; Bjurlin, M; Blease, CR; Brownson, R; Chambers, D; Chor, K; Davidoff, F; Edwards, N; Gagliardi, A; Kozica, S; May, C; Naci, H; Neta, G; Page, A; Partridge, SR; Rhoades, E; Trevithick, P; Trockel, M; three articles—Aveyard, H; O’Brien, J; Proctor, E; four articles—Glasgow, R and Powell, B; and six articles—Thompson, N

Citation network analysis of selected D&I frameworks

The citation network for our seed articles highlighted in Table 1 included 355 unique documents published between 1996 and 2014. There were 302,472 citation links connecting the articles in this network. The majority of citations was from 323 journal articles (91%), followed by 29 books (8%), and 3 in-proceedings (1%). We identified 274 unique first authors, each contributing 1.30 articles (SD = 0.84), on average. The majority of first authors provided one article to the network with only six authors contributing greater than three. We also identified 128 unique journals contributing to this network, each providing an average of 2.52 articles (SD = 4.04). While many journals contributed one article (n = 29), the top five journals producing the most articles were: Strategic Management Journal (n = 29), Academy of Management Journal (n = 25), Implementation Science (n = 20), Organization Science (n = 15), and Management Science (n = 10). All other journals contributed less than ten articles each. The top ten cited articles are shown in Table 3, with Szulanski’s Sticky Knowledge as the only primary framework reference from the Tabak review. The remainder of articles tended to focus on business practices and knowledge sharing, collaboration networks, and social and/or intellectual capital. The articles for the D&I framework network contributed between 64 and 12,680 citations, with a median of 489.
Table 3

Ten most cited articles within the D&I framework citation network

Title

First author

Journal

Year

Google Scholar Citations

Social capital, intellectual capital, and the organizational advantage [102]

Nahapiet

Academy of Management Review

1998

12,680

Dynamic capabilities: what are they? [103]

Eisenhardt

Strategic Management Journal

2000

10,085

The relational view: Cooperative strategy and sources of interorganizational competitive advantage [104]

Dyer

Academy of Management Review

1998

9681

Cultivating communities of practice: A guide to managing knowledge [105]

Wenger

Book

2002

8548

Review: Knowledge management and knowledge management systems: Conceptual foundations and research issues [106]

Alavi

MIS quarterly

2001

8166

Exploring internal stickiness: Impediments to the transfer of best practice within the firm [37]

Szulanski

Strategic Management Journal

1996

7694

Absorptive capacity: A review, reconceptualization, and extension [107]

Zahra

Academy of Management Review

2002

6194

The search-transfer problem: The role of weak ties in sharing knowledge across organization subunits [108]

Hansen

Administrative Science Quarterly

1999

5528

Collaboration networks, structural holes, and innovation: A longitudinal study [109]

Ahuja

Administrative Science Quarterly

2000

4140

Creating and managing a high performance knowledge-sharing network: the Toyota case [110]

Dyer

Strategic Management Journal

2000

3509

As illustrated in Fig. 3, the D&I framework citation network appears centered around the 2004 Greenhalgh et al. article with prominent ties to the Theoretical Domains Framework, the Knowledge to Action Framework, the Promoting Action on Research Implementation in Health Services Framework (PARiHS), the Consolidated Framework for Implementation Research (CFIR), and an article conceptualizing implementation outcomes, among others. A more complete picture of the network’s primary core is offered with the main path analysis, which consists of those ties above the 95% percentile score for traversal weight (0.0106). The main path, illustrated in Fig. 4, is comprised of the 15 articles listed in Table 4. A simple interpretation of the main path is that these articles are most important in holding the entire D&I framework citation network together. In this case, seven of the ten D&I framework seed articles are part of the main path, along with eight non-seed articles. Visually, one can inspect the main path and observe the chronological flow of influence from earlier to more recent publications. Kitson [25] and Klein [26] act as the primary originating sources of influence in the main path, which serve to influence Greenhalgh [27], Damschroder [28], and Proctor [29]. These five articles, along with Glasgow [30], all converge in Aarons [31], which acts as a major hub for the remainder of the more recent works on the periphery of the main path.
Fig. 3

D&I framework citation network. The majority of first authors provided only one article to the network with only six authors contributing greater than three including Hansen, M and Pronovost, P—four articles; Michie, S and Rycroft-Malone, J—five articles; Greenhalgh, T—seven articles; and Glasgow, R—eight articles

Fig. 4

The main path for a D&I framework citation network. A simple interpretation of the citation network main path is that these articles are the most important in holding the entire D&I framework citation network together. In this case, seven of the ten D&I framework seed articles were part of the main path, along with eight non-seed articles

Table 4

Main path articles for leading D&I research frameworks

Network vertexa

Seed article

Traversal weight

Author/year/article

10

Yes

0.34

Kitson et al. 1998. “Enabling the implementation of evidence based practice: a conceptual framework” [25]

1

Yes

0.24

Greenhalgh et al. 2004. “Diffusion of innovations in service organizations: systematic review and recommendations” [27]

6

Yes

0.18

Damschroder et al. 2009. “Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science” [28]

9

Yes

0.14

Klein and Sorra. 1996. “The challenge of innovation implementation.” [26]

7

Yes

0.12

Aarons et al. 2011. “Advancing a conceptual model of evidence based practice implementation in public service sectors” [31]

3

Yes

0.09

Glasgow et al. 1998. “Evaluating the public health impact of health promotion interventions: the RE-AIM framework.” [30]

8

Yes

0.03

Proctor et al. 2009. “Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges.” [29]

54

No

0.03

Stirman et al. 2012. “The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research” [111]

293

No

0.02

Tabak et al. 2012. “Bridging research and practice: models for dissemination and implementation research” [2]

297

No

0.02

Meyers et al. 2012. “The quality implementation framework: A synthesis of critical steps in the implementation process” [112]

300

No

0.02

Chaudoir et al. 2013. “Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures” [3]

309

No

0.02

Schoenwald et al. 2011. “Toward the effective and efficient measurement of implementation fidelity” [113]

310

No

0.02

Palinkas et al. 2011. “Mixed method designs in implementation research” [114]

311

No

0.02

Sanders. 2012. “Development, evaluation, and multinational dissemination of the Triple P Positive Parenting Program” [115]

312

No

0.02

Aarons et al. 2012. “The organizational social context of mental health services and clinician attitudes toward evidence based practice: a United States national study” [116]

aNetwork vertex is a designated point in the network where 1 through 10 indicates a seed article

Discussion

Using citation analysis, we identified the most frequently cited D&I frameworks and their relationships across time and discipline and mapped the knowledge network constituting the D&I framework field. We discovered that the Tabak framework review has been increasingly cited and that it was included in the periphery of the main D&I framework network path indicating its value as a recognized resource for D&I researchers and practitioners. We identified the leading journals and authors contributing to the D&I framework literature using methods that limit cognitive biases associated with traditional literature searches using keywords. Using the CNA tool to conduct our structured literature review, we were able to identify the main path articles that signify those most important in holding the entire D&I framework citation network together. Overall, D&I researchers and practitioners may consider frequency of citation and this network structure when planning implementation efforts to build upon this foundation and promote systematic advances in D&I science. Further work is necessary to delineate how these frameworks are being used in the literature, framework selection criteria for planning D&I research efforts, the core components of these frameworks, and how framework use relates to improved implementation outcomes [3].

This study provides insight into at least two aspects of the evolving D&I scientific field. First, it confirms that D&I research has witnessed a surge of frameworks with most developed in the last two decades [2]. However, we found that the majority of articles were rarely cited, leaving only a few highly cited frameworks. It is difficult to know whether more recent frameworks will be used or not based on this analysis though several recent articles, including the Tabak review, were highly cited. Nonetheless, there does appear to be framework saturation creating an increasing need to delve further into better understanding the current cadre rather than creating new D&I frameworks. Second, taking into consideration citation rates and this network structure may be a key factor to consider when choosing a framework, in addition to the socioecological level, construct flexibility, and location on the D&I spectrum. For example, increasing citations and centrality in the network indicates more literature is available to highlight the advantages and disadvantages of using a given framework. In addition, there may be more operational and measurement resources with increasing centrality. Taking these additional aspects into consideration creates opportunities to scrutinize frameworks, starting with those in the main path, and advance D&I science by examining issues of fidelity, core and adaptive components, measurement, and relationships to implementation outcomes [1].

We found a broad range of scientific fields contributing to the D&I citation network given our use of Google Scholar™ and its extensive search capabilities [7, 19]. This reinforces the need to scan literature outside of health-related fields to discover new guidance for D&I sciences. For example, other than the specialized journal Implementation Science, which focuses specifically on the field, most citations of the Tabak framework review article were from public health journals due in part to it being a narrative review that used snowball sampling methods and focused on health. In addition, the journals other than Implementation Science, which published the highest number of citations in the broader D&I framework network, were all in the management and business fields. This is consistent with a prior review of leading management journals that found a significant degree of knowledge translation and organizational change literature relevant to D&I in healthcare [32]. While there is some current cross-over among these fields, they are often quite distinct and separate from each other when it comes to research and practice. Taken together, our findings suggest that greater efforts to scan across these journals and fields could provide unique transdisciplinary collaborations and innovation opportunities to hasten D&I research and practice. For that matter, D&I advances could also serve to improve management and business practices.

However, citing a framework does not imply use or specify what its application entails. How to operationalize determinants of practice across frameworks also needs to be better understood to advance D&I science. A recent study examined use of the KTA framework using citation analysis and systematic review to see if the framework was used in practice and how [33]. The authors found that it was used with varying degrees of completeness from a simple reference to integration into the design, delivery, and evaluation of the implementation activities. The latter contributing most to advancing D&I science and generalizability of outcomes. Similarly, another recent systematic review examined use of the CFIR among empirical studies in the peer-reviewed literature [34]. Twenty-six articles met inclusion criteria across a breadth of settings and units of analysis. Justification for which CFIR constructs were selected, integration throughout the research study, and relation to outcomes remained poorly articulated, again limiting contributions to D&I research more broadly. Furthermore, systematic efforts to reconcile determinants of healthcare professional practice across 12 different frameworks have generated practical checklists and implementation strategy recommendations to support implementation and quality improvement efforts [35]. Better understanding framework use, consolidation and operationalization of framework determinants, not just citations, could yield more to consider when selecting and using D&I frameworks for research and practice.

There are several limitations to our study approach. First, framework citation rates are influenced by a multitude of factors including journal impact factor, the authors’ fame and publication rate, the degree of research in a given field, whether citation is perceived as positive or negative, and do not necessarily indicate the quality of a given publication or framework [57, 19]. Nonetheless, citation rates do serve as an approximation of the impact of a scholarly work. We also used an expert-led review article for seed article identification and a robust network analysis tool, coupled with citation rate data, to provide our snapshot of the scientific development of the D&I framework field with substantial face validity. Second, there could be issues with respect to language and the definition of D&I research leading to ascertainment bias. Using our comprehensive CNA approach in Google Scholar™, rather than keyword searches for example, actually created a broader scope for our study. Last, whether the use of highly cited documents (e.g., textbooks) as seed articles, rather than the journal articles selected as seeds in our study, would dramatically change our findings is unclear. Our network tool was inclusive of such documents although they were the minority of articles in both network analyses. Indeed, publishing frameworks outside of journal articles creates challenges, both in terms of physically obtaining the material and being able to grasp the conceptual and operational components dispersed throughout a given textbook. Perhaps corresponding peer-reviewed articles serving as a book review, preferably in open-access formats to improve dissemination, could help mitigate access and citation issues [36].

Conclusion

In conclusion, bibliometric analysis is one way to understand how D&I frameworks are used in the development of D&I science. We used a bibliometric citation analysis tool to help identify the most prevalent models influencing D&I. D&I researchers and practitioners may consider frequency of citation and this network structure when planning implementation efforts to build upon this foundation and promote systematic advances in D&I science.

Abbreviations

CFIR: 

Consolidated Framework for Implementation Research

CNA: 

Citation Network Analyzer

D&I: 

Dissemination and implementation

KTA: 

Knowledge to Action Framework

PARiHS: 

Promoting Action on Research Implementation in Health Services Framework

TDF: 

Theoretical Domains Framework

Declarations

Acknowledgements

Ryan Blake, BS, for administrative and data collection support.

Funding

Dr. Skolarus was supported by a VA HSR&D Career Development Award-2 (CDA 12–171) and the Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) Program, National Cancer Institute, 1 R25 CA171994-01A1 during this study. This study did not receive any dedicated funding.

Availability of data and materials

Data for this project is stored on secure servers. Data can be made publicly available upon request.

Authors’ contributions

The individual contributions of the authors are as follows: TS, TL, RT, JH, JL, and AS contributed to the study conception and design. TS, TL, and JL contributed to the acquisition of data. TS, TL, JL, JH, RT, and AS contributed to the analysis and interpretation of data. TS, TL, and AS drafted the manuscript. TS, TL, RT, JH, JL, and AS made critical revisions. All authors read and approved the final manuscript.

Ethics approval and consent to participate

This study was deemed not regulated by the Institutional Review Board at the University of Michigan.

Consent for publication

N/a.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Center for Clinical Management Research, VA Ann Arbor Healthcare System
(2)
Dow Division of Health Services Research, Department of Urology, University of Michigan
(3)
Urology Section, VA Ann Arbor Healthcare System, Department of Urology, University of Michigan
(4)
Department of Political Science, College of Literature, Science and the Arts, University of Michigan
(5)
Prevention Research Center in St. Louis/George Warren Brown School of Social Work at Washington University in St. Louis
(6)
Maxwell School of Citizenship and Public Affairs, Syracuse University
(7)
Department of Learning Health Sciences, University of Michigan Medical School, University of Michigan

References

  1. Brownson R, Colditz G, Proctor E, editors. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press; 2012.Google Scholar
  2. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50.PubMedPubMed CentralView ArticleGoogle Scholar
  3. Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8:22.PubMedPubMed CentralView ArticleGoogle Scholar
  4. Marx W, Bornmann L. Change of perspective: bibliometrics from the point of view of cited references-a literature overview on approaches to the evaluation of cited references in bibliometrics. Scientometrics. 2016;109(2):1397–415.PubMedPubMed CentralView ArticleGoogle Scholar
  5. Bornmann L, Marx W. How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons. EMBO Rep. 2013;14(3):226–30.PubMedView ArticleGoogle Scholar
  6. Garfield E. The history and meaning of the journal impact factor. JAMA. 2006;295(1):90–3.PubMedView ArticleGoogle Scholar
  7. Bakkalbasi N, Bauer K, Glover J, Wang L. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomed Digit Libr. 2006;3:7.PubMedPubMed CentralView ArticleGoogle Scholar
  8. Harris JK. Connecting discovery and delivery: the need for more evidence on effective smoking cessation strategies for people living with HIV/AIDS. Am J Public Health. 2010;100(7):1245–9.PubMedPubMed CentralView ArticleGoogle Scholar
  9. Harris JK, Beatty KE, Lecy JD, Cyr JM, Shapiro RM 2nd. Mapping the multidisciplinary field of public health services and systems research. Am J Prev Med. 2011;41(1):105–11.PubMedView ArticleGoogle Scholar
  10. Harris JK, Lecy J, Hipp JA, Brownson RC, Parra DC. Mapping the development of research on physical activity and the built environment. Prev Med. 2013;57(5):533–40.PubMedPubMed CentralView ArticleGoogle Scholar
  11. Harris JK, Luke DA, Zuckerman RB, Shelton SC. Forty years of secondhand smoke research: the gap between discovery and delivery. Am J Prev Med. 2009;36(6):538–48.PubMedView ArticleGoogle Scholar
  12. Lecy J, Schmitz HP, Swedlund H. NGO and NPO effectiveness: a modern synthesis. Voluntas. 2012;23(2):434–57.View ArticleGoogle Scholar
  13. Shadish WR, Lecy JD. The meta-analytic big bang. Res Synth Methods. 2015;6(3):246–64.PubMedView ArticleGoogle Scholar
  14. Harper G, Peattie K. Tracking the influence of the first special journal issue on ‘Green Marketing’: a citation network analysis. Soc Bus. 2011;1(3):239–61.View ArticleGoogle Scholar
  15. Jesse D. Lecy and Kate E. Beatty, Representative literature reviews using constrained snowball sampling and citation network analysis 2012. Available at SSRN: https://ssrn.com/abstract=1992601 or http://dx.doi.org/10.2139/ssrn.1992601.Google Scholar
  16. Jesse Lecy and Diego Moreda (2013). “cna: citation network analyzer.” R package version 0.3-4.Google Scholar
  17. Lecy JD, Mergel IA, Schmitz HP. Networks in public administration: current scholarship in review. Public Manag Rev. 2014;16(5):643–65.View ArticleGoogle Scholar
  18. Noruzi A. Google scholar: the new generation of citation indexes. Libri:Int J Libr Inform Serv. 2005;55(4):170–80.View ArticleGoogle Scholar
  19. Maslov S, Redner S. Promise and pitfalls of extending Google’s PageRank algorithm to citation networks. J Neurosci. 2008;28(44):11103–5.PubMedView ArticleGoogle Scholar
  20. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37.PubMedPubMed CentralView ArticleGoogle Scholar
  21. Francis JJ1, O'Connor D, Curran J. Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci. 2012;7:35.Google Scholar
  22. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map? J Contin Educ Heal Prof. 2006;26(1):13–24.View ArticleGoogle Scholar
  23. Wd N, Mrvar A, Batagelj V. Exploratory social network analysis with Pajek. New York: Cambridge University Press; 2005.Google Scholar
  24. Batagelj V, Doreian P, Ferligoj A, Kejzar N. Understanding Large Temporal Networks and Spatial Networks: Exploration, Pattern Searching, Visualization and Network Evolution. West Sussex: Wiley; 2014.Google Scholar
  25. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7(3):149–58.PubMedPubMed CentralView ArticleGoogle Scholar
  26. Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manag Rev. 1996;21(4):1055–80.Google Scholar
  27. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.PubMedPubMed CentralView ArticleGoogle Scholar
  28. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4Google Scholar
  29. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24–34.PubMedView ArticleGoogle Scholar
  30. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.PubMedPubMed CentralView ArticleGoogle Scholar
  31. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23.PubMedView ArticleGoogle Scholar
  32. Harlos K, Tetroe J, Graham ID, Bird M, Robinson N. Mining the management literature for insights into implementing evidence-based change in healthcare. Health Policy. 2012;8(1):33–48.Google Scholar
  33. Field B, Booth A, Ilott I, Gerrish K. Using the knowledge to action framework in practice: a citation analysis and systematic review. Implement Sci. 2014;9:172.PubMedPubMed CentralView ArticleGoogle Scholar
  34. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2016;11:72.PubMedPubMed CentralView ArticleGoogle Scholar
  35. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35.PubMedPubMed CentralView ArticleGoogle Scholar
  36. Baetens F, Prislan V. The dissemination of international scholarship: the future of books and book reviews. Leiden J Int Law. 2014;27(3):559–69.View ArticleGoogle Scholar
  37. Szulanski G. Exploring internal stickiness: impediments to the transfer of best practice within the firm. Strateg Manag J. 1996;17:27–43.View ArticleGoogle Scholar
  38. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, Grp PT. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33.PubMedPubMed CentralView ArticleGoogle Scholar
  39. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J, Grp KTS. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q. 2003;81(2):221-+.PubMedPubMed CentralView ArticleGoogle Scholar
  40. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, Blachman M, Dunville R, Saul J. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41(3–4):171–81.PubMedView ArticleGoogle Scholar
  41. Green LW, Ottoson JM, Garcia C, Hiatt RA. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health. 2009;30:151–74.PubMedView ArticleGoogle Scholar
  42. May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–54.View ArticleGoogle Scholar
  43. Frambach RT, Schillewaert N. Organizational innovation adoption—a multi-level framework of determinants and opportunities for future research. J Bus Res. 2002;55(2):163–76.View ArticleGoogle Scholar
  44. Pathman DE, Konrad TR, Freed GL, Freeman VA, Koch GG. The awareness-to-adherence model of the steps to clinical guideline compliance—the case of pediatric vaccine recommendations. Med Care. 1996;34(9):873–89.PubMedView ArticleGoogle Scholar
  45. Pronovost P, Berenholtz S, Needham D. Translating evidence into practice: a model for large scale knowledge translation. BMJ. 2008;337:a1714Google Scholar
  46. Ward V, House A, Hamer S. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. J Health Serv Res Policy. 2009;14(3):156–64.PubMedPubMed CentralView ArticleGoogle Scholar
  47. Mendel P, Meredith LS, Schoenbaum M, Sherbourne CD, Wells KB. Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in health services research. Adm Policy Ment Health. 2008;35(1–2):21–37.PubMedView ArticleGoogle Scholar
  48. Atun R, de Jongh T, Secci F, Ohiri K, Adeyi O. Integration of targeted health interventions into health systems: a conceptual framework for analysis. Health Policy Plan. 2010;25(2):104–11.PubMedView ArticleGoogle Scholar
  49. Bowen S, Zwi AB. Pathways to “evidence-informed” policy and practice: a framework for action. PLoS Med. 2005;2(7):600–5.View ArticleGoogle Scholar
  50. Glisson C, Schoenwald SK. The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Ment Health Serv Res. 2005;7(4):243–59.PubMedView ArticleGoogle Scholar
  51. Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. 2008;34(4):228–43.PubMedView ArticleGoogle Scholar
  52. Weiner BJ, Lewis MA, Linnan LA. Using organization theory to understand the determinants of effective implementation of worksite health promotion programs. Health Educ Res. 2009;24(2):292–305.PubMedView ArticleGoogle Scholar
  53. Logan J, Graham ID. Toward a comprehensive interdisciplinary model of health care research use. Sci Commun. 1998;20(2):227–46.View ArticleGoogle Scholar
  54. Owen N, Glanz K, Sallis JF, Kelder SH. Evidence-based approaches to dissemination and diffusion of physical activity interventions. Am J Prev Med. 2006;31(4):S35–44.PubMedView ArticleGoogle Scholar
  55. Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci. 2007;2Google Scholar
  56. Jacobson N, Butterill D, Goering P. Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy. 2003;8(2):94–9.PubMedView ArticleGoogle Scholar
  57. Baumbusch JL, Kirkham SR, Khan KB, McDonald H, Semeniuk P, Tan E, Anderson JM. Pursuing common agendas: a collaborative model for knowledge translation between research and practice in clinical settings. Res Nurs Health. 2008;31(2):130–40.PubMedView ArticleGoogle Scholar
  58. Dearing JW, Maibach EW, Buller DB. A convergent diffusion and social marketing approach for disseminating proven approaches to physical activity promotion. Am J Prev Med. 2006;31(4):S11–23.PubMedView ArticleGoogle Scholar
  59. Dobbins M, Ciliska D, Cockerill R, Burnsley J, DiCenso A. A framework for the dissemination and utilization of research for health-care policy and practice. Online J Knowl Synth Nurs. 2002;9(7).Google Scholar
  60. Green LW, Orleans CT, Ottoson JM, Cameron R, Pierce JP, Bettinghaus EP. Inferring strategies for disseminating physical activity policies, programs, and practices from the successes of tobacco control. Am J Prev Med. 2006;31(4 Suppl):S66–81.PubMedView ArticleGoogle Scholar
  61. Kontos PC, Poland BD. Mapping new theoretical and methodological terrain for knowledge translation: contributions from critical realism and the arts. Implement Sci. 2009;4:1.PubMedPubMed CentralView ArticleGoogle Scholar
  62. Lomas J. Retailing research: increasing the role of evidence in clinical services for childbirth. Milbank Q. 1993;71(3):439–75.PubMedView ArticleGoogle Scholar
  63. Majdzadeh R, Sadighi J, Nejat S, Mahani AS, Gholami J. Knowledge translation for research utilization: design of a knowledge translation model at Tehran University of Medical Sciences. J Contin Educ Heal Prof. 2008;28(4):270–7.View ArticleGoogle Scholar
  64. Dreisinger ML, Boland EM, Filler CD, Baker EA, Hessel AS, Brownson RC. Contextual factors influencing readiness for dissemination of obesity prevention programs and policies. Health Educ Res. 2012;27(2):292–306.PubMedView ArticleGoogle Scholar
  65. Martin GW, Herie MA, Turner BJ, Cunningham JA. A social marketing model for disseminating research-based treatments to addictions treatment providers. Addiction. 1998;93(11):1703–15.PubMedView ArticleGoogle Scholar
  66. Elliott SJ, O'Loughlin J, Robinson K, Eyles J, Cameron R, Harvey D, Raine K, Gelskey D, Canadian Heart Health Dissemination Project S, Research Advisory G. Conceptualizing dissemination research and activity: the case of the Canadian heart health initiative. Health Educ Behav. 2003;30(3):267–82. discussion 283-266PubMedView ArticleGoogle Scholar
  67. Kramer DM, Cole DC. Sustained, intensive engagement to promote health and safety knowledge transfer to and utilization by workplaces. Sci Commun. 2003;25(1):56–82.View ArticleGoogle Scholar
  68. Farkas M, Jette AM, Tennstedt S, Haley SM, Quinn V. Knowledge dissemination and utilization in gerontology: an organizing framework. Gerontologist. 2003;43:47–56.PubMedView ArticleGoogle Scholar
  69. Robinson K, Elliott SJ, Driedger SM, Eyles J, O'Loughlin J, Riley B, Cameron R, Harvey D, Grp CSRA. Using linking systems to build capacity and enhance dissemination in heart health promotion: a Canadian multiple-case study. Health Educ Res. 2005;20(5):499–513.PubMedView ArticleGoogle Scholar
  70. Yuan CT, Nembhard IM, Stern AF, Brush JE Jr, Krumholz HM, Bradley EH. Blueprint for the dissemination of evidence-based practices in health care. Issue Brief (Commonw Fund). 2010;86:1–16.Google Scholar
  71. Harris JR, Cheadle A, Hannon PA, Forehand M, Lichiello P, Mahoney E, Snyder S, Yarrow J. A framework for disseminating evidence-based health promotion practices. Prev Chronic Dis. 2012;9:E22.PubMedView ArticleGoogle Scholar
  72. Nolan K, Schall MW, Erb F, Nolan T. Using a framework for spread: the case of patient access in the veterans health administration. Jt Comm J Qual Patient Saf. 2005;31(6):339–47.PubMedView ArticleGoogle Scholar
  73. Anderson M, Cosby J, Swan B, Moore H, Broekhoven M. The use of research in local health service agencies. Soc Sci Med. 1999;49(8):1007–19.PubMedView ArticleGoogle Scholar
  74. Bauman AE, Nelson DE, Pratt M, Matsudo V, Schoeppe S. Dissemination of physical activity evidence, programs, policies, and surveillance in the international public health arena. Am J Prev Med. 2006;31(4):S57–65.PubMedView ArticleGoogle Scholar
  75. Sogolow ED, Kay LS, Doll LS, Neumann MS, Mezoff JS, Eke AN, Semaan S, Anderson JR. Strengthening HIV prevention: application of a research-to-practice framework. AIDS Educ Prev. 2000;12(5):21–32.PubMedGoogle Scholar
  76. Orlandi MA. Health promotion technology transfer: organizational perspectives. Can J Public Health. 1996;87:S28–33.PubMedGoogle Scholar
  77. Winkler JD, Lohr KN, Brook RH. Persuasive communication and medical technology-assessment. Arch Intern Med. 1985;145(2):314–7.PubMedView ArticleGoogle Scholar
  78. Lester JP. The utilization of policy analysis by state agency officials. Knowl. 1993;14(3):267–90.View ArticleGoogle Scholar
  79. Funk SG, Tornquist EM, Champagne MT. A model for improving the dissemination of nursing research. West J Nurs Res. 1989;11(3):361–72.PubMedView ArticleGoogle Scholar
  80. Scullion PA. Effective dissemination strategies. Nurs Res. 2002;10(1):65–77.View ArticleGoogle Scholar
  81. Rogers EM. Diffusion of innovations. New York: Free Press of Glencoe; 1962.Google Scholar
  82. Kingdon JW. Agendas, alternatives, and public policies. Boston: Little, Brown; 1984.Google Scholar
  83. Fixsen DL, Naoom SF, Blasé KA, Friedman RM, Wallace F. Implementation research: a synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network; 2005.Google Scholar
  84. Green LW, Kreuter MW, Green LW. Health program planning: an educational and ecological approach. 4th ed. New York: McGraw-Hill; 2005.Google Scholar
  85. Havelock RG. Planning for innovation through dissemination and utilization of knowledge. Centre for Research on utilization of Scientifıc knowledge, Institute for Social Research, University of Michigan, 1969.Google Scholar
  86. Pettigrew AM, Ferlie E, McKee L. Shaping strategic change: making change in large organizations: the case of the National Health Service. Thousand Oaks: Sage Publications; 1992.Google Scholar
  87. Nieva VF, Murphy R, Ridley N, et al. From science to service: a framework for the transfer of patient safety. In: Henriksen K, Battles JB, Marks ES, Lewin DI, editors. Advances in patient safety: from research to implementation (Vol. 2: concepts and methodology). Agency for Healthcare Research and Quality: Rockville; 2005.Google Scholar
  88. Dodson EA, Brownson RC, Weiss SW. Policy dissemination research. In: Brownson R, Colditz G, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. Oxford: Oxford University Press; 2012.Google Scholar
  89. Kreuter MW, Casey CM, Bernhardt JM. Enhancing dissemination though marketing and distribution systems: a vision for public health. In: Brownson RC, Colditz G, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press; 2012.Google Scholar
  90. Damush T, Bravata DM, Plue L, Woodward-Hagg H, Williams LS. Facilitation of best practices (FAB) framework. Stroke QUERI Center annual report. 2008.Google Scholar
  91. TIDIRH Working Group. Interacting elements of integrating science, policy, and practice. In: Training institute for dissemination and implementation research in health. Chapel Hill: Conference proceedings; 2011.Google Scholar
  92. Bell J, Waters S. Doing Your Research Project: A Guide for First-Time Researchers. England: McGraw-Hill Education; 2014.Google Scholar
  93. Gratton C, Jones I. Research Methods for Sports Studies. London: Routledge; 2010.Google Scholar
  94. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst. 2003;1(1):2.PubMedPubMed CentralView ArticleGoogle Scholar
  95. Hersh W. Information retrieval: a health and biomedical perspective. New York: Springer; 2008.Google Scholar
  96. Dalrymple J, Burke B. Anti-oppressive practice: social care and the law. England: McGraw-Hill Companies, Incorporated; 2006.Google Scholar
  97. Aveyard H. Doing a literature review in health and social care: a practical guide. England: McGraw-Hill Education; 2014.Google Scholar
  98. Trevithick P. Social work skills: a practice handbook. England: Open University Press; 2005.Google Scholar
  99. Thompson N. Understanding social work: preparing for practice. England: Palgrave Macmillan; 2015.Google Scholar
  100. D'Cruz H, Gillingham P, Melendez S. Reflexivity, its meanings and relevance for social work: a critical review of the literature. Brit J Soc Work. 2007;37(1):73–90.View ArticleGoogle Scholar
  101. Dagenais S, Tricco AC, Haldeman S. Synthesis of recommendations for the assessment and management of low back pain from recent clinical practice guidelines. Spine J. 2010;10(6):514–29.PubMedView ArticleGoogle Scholar
  102. Nahapiet J, Ghoshal S. Social capital, intellectual capital, and the organizational advantage. Acad Manag Rev. 1998;23(2):242–66.Google Scholar
  103. Eisenhardt KM, Martin JA. Dynamic capabilities: what are they? Strateg Manag J. 2000;21(10–11):1105–21.View ArticleGoogle Scholar
  104. Dyer JH, Singh H. The relational view: cooperative strategy and sources of interorganizational competitive advantage. Acad Manag Rev. 1998;23(4):660–79.Google Scholar
  105. Wenger E, McDermott RA, Snyder W. Cultivating communities of practice: a guide to managing knowledge. Boston: Harvard Business School Press; 2002.Google Scholar
  106. Alavi M, Leidner DE. Review: knowledge management and knowledge management systems: conceptual foundations and research issues. Mis Quart. 2001;25(1):107–36.View ArticleGoogle Scholar
  107. Zahra SA, George G. Absorptive capacity: a review, reconceptualization, and extension. Acad Manag Rev. 2002;27(2):185–203.Google Scholar
  108. Hansen MT. The search-transfer problem: the role of weak ties in sharing knowledge across organization subunits. Admin Sci Quart. 1999;44(1):82–111.View ArticleGoogle Scholar
  109. Ahuja G. Collaboration networks, structural holes, and innovation: a longitudinal study. Admin Sci Quart. 2000;45(3):425–55.View ArticleGoogle Scholar
  110. Dyer JH, Nobeoka K. Creating and managing a high-performance knowledge-sharing network: the Toyota case. Strateg Manag J. 2000;21(3):345–67.View ArticleGoogle Scholar
  111. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7Google Scholar
  112. Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol. 2012;50(3–4):462–80.PubMedView ArticleGoogle Scholar
  113. Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Admin Pol Ment Health. 2011;38(1):32–43.View ArticleGoogle Scholar
  114. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health. 2011;38(1):44–53.PubMedView ArticleGoogle Scholar
  115. Sanders MR. Development, evaluation, and multinational dissemination of the triple P-positive parenting program. Annu Rev Clin Psychol. 2012;8:345–79.PubMedView ArticleGoogle Scholar
  116. Aarons GA, Glisson C, Green PD, Hoagwood K, Kelleher KJ, Landsverk JA, Hlth RNYM. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implement Sci. 2012;7Google Scholar

Copyright

© The Author(s). 2017

Advertisement