The KTA Framework [1] is being used in practice with varying degrees of completeness and theory fidelity when the conceptual framework is integrated into the implementation project. It is one of the most frequently cited conceptual frameworks for knowledge translation. A citation search of three databases tracking the source paper ‘Lost in knowledge translation: time for a map?’ identified 1,787 records between 2006 and July 2013. Some indication of the impact of this article can be gained from constructing a ‘normalised citation count’ for comparison purposes. The source paper was cited four times more frequently than the next highest cited article from the same journal published in the same year. It was cited 470 unique times across all databases. However, citation figures do not reflect how this conceptual framework has actually been applied in practice. We initially included 146 studies that reported using the KTA Framework. Closer examination revealed that usage varied considerably, ranging from simple attribution via a reference through to being integral to most aspects of the implementation work. Only ten studies reported and gave supportive examples of incorporating the KTA Framework in an integrated way. All these described the Action Cycle and seven referred to Knowledge Creation, articulating the KTA Framework in a way that was true to the source paper [1].
There was substantial variation in the setting and target audience for each documented change, the methods used to apply the KTA Framework and the terminology employed to report the phases within Knowledge Creation and the Action Cycle. This reflects the spread of the framework across a range of settings, to different health care services and systems and larger scale and smaller projects. The KTA Framework was adapted to different health service settings and resources, indicating a good fit for the diversity of real-world health care. However, the target audiences were primarily patients, the public and the nursing and allied health professions.
Most studies (8/10) were conducted in Canada where the KTA Framework originated. This frequency of use in Canada could be explained by the influence of national Canadian networks and dissemination activities. Perhaps more significantly, the KTA Framework is associated with the Canadian Institutes of Health Research [27]. Estabrooks and colleagues [28] comment that the Canadian research funding organisation had adopted the KTA Framework to guide knowledge translation, deploying specific grant mechanisms ‘to ensure involvement of knowledge users with researchers throughout the research process’ (p. 2). This may explain the varying degrees to which the framework was used.
The KTA Framework was enacted in a variety of ways, from informing to full integration, showing flexibility of use and that it can fit local circumstances and need. Use at a ‘lighter’ level through adapting or combining with other conceptual frameworks could be considered a strength, in that the KTA Framework offers the flexibility to be combined with other frameworks, being responsive to facilitating practitioner preferences and context-specific needs. It is important to note that Graham and colleagues have continued to publish on applications of the framework and the multiple factors to be considered [29]. In addition, this suggests conceptual frameworks do not have to be mutually exclusive.
Each study applied the framework in an idiosyncratic way. None reported using every phase of the KTA Framework. Only four studies detailed Knowledge Creation, yet existing knowledge was utilised in the other studies to identify knowledge-practice gaps, or as part of the Action Cycle. This flexibility was intended, as Graham and colleagues [1] state the framework can ‘…also accommodate different phases being accomplished by different stakeholders and groups (working independently of each other) at different points in time’ (p. 18). Implementation researchers and health professionals can learn from this flexibility. It may be useful for them to consider the extent to which they wish to follow or be guided by a conceptual framework before embarking on a knowledge translation project, especially regarding outcome measures because ‘....the focus of knowledge into action is ultimately to enhance health status’ [1 p. 18]. The Action Cycle was reported in all the integrated examples, illustrating theory fidelity in this specific subset of studies. The prevalence of the Action Cycle may simply reflect the cyclical nature of the change process evident in other common, quality improvement tools such as the ‘Plan, Do, Study, Act’ cycle [30]. Also, this conceptual framework may appeal because of a lack of jargon and a simple diagram depicts the knowledge translation process.
The integrated studies described different ways of integrating the KTA Framework, particularly the Action Cycle. Most studies focused on improving knowledge or awareness, supporting what we know about the preponderance of professional or educational knowledge translation strategies within interventions aiming to promote the uptake of evidence [31]. Nine of the studies reported assessing barriers to change [17]-[19],[21]-[26]. Knowledge-related barriers [32] were the most common, indicating the appropriateness of using educational strategies to address such barriers. Yet the albeit limited, evidence available indicates that bringing information close to the point of decision-making (such as using reminders or decision support tools) is likely to be more effective than using more traditional educational strategies (such as study, teaching or training) to try to address barriers and change practice [31],[33]. Only one study [26] reported using decision support tools as a knowledge translation strategy, although it is possible others did not report all the details of strategies they used to promote the adoption of their interventions. Knowledge translation strategies can include elements such as linkage and exchange, audit and feedback, informatics and patient-mediated and organisational interventions [29]. However, none of these knowledge translation strategies, designed to target different barriers, featured in the included studies, with one exception. Russell and colleagues [25] describe use of a knowledge broker, an example of a ‘linkage and exchange’-type strategy.
The importance of organisational or external factors and the ability to influence them is well recognised [1],[3],[33]-[35]. Authors identified many barriers relating to ‘environmental’ factors [32] such as lack of time and/or resources. Generally, it was difficult to ascertain whether the methods used captured, and indeed subsequently addressed, the full range of barriers. It may be that when people are consulted, they identify those barriers that they feel able to influence, such as knowledge or awareness, rather than organisational barriers, which could be perceived as more problematic or more distant. An exception was the study by Russell and colleagues [25] who reported using a questionnaire to assess a range of potential barriers and facilitators. Molfenter and colleagues [23] describe a strategy to assist clinicians with patient selection which could be interpreted as seeking to influence organisational barriers such as ‘competing priorities’, as well as addressing knowledge-related barriers. Légaré [32] recommends using established taxonomies developed for barriers and facilitators within knowledge translation projects. Our findings support this proposal. We further identified a need to use taxonomies when analysing or evaluating knowledge translation strategies [33],[34]. This study also reinforces the importance of reporting standards [8],[36], such as the new TIDieR checklist [37] to facilitate more explicit reporting of implementation studies and their subsequent inclusion in systematic reviews.
The monitoring, outcomes or sustaining phases of the Action Cycle were less often described, although three noted their plans for doing so [18],[24],[26]. This may reflect a publication bias, between reporting process and outcomes. Claude and colleagues [17] stated that these phases were beyond the scope of their project. Such work may require additional funding for longer term monitoring or strategies to sustain knowledge use over time. It may also be a reflection of the challenges for defining and reporting outcomes for knowledge translation projects. Tugwell and colleagues [26] highlight this particular challenge, commenting that most outcomes in arthritis research are about pain and function. In contrast, they wanted to evaluate the impact of their intervention on people’s ability to understand their choices and make informed decisions about treatment. Consequently, we recommend that the phase ‘select, tailor, implement intervention’ be amended to include ‘define and select outcomes and knowledge use measures’, as a prompt to those using the KTA Framework to specify such outcomes when selecting which knowledge translation strategies to use.
A notable feature of the KTA Framework is the development of knowledge tools or products as part of Knowledge Creation. Most studies created a range of products, either as part of Knowledge Creation [18],[20],[21],[26] or the Action Cycle [21],[23],[24]. Interestingly, a recent evaluation of the Canadian Institutes of Health Research (CIHR) Knowledge Translation Funding Program [38] presented results relating to knowledge translation products, academic outputs and capacity building together, giving the appearance, in our interpretation, that these different dimensions may be regarded as equally important.
Interest in using systematic literature reviews to investigate theories, models and conceptual frameworks has increased in recent years [7],[39]. Yet, this method may prove challenging, often because of limited and imprecise reporting. Davies and colleagues [40] note that less than 6% of 235 studies, albeit published before 1998, explicitly used theories of behaviour or behaviour change. They and we resorted to a simple taxonomy to describe the level and type of usage based on explicit reporting. A coding scheme, with 19 categories for theory use for behaviour change interventions, ranging from mentioned but not demonstrated, right through to theory refinement, has been developed [41]. This scheme may be useful for similar studies examining theory use. However, our review focused primarily on examining how a conceptual framework had been used in practice. Using the framework itself as a device through which to examine how it had been used seemed an appropriate and pragmatic approach for our purposes.
Bartholomew and Mullen [42] suggest that the ‘prevailing wisdom in the field of health-related behaviour change is that well-designed and effective interventions are guided by theory’ (p. S20). Others argue that the effectiveness and generalisability of implementation studies are hindered by weak theoretical underpinnings [40],[43],[44]. Our review, and similar studies [39],[40],[43]-[45], consistently comments on the limited, haphazard use of theory, even though theories can be applied in many different ways [41]. Primary studies, exploring the direct experience and perceptions of different stakeholders in implementation projects, which have been guided by conceptual frameworks, or theories, would add to our understanding of the utility and impact of these tools. A few authors have attempted this, such as McEvoy and colleagues [43] who examined benefits reported by authors using the Normalization Process Theory. A prospective design would strengthen research studies.
Future research could examine the studies which we categorised as using the KTA Framework to a lesser degree, perhaps for conceptual or persuasive reasons rather than instrumentally. It would be interesting to review the conceptual papers we excluded to explore how they were influenced by the framework and informed the development of conceptual frameworks more generally. Exploring the impact of the KTA Framework, and other conceptual frameworks, on patients and the public in terms of health improvement and outcomes would also be worthwhile, as would exploring their involvement in the application of the framework, not just as recipients of services but as key stakeholders in each phase.
Limitations
Our study had several limitations. Firstly, there is a risk of bias and subsequent error given the lead author conducted most of the initial screening, all data extraction and synthesis. This was necessary given the limited resources available to support the review process. It is possible some potentially relevant studies were excluded during the initial sift stage. Several strategies were used to counter this risk. For example, difficulties in applying exclusion/inclusion criteria were discussed by the team and all subsequent decisions were then resolved by consensus. The final list of integrated studies was agreed by two authors (BF and II), and the synthesis was discussed in detail by the team. Initial ‘screening out’ by title and abstract on the basis of partial information from Google Scholar may also have excluded relevant studies. This is a limitation of using Google Scholar for citation searches. We acknowledge that there are multiple choices available when conducting citation searches. In the absence of formal comparisons of citation search techniques, we decided to operationalise citation searching using Google Scholar. Although there may be some small variation in the actual sets of references retrieved by different citation searches, we have no reason to believe that we have systematically under- or over-represented particular types of studies in our sample. Non-English language studies were excluded, reducing access to examples of applying the KTA Framework published in other languages. This is noteworthy given Canadian research may be published in French language journals. Further, the time frame of the citation searches only captures work up to a certain point (from the date of publication of the source paper to July 2013). We recognise that assimilation and utilisation of an influential framework is a continuous process and that we have employed essentially a cross-sectional method to survey the literature. However, the method is replicable and could be repeated to update the review in future.
Selection bias is another limitation given that we aimed to identify papers reporting usage of the KTA Framework rather than considering or comparing with other conceptual frameworks. This reflected our focus on whether the KTA Framework is used in practice and, if so, how it is applied.
The search strategy was limited to citation searching of three databases. We did not follow up references (including book chapters) or contact authors of included or excluded studies. Neither did we seek out grey literature or search specific thesis/dissertation databases. Citation searching seeks to optimise sensitivity and specificity when seeking to identify reports of practical applications of a model or framework. Our experience certainly confirms that this search method circumvents the problems of variation in terminology typically encountered in topic-based searches of bibliographic databases. However, as shown by our study, citation searching in isolation from citation analysis—the detailed examination of full-text—is unable to discriminate between where a model is simply mentioned ‘in passing’ or even for ‘cosmetic’ reasons and where it represents an explicit intellectual and conceptual contribution.
We acknowledge that faced with the same task, another review team might choose to include papers reporting a single knowledge translation strategy. We did not include the many papers about clinical practice guidelines for two interrelated reasons. Firstly, we were interested in the practical application of the KTA Framework and thus in identifying papers where the KTA Framework appeared to be a fundamental guide to the work reported. Secondly, multifaceted strategies are more likely to be successful than a single strategy [27],[46],[47], as they target different barriers [40], which reflects our interest in the real-world use of the KTA Framework, where it is probably impossible to control and isolate one strategy [48]. However, it is worth noting that clinical guidelines could be categorised as knowledge tools/products and/or implementation strategies depending on if, and how, the guideline features in the framework.
Similarly, we acknowledge that some readers may be interested in precise reasons for exclusions. We have chosen to report the aggregate number of excluded papers on the PRISMA flow diagram (Figure 2). Due to overlapping categories (e.g. review papers, conceptual or descriptive papers and those describing a single knowledge translation strategy or not topically relevant), we excluded against a single criterion, even when multiple criteria applied, as practical considerations rendered it unnecessary to exhaustively document all possible reasons for exclusion for each paper.
Inevitably, decisions about including or excluding studies were reliant on subjective judgements about whether the KTA Framework had been reported in an integrated way, or not. A continuum of usage, ranging from referenced to integrated, was developed to aid this process. Judging between informed and directed was difficult, suggesting further refinement of the categories is required.
Data extraction and presenting results according to the phases were also challenging, given that the framework is dynamic, and can be non-sequential with overlap between phases [1]. There were occasions when we sensed that aspects of Knowledge Creation and Action Cycle had been done or combined, but as this was not explicit, we excluded these data. For example, the creation of websites, interactive e-learning modules, training packages and a protocol were reported as part of the Action Cycle [19],[23],[24], yet they could be knowledge tools/products. This reinforces the importance of theory fidelity and that authors refer to established taxonomies or reporting standards [36],[37],[40],[41] so we can understand how conceptual frameworks, theories and models are really used to guide practice or inform research.
