Information transfer: what do decision makers want and need from researchers?

Knowledge translation and exchange

In recent years the terms knowledge transfer and exchange, evidence-based decision-making, and evidence-informed health policy have become commonly used, but with little consensus on their definition, how they occur, or can be promoted [1, 2]. Furthermore, significant resources and time are invested in the production of research evidence that, if effectively transferred, could be used to inform policy and practice decisions and subsequently improve patient and population health outcomes [3]. In October 1994, the Prime Minister of Canada launched the National Forum on Health to involve and inform Canadians and to advise the federal government on innovative ways to improve the health system and the health of Canada's people. A key recommendation arising from the National Forum on Health [4] was the development of an evidence-informed health care system in Canada where policies and clinical decisions are influenced by high quality research knowledge. As a result, considerable research inquiry has focused on understanding the processes of evidence-informed decision-making, as well as on how to facilitate it.

In order to move toward an evidence-informed health care system, significant environmental changes are required. At a minimum, researchers must become more effective communicators of their research findings, gain a better appreciation of the context in which decision-makers function, and build more collaborative relationships with policy-makers, decision-makers and practitioners [3, 5, 6]. In addition, policy-makers, decision-makers and practitioners must become more receptive to the inclusion of the best available research evidence in the decision-making process, and be willing to collaborate with researchers to ensure that relevant and applicable research is conducted [7–9].

The knowledge transfer field in health care, while relatively young compared to other empirical fields, has evolved significantly in the past 40 years. It has identified barriers and facilitators of knowledge transfer, determined the effectiveness of dissemination strategies, explored decision-making processes and organizational capacity for change, and evaluated collaborative efforts to bring producers and users of research evidence together to develop, implement and interpret research evidence. Consistent barriers across all settings include lack of available time, lack of access to current research literature, limited critical appraisal skills, excessive literature to review, work environments that do not support research transfer and uptake, lack of decision-making authority to implement research results, organizational decision-making processes that are not conducive to research transfer and uptake, resistance to change, and limited resources for implementation [10–17].

While the process is complex and constantly changing, several generalizations about knowledge transfer and exchange (KTE) can be made. Traditional passive strategies used alone are relatively ineffective [18, 19]. Strategies that are more interactive and involve face-to-face contact show promising results with a variety of target populations [12, 20–23]. Involvement of decision-makers in the research process has been shown to be associated with higher degrees of uptake [24]. When the results or 'messages' of research results are tailored to the specific needs of decision-makers then perceived uptake is higher [3, 25, 26]. An emerging hypothesis is that a combination of strategies, resulting in an interactive, multi-component KTE program that reinforces relationships between research producers and users, and reaches potential users on multiple levels, may be most effective in achieving an evidence-informed health care system [27].

The CanChild Centre for Childhood Disability Research at McMaster University, funded since 1989 by the Ontario Ministry of Health and Long-Term Care, implements interactive, multi-component interventions that reinforce relationships between research producers and potential users. Two main goals of CanChild are to provide effective leadership and innovation in childhood disability research and information transfer, and to impact on knowledge, practice, services, and policy in childhood disability through programmatic research and research transfer [28]. CanChild accomplishes these goals by working collaboratively with community agencies, including all members of the Ontario Children's Treatment Centres (CTCs) and the Ontario Community Care Access Centres (CCACs), and to a modest extent the District Health Councils of Ontario.

Since its inception, CanChild has worked in partnership with all of the CTCs in Ontario. The CTCs provide developmental therapies and family-based services to children with a variety of developmental disabilities and their families [29]. More recently, CanChild has started working collaboratively with the CCACs of Ontario, which have become increasingly involved in the provision of services to children with disabilities. CCACs offer an access point to Ontario's long-term care system by: assessing and arranging for visiting health and professional services in people's homes; assessing, authorizing, and arranging for the provision of school health support services for children; providing information and referrals to the public about other community agencies and services available to them; and coordinating services such as nursing, physiotherapy, occupational therapy, speech-language therapy, dietician services, social work, personal support and homemaking [30]. The District Health Councils provide advice to the Ontario Ministry of Health and Long-Term Care on health needs and other health matters in their geographic areas. They also play an important role in the provision of health care information in their communities, as well as promote the integration of health services and identify health planning needs [31].

All three types of organizations were approached to participate in this study to explore different decision-makers' perspectives on knowledge translation. These organizations were chosen because they are all involved in developing services for children with disabilities, but have different relationships with CanChild. For example, at the time of this study there existed a longstanding partnership between the CTCs and CanChild, including a focused program of KTE between the two organizations. While CanChild had a working relationship with the CCACs, the relationship was in its infancy, and at the time of this study there had been fewer opportunities to promote the transfer and uptake of research information by CCACs. In addition, at the time of this study there was no formal, established relationship between the DHCs and CanChild. It was expected that variations in the respective missions and existing relationships with CanChild would provide interesting observations related to the uptake of research evidence in decision-making.

Among the most consistent issue identified by those working in the childhood disability field is the need for information that is of high quality, synthesized, easy to use, and easy to access [32]. In its attempt to address these information needs, CanChild reviewed the KTE literature to assist in the development of a KTE program that would facilitate relationships between research producers and users, and promote evidence-informed decision-making and practice [3, 5, 33–36]. Their review of the literature and interaction with the CTCs led to the adoption of these guiding principles. The dissemination source must be perceived as competent, credible, and trustworthy. The content must be perceived as relevant, usable, methodologically sound, and comprehensive to users. The medium must be accessible, user-friendly, and clearly understandable. Finally, the intended user must perceive the relevance of the materials to their own needs, and understand it in the context of their work.

These principles guided the development of the research dissemination program, which aims to enhance the incorporation of CanChild's and others' research evidence into policy and program decision-making [37]. A major objective is to facilitate the transfer and uptake of this research evidence specifically by CTCs and CCACs. The program is centered on the "Keeping Current" materials. These are brief 'bottom-line' systematic reviews of issues that have been assessed as being important 'hot topics' in the childhood disability field. They are written in plain language, are three to five pages in length, and can be read easily and quickly. Through Impact Surveys carried out by CanChild it has been shown that the 'Keeping Current' format is one that decision-makers and clinicians appreciate, and it has influenced thinking about issues and use of information [32]. While some evaluation has been conducted on this program, CanChild investigators have never systematically studied what information, in what formats, people want and need in their roles as clinicians, managers, directors, and CEOs. It is timely to evaluate the impact of such a program so that knowledge gained from this strategy can be used to enhance CanChild's program as well as inform KT strategies for other research-producing organizations.

It is understood that research uptake varies significantly across decision-making levels (i.e., CEOs, directors, managers, clinicians) Therefore, in this study it was recognized that responses would vary not only across the three organizational types, but also among the different decision-maker levels. For example, CEOs and senior managers/directors could utilize research evidence in decisions related to broad organizational policies concerning service provision or in recommendations for provincial health policies. Middle managers could use research evidence to inform decisions related to program planning. Clinicians could use research evidence to inform clinical practice, and senior health planners could use research evidence to inform recommendations for local and provincial resource allocation and service provision. Regardless of the decision type, it was expected that research evidence would be used by all participants in some way to make decisions during the study.

The purpose of this study was to undertake a systematic assessment of the need for research-based information by decision-makers working in community-based organizations. It is part of a more comprehensive knowledge transfer and exchange strategy that seeks to understand both the content required and the format/methods by which such information should be presented.

Design

The design was a cross-sectional telephone survey comprised of 65 questions (available from the first author upon request) that took approximately 20 to 25 minutes to complete.

Respondents

Decision-makers at any level within the organization, ranging from CEOs to front-line clinicians (i.e., speech therapists, physiotherapists, occupational therapists) were invited to participate in the study. Senior health planners who worked only at DHCs were also recruited.

Content of the survey

The survey asked questions about current practices, research use, and demographic information, as well as preferences for receiving research information. The survey was divided into five sections: 1) questions that sought to understand the current practices and culture of the organization were asked first, in order to provide the investigators with a more complete picture of the attitudes and motives of the respondents; 2) questions about access to research evidence, use by the individual, perceived barriers to the access, and use of research evidence; 3) questions about which formats and styles of research information were preferred, and the perceived use of research by the organization, as well as the field of health providers as a whole; 4) questions about possible mitigating factors in transferring information from researchers to service providers (e.g. established relationship between research producers and users), and whether the current circumstances of that transfer were satisfactory; and 5) questions about current practices, culture, barriers and mitigating factors, and the demographic structure of the organization and region.

Scoring and data analysis

A variety of Likert scales, all consisting of five points, were used in this survey. Possible responses for questions 11 to 16 included 'Not an issue', 'a minor issue', 'a moderate issue', 'a serious issue', and 'a very serious issue'. For questions 36 to 41, the following response options were available: 'definitely won't', 'probably won't', 'may', 'probably will', and 'definitely will'. For questions 42 to 44 response options included: 'excellent', 'good', 'moderate', 'fair', 'poor'. Finally, for questions 46 to 56 responses included: 'strongly agree', 'moderately agree', 'neither agree or disagree', 'moderately disagree', 'strongly disagree'. Prior to analysis all of the scores were transformed to be consistent in direction, which meant scores ranged from one to five, with higher scores representing more positive perceptions.

Rather than relying exclusively on scores of individual items, groups of related items were scored and prorated according to the number of items in the group. Thus, for example, an 'integrated barrier score' was derived from items 11–16 of the questionnaire, grouped into scale scores based on a three-point rating which combined ratings that appeared similar. For example 'not an issue' and 'minor issue' were scored as 'generally not an issue', 'moderate issue' stayed as is, and 'a serious issue' and 'a very serious issue' were scored as 'generally a very serious issue'. After adjusting the data to reflect this reclassification, the average scores for each of items 11–16 were summed and divided by the total number of items (n = 6) to derive an 'integrated barrier score'. The same procedure was used to derive a 'research culture receptivity score' for items 45–50 of the questionnaire.

The analysis used descriptive statistics to report the patterns observed with the survey materials. Because item non-response varied, there are differences in the total number of participants for which data is available for different items. To explore variations across the three types of organizations involved in this study, analyses of variance (ANOVAs) were used, applying Tukey's multiple comparison test for post-hoc analyses where the ANOVA was significant at p < 0.05. Given the exploratory nature of this study, p < 0.05 was used so as to be more inclusive of potentially important variables.

Administration of the survey

Respondents received an initial phone call to schedule a time to conduct the survey. The interviews were performed and data collected between January and April 2002. While research use tends to be overestimated by health care professionals when self-reported measures are used rather than a combination of direct observation and self-report [38], a self-reported telephone administered survey was adapted from previously published research [18, 39] for this study. Two methods were used to reduce respondent overestimation of research use: respondents were assured their responses would be kept confidential and anonymous, and they were asked to give specific examples of research use. Previous research has demonstrated less overestimation of research use among public health professionals when concrete examples of research use are sought [39]. Furthermore, given that this was the first study of its kind to be conducted with this sample, it was expected that direct observation of individuals or teams would severely reduce participation in the study.

Ethics

Ethics approval was sought and obtained from the McMaster University Research Ethics Board.

Evidence of the validity of the questionnaire

Responses to four survey questions provide evidence of construct (discriminant) validity of the approach used in this survey. In response to the question: "With respect to its ease of use, how would you rate the quality of research information you have received recently?" there was a significant difference across the three organizations in favour of the CTCs. The overall mean score was 3.82 with mean scale scores of 4.11 for CTCs, 3.88 for DHCs, and 3.57 for the CCACs (p < 0.01). This finding was taken to reflect the active and collaborative relationship between CanChild and the CTCs that resulted in relevant and timely research being disseminated to them. In response to the statement: "My organization routinely actively looks for research information before making decisions", the overall mean score was 3.83, with mean DHC responses averaging 4.54, the CTCs 3.81, and the CCACs 3.35 (p < 0.01). This finding is thought to reflect the research nature of the DHCs and the 'clinical' focus of the other programs, in which research information informs, but is not essential to all decisions taken at the program planning and clinician level. Similarly, in response to the statement: "My organization provides ongoing training in research methods and critical appraisal to staff", the overall mean value was 2.75, with the DHC respondents reporting a mean of 3.50, the CTCs a mean of 2.63, and the CCACs a mean of 2.31 (p < 0.01). Finally, in response to the statement: "Overall, my organization provides adequate resources (financial or personnel) to implement decisions that are based on scientific evidence" the overall mean value was 3.62, and there was once again a gradient in favour of the DHC respondents (mean 4.04), compared with the CTCs (mean 3.52) and the CCACs (mean 3.43) (p = 0.04).

Barriers to research transfer

Data on the barrier items 11 to 16 were then aggregated and analyzed to determine if there were differences overall of perceptions of barriers between the three organization types. The mean overall barrier score on a three-point scale was 1.72 (higher scores report perception of greater barriers). Baseline characteristics such as age, type of organization being surveyed and respondent's highest level of education were not significantly associated with perceptions about barriers. There were statistically significant variations across organization type with the CCACs reporting the highest perception of barriers to the use of research information with a score of 1.88, while the CTCs reported a score of 1.65, and the DHCs a score of 1.59 (p < 0.006).

Organizational characteristics

Decision-makers' perceptions of knowledge translation strategies

Questions about the preferred formats for receiving information showed that people's first choices were for executive summaries (53.2%), abstracts (29.3%) and original articles (17.4%). Second choices were for abstracts (45.7%), original articles (28.2%) and executive summaries (26.1%).

Assessment of perceived impact of research information on decision-making