- Open Access
- Open Peer Review
A KT intervention including the evidence alert system to improve clinician’s evidence-based practice behavior—a cluster randomized controlled trial
© Campbell et al.; licensee BioMed Central Ltd. 2013
- Received: 24 January 2013
- Accepted: 4 November 2013
- Published: 13 November 2013
It is difficult to foster research utilization among allied health professionals (AHPs). Tailored, multifaceted knowledge translation (KT) strategies are now recommended but are resource intensive to implement. Employers need effective KT solutions but little is known about; the impact and viability of multifaceted KT strategies using an online KT tool, their effectiveness with AHPs and their effect on evidence-based practice (EBP) decision-making behavior. The study aim was to measure the effectiveness of a multifaceted KT intervention including a customized KT tool, to change EBP behavior, knowledge, and attitudes of AHPs.
This is an evaluator-blinded, cluster randomized controlled trial conducted in an Australian community-based cerebral palsy service. 135 AHPs (physiotherapists, occupational therapists, speech pathologists, psychologists and social workers) from four regions were cluster randomized (n = 4), to either the KT intervention group (n = 73 AHPs) or the control group (n = 62 AHPs), using computer-generated random numbers, concealed in opaque envelopes, by an independent officer. The KT intervention included three-day skills training workshop and multifaceted workplace supports to redress barriers (paid EBP time, mentoring, system changes and access to an online research synthesis tool). Primary outcome (self- and peer-rated EBP behavior) was measured using the Goal Attainment Scale (individual level). Secondary outcomes (knowledge and attitudes) were measured using exams and the Evidence Based Practice Attitude Scale.
The intervention group’s primary outcome scores improved relative to the control group, however when clustering was taken into account, the findings were non-significant: self-rated EBP behavior [effect size 4.97 (95% CI -10.47, 20.41) (p = 0.52)]; peer-rated EBP behavior [effect size 5.86 (95% CI -17.77, 29.50) (p = 0.62)]. Statistically significant improvements in EBP knowledge were detected [effect size 2.97 (95% CI 1.97, 3.97 (p < 0.0001)]. Change in EBP attitudes was not statistically significant.
Improvement in EBP behavior was not statistically significant after adjusting for cluster effect, however similar improvements from peer-ratings suggest behaviorally meaningful gains. The large variability in behavior observed between clusters suggests barrier assessments and subsequent KT interventions may need to target subgroups within an organization.
Registered on the Australian New Zealand Clinical Trials Registry (ACTRN12611000529943).
- Allied health
- Evidence-based practice
- Online KT tool
Cerebral palsy (CP) is the most common physical disability in childhood . Of people with CP, three in four are in pain; one in two have an intellectual disability; one in three cannot walk; one in three have a hip displacement; one in four cannot talk; one in four have epilepsy; one in four have a behavior disorder; one in four have bladder control problems; one in five have a sleep disorder; one in five dribble; one in ten are blind; one in fifteen are tube fed; and one in twenty-five are deaf . Allied health professionals (AHPs) who treat people with CP are therefore faced with complex clinical decision making. Also, like many other fields, new evidence-based CP treatments are rapidly emerging . AHPs provide the majority of health services to these people and therefore need to have up-to-date knowledge and skills in providing evidence-based interventions. AHPs endorse providing evidence-based care [4, 5], but goodwill alone does not guarantee the latest research is translated and applied within practice [6, 7]. Survey research suggests that there is a significant gap between best available evidence and what treatments are actually offered to people with CP [8, 9]. Lack of time , lack of skill searching and appraising research [11, 12], and lack of access to databases compounded by large volumes of published research are barriers to new knowledge being translated in a timely and efficient way .
Knowledge translation (KT) strategies including workshops , mentoring , outreach visits , audit and feedback  and reminders and memos  aim to embed research into practice and lead to small to moderate changes in health professionals’ behavior. Even though KT is an emergent science, it is known that KT strategies should be tailored to be context specific, and planned in response to a thorough assessment of barriers and facilitators [19, 20]. Although there is no firm evidence that multifaceted strategies are more effective than single interventions it is plausible that they would be more effective if each component and the overall strategy were designed in response to a barriers analysis . In the field of CP, a tailored KT intervention was pilot tested with good results, but the lack of a controlled comparison group precludes certainty of the findings .
Previous studies measuring the effectiveness of evidence-based information resources (5S pyramid level 3) detected a change in use however did not detect a change in evidence-based practice (EBP) behavior [27, 28]. Dobbins et al.  found that targeted messages (5S pyramid level 3 – 4) were more effective than knowledge brokering and access to research evidence for incorporating evidence into public health policies and programs. Although evidence-based information resources are available for AHPs (PEDro, OTseeker, SpeechBite) they are at 5S pyramid level 3 (synopses), and no studies have rigorously evaluated the usefulness of these tools. There are no KT tools (5S pyramid levels 4 or 5) found in literature specifically targeting AHPs working with people with CP.
KT tools presenting up to date research in a user-friendly way, is however only one piece of a KT strategy. Changing EBP behavior is complex as there is a range of behaviors required to be an ‘evidence-based AHP.’ Previous studies have either used self-developed measures [30–33] or have only measured a narrow domain of EBM behavior [34, 35]. KT research in the allied health professions measuring EBP behavior across a range of AHPs is also absent from our evidence base [36, 37].
The primary aim of this cluster randomized controlled trial (RCT) was to evaluate the effectiveness of a multifaceted KT intervention for improving EBP behavior of AHPs. The central element of the KT intervention was an online evidence-based information resource called the Evidence Alert System (EAS). The EAS contained actionable messages (5S pyramid level 4 and 5), clinical decision-making tools and used the ‘top-down’ approach . The other elements of the multifaceted intervention (workshop, mentoring and documentation changes) reinforced, educated and supported the approach set out in the EAS ensuring that the decision-making tools were embedded into the participant’s workflow. The secondary aims were to measure the effect of the KT intervention on EBP knowledge and attitudes. Our study sought to address key gaps in the current KT evidence by: using an RCT to measure the effect of a multi-component KT intervention centred around the EAS; measuring a wide range of EBP behaviors; and sampling a wide range of AHPs. Aims were measured at the individual participant level. Findings are reported according to the updated CONSORT statement for cluster randomized trials  (See Additional file 1).
Trial design and study setting
A multi-site evaluator-blinded, cluster RCT was conducted in a community based CP service in New South Wales (NSW), Australia. NSW is the largest state with a population of approximately 7.25 million people (32% of Australia’s total population). The CP service had 16 sites across NSW, organized into four geographically distinct regions, where AHP services were provided. Each region had centralized management for the sites within its boundaries including clinical seniors, professional development activities and mentoring, and thus was considered a natural cluster grouping. An independent officer not associated with the trial, used computer generated random numbers, to create four opaque envelopes based upon simple randomization. Four geographically distinct clusters were randomized to the intervention or control group. Cluster randomization was chosen to reduce risk of contamination that may have occurred if individuals working at the same site were randomized to different interventions. Individual participants were consented after randomization for pragmatic reasons. The first author (LC) obtained participants’ written consent and data collection took place before and after the workshops, at worksites or nearby locations, between June 2009 and August 2009.
The project was approved by the National Health and Medical Research Council Human Research Ethics Committee at Cerebral Palsy Alliance (Approval number: 2009-05-01) and University of Notre Dame Ethics Committee. The study was registered with Australian New Zealand Clinical Trials Registry (ACTRN12611000529943).
The theoretical model underpinning the project was the KTA process (Figure 1) developed by KT field leaders . The KTA process first, involves knowledge creation (i.e., production of research syntheses) and second, knowledge application (i.e., identification of the research-practice gap; adaption of the research syntheses to local context; identification of utilization barriers; selection of tailored KT strategies to redress barriers; and monitoring, evaluating and sustaining EBP implementation use). Emerging evidence suggests that KT interventions underpinned by theory may be superior to those that are not theoretical-informed, although more research is needed to confirm this . The advantage of theory-informed KT interventions is that they offer a generalizable framework for other researchers and organizations and provide guidance for designing KT interventions to overcome known barriers .
Assessment of barriers and facilitators
Theoretical basis and strategies to address modifiable barriers
Barrier: Lack of confidence/skill searching, appraising and synthesizing research evidence
Underpinning theory or group of theories
Problem based learning, learning styles
Workshops used problem based learning approach and a variety of approaches to ensure that different learning styles were catered to, maximizing the likelihood of increased confidence and skill levels
Accurate, relevant research evidence on cerebral palsy assessment and treatment was provided via the EAS building skill by modeling synthesis and summary of treatment areas. The EAS bypassed the need for high-level appraisal skills.
AHPs were included in the problem solving process during mentoring sessions and aimed to increase confidence and build skill base.
BARRIER: LACK OF TIME
Group of theories that the intervention relates to
The provision of accurate, relevant research evidence bypassed the need for extensive time spent searching and appraising research via databases and journals.
Paid EBP time in policy
Paid, protected time for AHPs to engage in EBP activities was provided
Changing policy suggested management ‘buy in’ and endorsement to support changes throughout the organization (leadership theory)
Documentation changes including a reminder system
Total quality management (TQM)
Patient documentation and work processes were reorganized to support clinical decision making and save time (reminder systems, checklists and directing participants to the EAS)
BARRIER: EVIDENCE CONSIDERED AS NOT CLINICALLY RELEVANT
Group of theories that the intervention relates to
Workshop teaching EAS
AHPs were involved in the problem solving process, so that they ‘owned’ and were a part of the process and could see the applicability of the EAS. Having the 8 week period in between workshops, allowed independent learning and time to apply the EAS information to a real client
Facilitators aimed to convince AHPs of the relevance of research in their area by exploring the EAS through clinical examples and role playing
An appealing product (the EAS) was developed and this was disseminated in a variety of ways (workshop, mentoring, documentation changes)
BARRIER: NO ACCESS TO FULL ARTICLES AND RESEARCH DATABASES
Group of theories that the intervention relates to
All staff members at every level of the organization had access to current cerebral palsy evidence and exchange of information via mentoring sessions and team meetings was promoted
BARRIER: SOME STAFF WITH NEGATIVE ATTITUDES TOWARDS EBP
Group of theories that the intervention relates to
Credible staff facilitated workshops, modeled positive attitudes and
emphasized ‘buy in’ from decision-makers in the organization
Mentors were selected with positive attitudes towards EBP so that target behavior was modeled
KT intervention with corresponding KTA phases
WHAT PART OF THE KTA CYCLE DID THE INTERVENTION IMPACT?
WHO IMPLEMENTED IT?
Strategic planning meetings
Policy changes (policies developed however not implemented until RCT)
Provision of paid, dedicated EBP time
Provision of a policy endorsed EBP mentoring program
Mandated and compulsory use of psychometrically sound outcome measures with all clients embedded in workflow e.g., included within mandatory Individual Family Service Plans
Evidence alert system development
During RCT (8-weeks; June – Aug 2009)
Skills training workshops (three-days)
Paid EBP time, mentoring, compulsory use of outcome measures (see policy changes above), documentation changes including reminder systems
Development of multifaceted intervention
Experimental group intervention
Part one, (two days) of the interactive workshop provided training to apply the EAS to decision-making within daily clinical work. A series of clinical examples were explored using the interface of the EAS, training about evidence levels, clinical decision-making algorithms and use of two psychometrically sound, cross disciplinary outcome measures.
Part two (one day) of the workshop eight weeks later involved participants presenting a case study detailing how they used the EAS to inform their clinical decision making with a real patient. This was followed by discussion with a small group of colleagues designed to help participants demonstrate the integration of their learning into their own clinical work.
Investigators and each senior clinician  led the workshops using knowledge brokering strategies . There was a mix of instructional techniques including didactic, interactive, role-playing and reflection. There was collaboration within and between professional groups.
On day-1 of the 3-day workshop, participants were informed that they had access to the EAS and that there were policy changes including: paid, quarantined EBP time; changes to client documentation including reminders to use the EAS; and embedding outcome measurement within workflow and mentoring by knowledge brokers.
The KT intervention was directed at the cluster level (three-day workshop-part one, access to the EAS and policy changes) and individual level (mentoring, and three-day workshop, part two). Details of the KT intervention are shown in Table 2.
The control group (total n = 62; region C = 29, region D = 33) received an equal intensity intervention about communication skills with no EBP content and no use of the EAS: three-day workshop about AHP-client communication skills and workplace supports (paid communication time, strategic planning, mentoring) related to communication skills. To minimize the risk of contamination, the control group was not informed about the EAS, paid EBP time, knowledge brokers, or mentoring until the end of the trial. The changes to documentation were not implemented in the control group clusters until the end of the RCT.
The primary endpoint was change in self- and peer-rated EBP behavior from baseline to eight weeks (individual and cluster level) measured using Goal Attainment Scaling (GAS) . Participants rated themselves against the self-GAS scales, and then to limit measurement bias, in a separate environment, a well-acquainted peer rated their performance on the peer-GAS scales. Selection of the GAS instrument increased study rigor because it overcame known instrumentation limitations in the KT literature surrounding EBP behavior measurement, including: responsivity—GAS has established validity, reliability, and exquisite responsivity to change, whereas systematic review evidence indicates that for nearly all valid and reliable EBP instruments, test responsivity is unknown ; tailoring—GAS is an individualized measure of change, and so progress towards any target behavior (including health professional behaviors ) could be validly, reliably, and sensitively measured, including tailored EBP behaviors unique to the study site, e.g., notifications to the CP Register; comprehensive measurement – GAS is an individualized measure of change, and so we could comprehensively measure all desired EBP behaviors, whereas systematic review evidence indicates that other psychometrically sound EBP instruments measure knowledge instead of behavior, or are limited because they only measure one discrete aspect of EBP behavior [29, 42, 44–46]; and lack of gold standard tool—accurate, gold-standard, flawless measurement of EBP behavior is not yet established in literature . Even though direct observation of EBP behavior (such as simulated patients, video/audio recordings of practice) is perceived as methodologically preferable to indirect (proxy) reports of EBP behavior (such as chart audit, patient report, self-report, or peer-report), systematic review evidence indicates that direct measures often fail validity testing . This could have introduced other flaws to our clinical trial. Moreover, collecting direct measures throughout NSW, being a state-wide service, would have introduced prohibitive trial costs (NSW’s landmass is 3.25 times larger than the United Kingdom, and is larger than California and New Mexico combined), when the cost-benefit of a potentially invalid measure is weighed-up. Even though self-report proxy measures are an imperfect measure of actual behavior , leading KT agencies, such as the Canadian Institutes of Health Research advocate for self-report because the process of self reflection plays a critical role in initiating behavioral changes within organizations. In light of current EBP behavior measurement limitations, GAS offered the best way forward since it was psychometrically sound, it comprehensively measured EBP behavior, was practical across an entire state and could be tailored to the study site.
The GAS scales were devised by a multidisciplinary panel of experts familiar with EBP behaviors of the eligible AHPs, as per literature recommendations for scale establishment. Twenty-five goal scales were developed, one-half relating to EBP behaviors and the other one-half relating to communication behavior for the control group. The scales measured EBP behaviors such as: use of gold standard goal-setting tools to plan services; use of CP classification systems to accurately prognosticate; use of evidence (e.g., via the EAS) to quickly choose evidence-based classification systems, interventions and outcome measures; and use of gold standard outcome measures to routinely evaluate services. The GAS scales are available from the corresponding author by request. As per the test manual, raw scores were converted to GAS T-scores, enabling inferential statistical analysis of continuous data.
Self- and peer-rated attitude changes were measured using subsets three and four of the Evidence-Based Practice Attitude Scale (EBPAS) , which is psychometrically permissible. EBP knowledge was measured via open-ended exam questions with right/wrong answers, pre-defined by the panel of experts, derived from published evidence.
EAS utilization was measured by number of web page hits collected via a software program that tracked cluster-specific IP addresses in batches. Web hit data collection was concealed from participants, minimizing the likelihood of observer bias affecting EAS use.
An adverse event log was not required because the intervention was educational in nature and therefore posed no risk.
Blinding was judiciously applied wherever pragmatically possible, resulting in a single-blinded trial. This included: independent evaluator blinding to group allocation and phase of the trial when scoring outcome data; and partial participant and facilitator blinding to the specific EBP behavior of interest to the investigators. Participants and workshop facilitators were clearly aware of the content of the workshops, however were not aware of which intervention (KT intervention or communication skills) was of specific interest to the researchers. Fidelity of the evaluator blinding was not formally investigated.
We sought to test the efficacy of an organizational KT intervention and therefore conducted the study within one agency, which is the largest of its kind in Australia. This methodological decision imposed pragmatic limitations on the obtainable sample frame. We successfully recruited 88% of the available sampling frame, however the total number of employees at the agency was less than the number of participants required to reach statistical power if correlation of outcome variables within sites was observed (intra-cluster correlation). A sample size calculation identified the probability of detecting an effect size of 1 at an alpha level of 0.05 (one-tail) and a power of 90%. For Goal Attainment Scaling [mean T-score = 50, standard deviation (sd) = 10] an improvement of 10 points or more in the KT intervention group than the control group was sought, (improvement of 1 sd). The expert panel agreed that a 10-point increase in GAS T-scores equated to significant clinical improvement in EBP behaviors. The calculation assumed a 20% non-consent rate and a 20% attrition rate indicating a sample size requirement of 72 (38 per group) for a non-cluster trial. We enrolled 135 professionals (n = 73 interventions and n = 62 controls) at four sites. Based on estimating an intra-cluster correlation co-efficient (ICC) of 0.1 we calculated that the study was underpowered to demonstrate an improvement of 10 points between groups if a cluster effect of this size was observed (Variance Inflation Figure = 4.3).
All statistical analysis was carried out with individual participants as the unit of analysis on an intention-to-treat basis by using SPSS for Windows 19.0.0 (SPSS Inc, Chicago, IL) and SAS 9.3 (SAS Institute, Cary NC).
We conducted generalized linear regression analysis for primary and secondary endpoints, using post intervention GAS T-score as the outcome variable and adjusting for potential confounding variables (baseline GAS T-score, profession, group allocation, grade level, and years in the disability field). Effect sizes with 95% confidence intervals (CIs) were calculated and significance was set at 0.05. These estimates would underestimate the standard errors and confidence intervals for the effect size if participant outcomes are correlated within cluster sites, thus mixed effects models with cluster included as a random effect were used to adjust for a cluster effect to calculate the effect size for each outcome . ICC was calculated from the mixed effects model and bootstrapping (1,000 samples generated) was performed to calculate 95% confidence intervals for the ICC.
Baseline characteristics of participants
n = 73 (%)
n = 62 (%)
Level 2 (clinical specialist)
Level 3 (clinical senior)
Manager or other
Years’ experience in disability field
2-4 years 11 months
5-9 years 11 months
Previous EBP continuing education?
Primary and secondary outcomes
Treatment n = 73
Control n = 62
Mixed effects model
Difference (95% CI)
ICC (95% CI)
Difference (95% CI)
EAS page hits**
EBP attitude EBPAS
Self subset 3
Peer subset 3
Effectiveness of KT intervention
Primary outcome—EBP behaviors
Self-rated GAS T-scores increased more in the intervention group compared to controls however this difference was not statistically significant after adjusting for the cluster effect; Effect size 4.43 [95% CI -10.63 to 19.49 (p = 0.56)] (Table 4). Baseline self-rated GAS T-scores were a predictor in the model [Effect size 0.71 (95% CI 0.52–0.90) (p < 0.0001)]; indicating lower performers improved but remained lower performers, and higher performers improved and remained leading performers. No other covariates were significantly predictive of outcome.
Peer-rated GAS T-scores of the intervention group also increased compared to controls, but this difference was also not statistically significant after adjusting for the cluster effect: effect size 6.75 [95% CI -16.95 to 30.44 (p = 0.57)] (Table 4). Similar to the self-rated GAS T-scores, the final peer-rated GAS T-score was predicted by the baseline peer-rated GAS T-score [effect size 0.30 (95% CI 0.150.45) (p < 0.0001)]. No other covariates were significantly predictive of peer-rated GAS T-scores. The peer-rated GAS T-scores for each cluster mirrored the self-rated GAS cluster T-scores, suggesting the observed study effects were behaviorally meaningful, despite low study power to demonstrate a statistically significant difference.
Secondary outcomes—EBP knowledge and attitudes
EBP knowledge scores increased compared to controls, with a statistically significant effect size of 2.97 (95% CI 1.97, 3.97, p < 0.0001). The ICC for this outcome was zero, and this effect remained statistically significant after adjusting for the cluster effect: 2.97 (95% CI 1.97, 3.97, p < 0.0001). Baseline score (p < 0.0001) and professional category (p = 0.03) were also predictors in the model. There was minimal to no correlation between participants within sites for self- or peer-rated EBP attitudes, however we did not demonstrate a statistically significant intervention effect (Table 4). The intervention group accessed the EAS more than the control group (KT intervention group 6,123 total hits; control group 1,677 hits).
Mean outcome scores for each cluster
Outcome score N, mean (sd) per cluster
Cluster 1 (Exp)
Cluster 2 (Exp)
Cluster 3 (Control)
Cluster 4 (control)
Self EBPAS subset 3 score
Self EBPAS subset 4 score
Peer EBPAS subset 3 score
Peer EBPAS subset 4 score
We conducted a cluster RCT to evaluate whether a multifaceted KT strategy changed AHP’s EBP behaviors. Both clusters in the KT intervention group improved within the study period, but not statistically significantly more than the control group. We consider this null finding to be a probable type II error because our study was underpowered owing to the fact that the number of participants required to account for clustering of EBP behaviors within sites exceeded the number of employees available. Our study demonstrated increased use of our evidence-based resource (the EAS), however we were unable to confirm that this translated to a statistically significant change in EBP behavior. This finding is in line with previous research involving evidence-based resources [27, 28]. Owing to the type II error, we remain unsure of the true effect of our KT intervention, but we discovered a number of potentially important findings that may contribute to future KT endeavours and the body of research.
The high ICCs (ranging from 0.33 to 0.64) for EBP behavior measures, indicated substantial correlation of behaviors within clusters, and indicated differences in behaviors between clusters. When we examined the mean change scores for each cluster, one of the four clusters (cluster 4), which was randomly allocated to the control group, was an obvious outlier with high baseline GAS T-scores, high baseline knowledge scores and increased self- and peer-rated GAS T-scores over the study period.
Variability between natural groupings (such as clinical, departmental or regional) has been noted in the KT literature previously [29, 32]. Perhaps the high baseline EBP scores for the cluster 4 reflected positive EBP culture and practices due to cluster 4’s manager [32, 50, 51]. The notion that a manager can strongly influence research culture is by no means new [29, 52], as some opinion leaders are known to strongly influence EBP behavior [50, 53]. The cluster 4’s manager was active in promoting EBP behavior among staff. A large range of KT interventions were in place in cluster 4 prior to this study, including audit and feedback, financial incentives, workshops and mentoring. It is conceivable that cluster 4 therefore had both better readiness and receptivity to EBP supports as they had essentially been engaging in active KT for a longer period than the other clusters . That said, positive EBP culture is considered to be related to positive EBP attitudes  and EBPAS scores measuring attitude change of cluster 4 were no different from the other clusters at baseline or eight weeks. This may have reflected measurement error, or may indicate that positive attitudes in cluster 4 were not necessary as mandatory policies within that cluster were the driving force behind the higher GAS scores.
Our hypothesis that the KT intervention would improve knowledge was supported with the KT intervention group knowledge exam scores showing a statistically significant improvement compared to the control group. This finding supports previous research suggesting that knowledge change alone does not consistently translate into behavior change [7, 32, 37, 54]. Interestingly, change in knowledge scores was not affected by the cluster effect suggesting that knowledge is not as susceptible to peer influences as behavior.
We found no correlation between behavior, knowledge, and attitude change scores within and between clusters. Attitudes remained unchanged. We hypothesise the lack of change in EBP attitudes in our study may be explained by: (1) high baseline EBP attitudes and there was conceivably a ceiling effect on the EBPAS. This was plausible as EBP had been a focus in the organization for some time prior to the RCT. In this case, positive attitudes at baseline, increased knowledge scores and policy changes may together have resulted in the behaviorally meaningful changes observed. There is however no normative data for AHPs on the EBPAS, so it is difficult to say whether or not baseline attitudes were high compared to AHPs in other organisations; (2) EBPAS subsets potentially not being sensitive enough to detect attitude change and the psychometrics for sensitivity in this population are unknown; (3) the EBPAS being an accurate, sensitive measure and that attitudes did not improve from the KT intervention. This third possibility supports the notion that improved knowledge was not adequate to lead to statistically significant behavior change, and that a shift in attitudes was also needed . Conversely, the behaviorally significant change that was observed potentially bypassed the need for attitude change by employing strategies such as mandatory use of documentation and outcome measures; and (4) EBP attitudes taking a longer period of time than knowledge to change, and the eight-week trial was too short to detect change.
Strengths and limitations
The study had a number of strengths including the rigorous design and broad robust behavior measurement. Our chosen measurement instrument (GAS) was sensitive to change [56, 57] and appeared accurate as self- and peer-rated scores mirrored each other. Distinguishing features of our study were that we measured a wide set of behaviors among AHPs working with people with CP. The mix of AHPs in our sample is fairly representative of other community based disability organizations, increasing external validity. This is the first RCT in the KT literature involving social workers, psychologists or occupational therapists . The KT intervention itself was a study strength being based on a solid theoretical model [21–23], in response to a comprehensive barriers assessment, with desired outcomes clearly defined, and included a range of interventions, not only educational interventions .
There are a number of study limitations. First and foremost, the pragmatic constraints that limited the number of available clusters and participants led to low statistical power causing a probable type II error. Second, the large differences observed between clusters suggest that we potentially should have tailored the KT intervention to each cluster rather than the whole organization. Third, the evidence base regarding whether proxy behavior measures represent actual behavior is not firmly established, but with preferred rival direct measures also lacking validity and reliability [41, 58]. Moreover, direct measurement was not affordable in our study given the geography involved, and indirect measurement tools were therefore used [43, 59]. To minimize measurement bias, systematic review recommendations regarding indirect measures were followed, and included using: acceptable indirect measures [41, 59] (such as self- and peer-rated behavior triangulated with unbiased web hit data) ; measurement tools with strong psychometric properties ; more than one tool to measure behavior change ; and a sound theoretical model as a basis of the intervention . The time frame of the trial was short considering that many EBP behaviors and system/organizational changes (such as documenting client goals and mentoring) take time to develop . A follow-up study is needed to measure whether the EBP behaviors were sustained . The return rate of the GAS exam form and EBPAS was not perfect (60-82%), with the eight-week data having more missing data.
KT literature recommends tailoring KT interventions to overcome known barriers within organizations [19, 20], however our findings suggest that this may need to go even further with KT interventions being designed for subgroups within an organization. The impact of different workplace culture may mean that there are dramatically different barriers needing different KT interventions to be effective . Considering the importance of management-led change, targeting policy makers and managers may be beneficial. This has been done in the public health sector , however no studies customizing KT to policy makers/management was found in the allied health literature. Our study provides extremely rich pilot study data to planning and conducting an adequately powered cluster RCT in future.
Our study highlighted the methodological challenges of conducting empirical research in a community-based organization with fixed cluster and participant numbers. Whether or not RCTs are a feasible option in community organizations is debatable, and it may be that other research designs are more appropriate [29, 61]. Researchers, policy makers, and clients need to effectively collaborate to ensure that reliable, relevant research becomes embedded into everyday care in a timely way. Considering that the cornerstone of KT is access to reliable research, the authors plan to make the EAS publically available.
The authors would like to thank Cerebral Palsy Alliance for their support of this study, for understanding the importance of EBP and adopting systemic changes. The authors also wish to acknowledge the clinical consultants (Cathy Morgan, Salli-Ann Craik, Natalie Morton, Leigha Dark and Elise Stumbles) and research staff at Cerebral Palsy Alliance for their leadership, contributions and assistance, and most importantly we would like to thank staff for their participation in the study.
- Reddihough DS, Collins KJ: The epidemiology and causes of cerebral palsy. Aust J Physiother. 2003, 49: 7-14.View ArticlePubMedGoogle Scholar
- Novak I, Hines M, Goldsmith S, Barclay R: Clinical prognostic messages from a systematic review on cerebral palsy. Pediatrics. 2012, 130: e1285-e1312. 10.1542/peds.2012-0924.View ArticlePubMedGoogle Scholar
- Straus S, Haynes R: Managing evidence-based knowledge: the need for reliable, relevant and readable resources. Can Med Assoc J. 2009, 180: 942. 10.1503/cmaj.081697.View ArticleGoogle Scholar
- Heiwe S, Kajermo KN, Tyni-LennÈ R, Guidetti S, Samuelsson M, Andersson IL, Wengstrom Y: Evidence-based practice: attitudes, knowledge and behaviour among allied health care professionals. Int J Qual Health Care. 2011, 23 (2): 198-209. 10.1093/intqhc/mzq083.View ArticlePubMedGoogle Scholar
- Stevenson K, Lewis M, Hay E: Do physiotherapists' attitudes towards evidence-based practice change as a result of an evidence-based educational programme?. J Eval Clin Pract. 2004, 10: 207-217. 10.1111/j.1365-2753.2003.00479.x.View ArticlePubMedGoogle Scholar
- Davis D: Continuing education, guideline implementation, and the emerging transdisciplinary field of knowledge translation. J Contin Educ Health Prof. 2006, 26: 5-12. 10.1002/chp.46.View ArticlePubMedGoogle Scholar
- Novak I, McIntyre S: The effect of education with workplace supports on practitioners' evidence-based practice knowledge and implementation behaviours. Aust Occup Ther J. 2010, 57 (6): 93-386.View ArticleGoogle Scholar
- Saleh M, Korner-Bitensky N, Snider L, Malouin F, Mazer B, Kennedy E, Roy MA: Actual vs. best practices for young children with cerebral palsy: A survey of paediatric occupational therapists and physical therapists in Quebec, Canada. Dev Neurorehabil. 2008, 11: 60-80. 10.1080/17518420701544230.View ArticlePubMedGoogle Scholar
- Hanna SE, Russell DJ, Bartlett DJ, Kertoy M, Rosenbaum PL, Wynn K: Measurement practices in pediatric rehabilitation: a survey of physical therapists, occupational therapists, and speech-language pathologists in Ontario. Phys Occup Ther Pediatr. 2007, 27: 25-42.PubMedGoogle Scholar
- O'Connor S, Pettigrew C: The barriers perceived to prevent the successful implementation of evidence based practice by speech and language therapists. Int J Lang Comm Disord. 2009, 44: 1018-1035.Google Scholar
- McCluskey A: Occupational therapists report a low level of knowledge, skill and involvement in evidence-based practice. Aust Occup Ther J. 2003, 50: 3-12. 10.1046/j.1440-1630.2003.00303.x.View ArticleGoogle Scholar
- Salbach N, Jaglal S, Korner-Bitensky N, Rappolt S, Davis D: Practitioner and organizational barriers to evidence-based practice of physical therapists for people with stroke. Phys Ther. 2007, 87: 1284. 10.2522/ptj.20070040.View ArticlePubMedGoogle Scholar
- Glasziou P, Ogrinc G, Goodman S: Can evidence-based medicine and clinical quality improvement learn from each other?. BMJ Qual Saf. 2011, 20: i13. 10.1136/bmjqs.2010.046524.View ArticlePubMedPubMed CentralGoogle Scholar
- Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009, 2: CD003030-PubMedGoogle Scholar
- Parmelli E, Flodgren G, Beyer F, Baillie N, Schaafsma ME, Eccles MP: The effectiveness of strategies to change organisational culture to improve healthcare performance: a systematic review. Implement Sci. 2011, 6: 33. 10.1186/1748-5908-6-33.View ArticlePubMedPubMed CentralGoogle Scholar
- O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA: Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007, 4: CD000409-PubMedGoogle Scholar
- Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD: Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006, 2: CD000259-PubMedGoogle Scholar
- Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J: The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009, 3: CD001096-PubMedGoogle Scholar
- Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE: Knowledge translation of research findings. Implement Sci. 2012, 7: 50. 10.1186/1748-5908-7-50.View ArticlePubMedPubMed CentralGoogle Scholar
- Davies HTO, Powell AE, Rushmer RK: Healthcare professionals’ views on clinician engagement in quality improvement. 2007, Available at: http://www.health.org.uk/publications/engaging-clinicians-report/ (Accessed: 28/07/2011)Google Scholar
- Davies P, Walker AE, Grimshaw JM: A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010, 5: 14. 10.1186/1748-5908-5-14.View ArticlePubMedPubMed CentralGoogle Scholar
- Graham Logan J, Harrison M, Straus S, Tetroe J, Caswell W, Robinson N: Lost in knowledge translation: Time for a map?. J Contin Educ Health Prof. 2006, 26: 13-24. 10.1002/chp.47.View ArticleGoogle Scholar
- Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M: Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007, 85: 93-138. 10.1111/j.1468-0009.2007.00478.x.View ArticlePubMedPubMed CentralGoogle Scholar
- McKinlay RJ, Cotoi C, Wilczynski NL, Haynes RB: Systematic reviews and original articles differ in relevance, novelty, and use in an evidence-based service for physicians: PLUS project. J Clin Epidemiol. 2008, 61: 449-454. 10.1016/j.jclinepi.2007.10.016.View ArticlePubMedGoogle Scholar
- Badgett R: Why would physicians undervalue reviews by the Cochrane Collaboration?. J Slin Epidemiol. 2008, 61: 419-421. 10.1016/j.jclinepi.2007.11.022.View ArticleGoogle Scholar
- Chambers D, Wilson PM, Thompson CA, Hanbury A, Farley K, Light K: Maximizing the impact of systematic reviews in health care decision making: a systematic scoping review of knowledge translation resources. Milbank Q. 2011, 89: 131-156. 10.1111/j.1468-0009.2011.00622.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Gülmezoglu A, Langer A, Piaggio G, Lumbiganon P, Villar J, Grimshaw J: Cluster randomised trial of an active, multifaceted educational intervention based on the WHO Reproductive Health Library to improve obstetric practices. BJOG: Int J Obstet Gynaecol. 2007, 114: 16-23.View ArticleGoogle Scholar
- Haynes R, Holland J, Cotoi C, McKinlay R, Wilczynski N, Walters L, Jedras D, Parrish R, McKibbon K: McMaster PLUS: A cluster randomized clinical trial of an intervention to accelerate clinical use of evidence-based information from digital libraries. J Am Med Inform Assoc. 2006, 13: 593-600. 10.1197/jamia.M2158.View ArticlePubMedPubMed CentralGoogle Scholar
- Dobbins M, Hanna S, Ciliska D, Manske S, Cameron R, Mercer S, O'Mara L, DeCorby K, Robeson P: A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implement Sci. 2009, 4: 61. 10.1186/1748-5908-4-61.View ArticlePubMedPubMed CentralGoogle Scholar
- Bekkering G, Hendriks H, Van Tulder M, Knol D, Hoeijenbos M, Oostendorp R, Bouter L: Effect on the process of care of an active strategy to implement clinical guidelines on physiotherapy for low back pain: a cluster randomised controlled trial. Br Med J. 2005, 14: 107-Google Scholar
- Rebbeck T, Maher C, Refshauge K: Evaluating two implementation strategies for whiplash guidelines in physiotherapy: A cluster-randomised trial. Austr J Physiother. 2006, 52: 165. 10.1016/S0004-9514(06)70025-3.View ArticleGoogle Scholar
- Pennington L, Roddam H, Burton C, Russell I, Russell D: Promoting research use in speech and language therapy: a cluster randomized controlled trial to compare the clinical effectiveness and costs of two training strategies. Clin Rehabil. 2005, 19: 387. 10.1191/0269215505cr878oa.View ArticlePubMedGoogle Scholar
- Stevenson K, Lewis M, Hay E: Does physiotherapy management of low back pain change as a result of an evidence-based educational programme?. J Eval Clin Pract. 2006, 12: 365-375. 10.1111/j.1365-2753.2006.00565.x.View ArticlePubMedGoogle Scholar
- Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer H, Kunz R: Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. BMJ. 2002, 325: 1338. 10.1136/bmj.325.7376.1338.View ArticlePubMedPubMed CentralGoogle Scholar
- Ramos KD, Schafer S, Tracz SM: Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003, 326: 319. 10.1136/bmj.326.7384.319.View ArticlePubMedPubMed CentralGoogle Scholar
- Dizon JM, Grimmer-Somers KA, Kumar S: Current evidence on evidence-based practice training in allied health: a systematic review of the literature. Int J Evid Based Healthc. 2012, 10: 347-360. 10.1111/j.1744-1609.2012.00295.x.View ArticlePubMedGoogle Scholar
- Scott SD, Albrecht L, O'Leary K, Ball GDC, Hartling L, Hofmeyer A, Jones CA, Klassen TP, Burns KK, Newton AS: Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012, 7: 70. 10.1186/1748-5908-7-70.View ArticlePubMedPubMed CentralGoogle Scholar
- Campbell MK, Piaggio G, Elbourne DR, Altman DG: Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012, 345: e5661. 10.1136/bmj.e5661.View ArticlePubMedGoogle Scholar
- Flodgren G, Parmelli E, Doumit G, Gattellari M, O'Brien MA, Grimshaw J, Eccles MP: Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Library. 2011, 8-Google Scholar
- Russell D, Rivard L, Walter S, Rosenbaum P, Roxborough L, Cameron D, Darrah J, Bartlett D, Hanna S, Avery L: Using knowledge brokers to facilitate the uptake of pediatric measurement tools into clinical practice: A before-after intervention study. Implement Sci. 2010, 5: 92. 10.1186/1748-5908-5-92.View ArticlePubMedPubMed CentralGoogle Scholar
- Kiresuk T, Sherman R: Goal attainment scaling: A general method for evaluating comprehensive community mental health programs. Community Ment Health J. 1968, 4: 443-453. 10.1007/BF01530764.View ArticlePubMedGoogle Scholar
- Shaneyfelt T, Baum K, Bell D, Feldstein D, Houston T, Kaatz S, Whelan C, Green M: Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006, 296: 1116. 10.1001/jama.296.9.1116.View ArticlePubMedGoogle Scholar
- Cusick A, Ottenbacher K: Goal attainment scaling: continuing education evaluation tool. J Contin Educ Health Prof. 1994, 14: 141-154. 10.1002/chp.4750140303.View ArticleGoogle Scholar
- Curran JA, Grimshaw JM, Hayden JA, Campbell B: Knowledge translation research: The science of moving research into policy and practice. J Contin Educ Health Prof. 2011, 31: 174-180. 10.1002/chp.20124.View ArticlePubMedGoogle Scholar
- Straus SE, Ball C, Balcombe N, Sheldon J, McAlister FA: Teaching evidence based medicine skills can change practice in a community hospital. J Gen Intern Med. 2005, 20: 340-343. 10.1111/j.1525-1497.2005.04045.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Lucas BP, Evans AT, Reilly BM, Khodakov YV, Perumal K, Rohr LG, Akamah JA, Alausa TM, Smith CA, Smith JP: The impact of evidence on physicians,Äô inpatient treatment decisions. J Gen Intern Med. 2004, 19: 402-409. 10.1111/j.1525-1497.2004.30306.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Hrisos S, Eccles M, Francis J, Dickinson H, Kaner E, Beyer F, Johnston M: Are there valid proxy measures of clinical behaviour? a systematic review. Implement Sci. 2009, 4: 37. 10.1186/1748-5908-4-37.View ArticlePubMedPubMed CentralGoogle Scholar
- Aarons G: Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS). Ment Health Serv Res. 2004, 6: 61-74.View ArticlePubMedPubMed CentralGoogle Scholar
- Donner A, Klar N, Klar NS: Design and analysis of cluster randomization trials in health research. 2000, London: ArnoldGoogle Scholar
- Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004, 82: 581-629. 10.1111/j.0887-378X.2004.00325.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Rogers EM: Diffusion of innovations. 1995, New York: Free PressGoogle Scholar
- Aarons G, Sawitzky A: Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Adm Policy Ment Health Ment Health Serv Res. 2006, 33: 289-301. 10.1007/s10488-006-0039-1.View ArticleGoogle Scholar
- French B, Thomas L, Baker P, Burton C, Pennington L, Roddam H: What can management theories offer evidence-based practice? A comparative analysis of measurement tools for organisational context. Implement Sci. 2009, 4: 28. 10.1186/1748-5908-4-28.View ArticlePubMedPubMed CentralGoogle Scholar
- McCluskey A, Lovarini M: Providing education on evidence-based practice improved knowledge but did not change behaviour: a before and after study. BMC Med Educ. 2005, 5: 40. 10.1186/1472-6920-5-40.View ArticlePubMedPubMed CentralGoogle Scholar
- Graham ID, Bick D, Tetroe J, Straus SE, Harrison MB: Measuring outcomes of evidence-based practice: Distinguishing between knowledge use and its impact. Eval Impact Implement Evidence-Based Pract. 2010, 1: 18-Google Scholar
- Steenbeek D, Gorter JW, Ketelaar M, Galama K, Lindeman E: Responsiveness of goal attainment scaling in comparison to two standardized measures in outcome evaluation of children with cerebral palsy. Clin Rehabil. 2011, 25: 1128-1139. 10.1177/0269215511407220.View ArticlePubMedGoogle Scholar
- Flodgren G, Eccles M, Shepperd S, Scott A, Parmelli E, Beyer F: An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes. Cochrane Database Syst Rev (Online). 2011, 7: CD009255-Google Scholar
- Dickinson HO, Hrisos S, Eccles MP, Francis J, Johnston M: Statistical considerations in a systematic review of proxy measures of clinical behaviour. Implement Sci. 2010, 5: 20. 10.1186/1748-5908-5-20.View ArticlePubMedPubMed CentralGoogle Scholar
- Eccles M, Hrisos S, Francis J, Kaner E, Dickinson H, Beyer F, Johnston M: Do self- reported intentions predict clinicians' behaviour: a systematic review. Implement Sci. 2006, 1: 28. 10.1186/1748-5908-1-28.View ArticlePubMedPubMed CentralGoogle Scholar
- Thomson O, Freemantle N, Oxman A, Wolf F, Davis D, Herrin J: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database System Rev (Online). 2001, 1: CD003030-Google Scholar
- Walshe K: Understanding what works–and why–in quality improvement: the need for theory-driven evaluation. Int J Qual Health Care. 2007, 19: 57-59. 10.1093/intqhc/mzm004.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.