Thrombolysis ImPlementation in Stroke (TIPS): evaluating the effectiveness of a strategy to increase the adoption of best evidence practice – protocol for a cluster randomised controlled trial in acute stroke care
- Christine L Paul1, 2Email author,
- Christopher R Levi1, 2, 3,
- Catherine A D’Este4,
- Mark W Parsons3,
- Christopher F Bladin5,
- Richard I Lindley6, 7,
- John R Attia1, 2,
- Frans Henskens1, 2,
- Erin Lalor8,
- Mark Longworth9,
- Sandy Middleton10,
- Annika Ryan1, 2,
- Erin Kerr3,
- Robert W Sanson-Fisher1, 2 and
- the Thrombolysis ImPlementation in Stroke (TIPS) Study Group
© Paul et al.; licensee BioMed Central Ltd. 2014
Received: 3 December 2013
Accepted: 24 February 2014
Published: 25 March 2014
Stroke is a leading cause of death and disability internationally. One of the three effective interventions in the acute phase of stroke care is thrombolytic therapy with tissue plasminogen activator (tPA), if given within 4.5 hours of onset to appropriate cases of ischaemic stroke.
To test the effectiveness of a multi-component multidisciplinary collaborative approach compared to usual care as a strategy for increasing thrombolysis rates for all stroke patients at intervention hospitals, while maintaining accepted benchmarks for low rates of intracranial haemorrhage and high rates of functional outcomes for both groups at three months.
Methods and design
A cluster randomised controlled trial of 20 hospitals across 3 Australian states with 2 groups: multi- component multidisciplinary collaborative intervention as the experimental group and usual care as the control group. The intervention is based on behavioural theory and analysis of the steps, roles and barriers relating to rapid assessment for thrombolysis eligibility; it involves a comprehensive range of strategies addressing individual-level and system-level change at each site. The primary outcome is the difference in tPA rates between the two groups post-intervention. The secondary outcome is the proportion of tPA treated patients in both groups with good functional outcomes (modified Rankin Score (mRS <2) and the proportion with intracranial haemorrhage (mRS ≥2), compared to international benchmarks.
TIPS will trial a comprehensive, multi-component and multidisciplinary collaborative approach to improving thrombolysis rates at multiple sites. The trial has the potential to identify methods for optimal care which can be implemented for stroke patients during the acute phase. Study findings will include barriers and solutions to effective thrombolysis implementation and trial outcomes will be published whether significant or not.
Australian New Zealand Clinical Trials Registry: ACTRN12613000939796
KeywordsCollaborative intervention Thrombolysis Acute stroke Evidence-based practice Quality improvement Cluster randomised controlled trial Multidisciplinary approach
Effective treatments for stroke can provide major reductions in health burden
Stroke is a major cause of death and disability internationally, with over 5 million deaths from stroke worldwide . Stroke accounted for approximately 1 in every 19 deaths in the United States in 2009 , with US projections indicating that an additional 4 million people will have had a stroke by 2030 , a 22% increase in prevalence from 2013 . The mean cost per person for stroke care in the United States in 2009 was estimated at $6,018 . In 2009, The Australian Bureau of Statistics reports an estimated 381,400 Australians (1.8% of the total population) had suffered a stroke , with 35% reporting at least one impairment as a result of that stroke  and 62% of these reporting their stroke as their main disabling condition . Stroke was the underlying cause of death for 11,220 Australians in 2010  and is the second leading cause of death and a leading cause of long-term disability . In 2012, the total financial cost of stroke was around $5 billion  with an estimated burden of disease cost of $49.3 billion . In 2012, there were nearly 50,000 strokes in Australia, with approximately 12,000 people dying as a result of their stroke . In 2009 to 2010, 35,345 hospitalisations were recorded in Australia with a principal diagnosis of stroke  and an average length of stay of 9 days . The majority of strokes (89%) are admitted to hospital , and approximately 50% of sufferers are left either deceased or dependent .
Improved outcomes for acute stroke can be achieved by: use of aspirin within 48 hours to provide a modest absolute risk reduction of 1% ; Stroke Care Units (SCU) ; and thrombolytic therapy using intravenous tissue plasminogen activator (tPA) administered to appropriate patients according to guidelines and within 4.5 hours of symptom onset [14, 15]. A systematic review of randomised trials of intravenous tPA within 6 hours of acute ischaemic stroke onset found intravenous tPA significantly increased the odds of being alive and independent (modified Rankin Scale of 0 to 2) at 3-month follow-up for those treated within 3 hours of onset . Sustained improvements in functional outcome and health-related quality of life have been identified at up to 18-month follow-up . Treatment benefits for tPA are highest if administered within 3 hours of stroke onset [16, 18, 19] but can extend beyond the 3-hour window , with tPA considered safe for eligible patients up to 4.5 hours after stroke onset . The ‘number needed to treat’ with tPA to prevent death and dependency is between 6 and 14, but may be higher depending on the favourable outcome definition and method of calculation [15, 22, 23]. Although there has been some debate about the strength of the evidence provided by thrombolysis [24, 25], the treatment is endorsed via national and international guidelines [26–29].
Thrombolysis is a cost-effective stroke treatment when used appropriately
Cost-effectiveness analyses have indicated that high-cost treatments for acute stroke can provide major public health benefits even when minor reductions in disability are achieved . A number of studies have identified thrombolysis as being a cost-effective treatment when used appropriately [31–34]. A UK study estimated that over a lifetime, tPA was associated with cost-savings of £96,565 per quality-adjusted life year . The Model of Resource Utilization, Costs, and Outcomes for Stroke (MORUCOS) study found that tPA saved 155 disability-adjusted life years in a group of 256 eligible patients and concluded that tPA is more effective and cost-saving than treatment with aspirin .
Rates of thrombolysis remain low internationally
Despite evidence for the cost-effectiveness and safety of thrombolysis along with international recommendations being in place for a number of years [21, 35, 36], there has been little overall growth in thrombolysis rates. Thrombolysis rates have remained relatively low in a number of countries over the past decade: In the US, tPA administration rates increased from 0.87% in 2001 to 2.40% in 2006 . In the UK in 2008, while an estimated 26% of acute stroke patients were eligible for thrombolysis, only 1.4% were administered the treatment . Only 4.3% of ischemic stroke patients in the California Acute Stroke Pilot Registry received thrombolysis . In Australia, despite making major progress in terms of wide establishment of stroke care units, tPA rates are approximately 7% .
Achieving large-scale increases in rates of thrombolysis is challenging
Framework for situational analysis exploring phases of care, tasks, staff roles and time frames
1. Pre-hospital assessment
Assess potential tPA eligibility
<2 h of onset
Whether probable stroke or stroke mimic, define time of onset, consider comorbidity/frailty, estimate for survival
Deliver to tPA-capable hospital
Handover to triage or Acute Stroke Team
Assess tPA eligibility
ED or SCU
<3.5 h of onset
Onset time, collateral history, mRS, anticoagulants, seizure, serious or advanced terminal illness, history of intracranial haemorrhage or subarachnoid haemorrhage, major internal surgery in last 21 days, heart attack with IV thrombolysis in last 72 hours, stroke severity assessment with National Institutes of Health Stroke Scale, Glasgow Coma Scale, observations
Handover to SCU
to Acute Stroke Team
3. Clinical assessment
Assess tPA eligibility
Onset time certainty, functional independence, comorbidity, current medication, relative and absolute contraindications, NIHSS score, collect blood, notify CT scanning & organise transport, organise ECG
Handover to Imaging
Non- contrast CT scan (or perfusion CT or MRI)
Handover to SCU
to SCU Nurse
5. Final clinical assessment
Confirm tPA eligibility
Review information from phase 1-3, focussed history to check for stroke mimic, onset certainty, premorbid functional independence, medication, contraindications, NIHSS, rapid cardiac and vascular screen, check blood sent to lab, review ECG, review NCCT on console with radiographer and again for degree of ischaemic change & possibility of stroke mimic, discuss scenarios with patient and family
6. Preparation & delivery
<4.5 h of onset
Check serum glucose level, anticoagulant medication, International Normalized Ratio, Blood Pressure, NIHSS.
Treat serum glucose or Blood Pressure if necessary and reassess
Administer thrombolysis, with amount based on estimated patient weight
Monitor and manage neurological status (NIHSS at 0 h, 1 h, 24 hrs), blood pressure and serum glucose level
24 h from tPA delivery
Achieving multi-site change in thrombolysis rates for acute stroke poses an even more complex challenge, with very few published trials. One study looked at the benefits and challenges of establishing stroke units , and two trials have explored thrombolysis in particular. The US Get-With-The Guidelines stroke project – a national quality improvement program – demonstrated an increase in adherence to guideline recommendations for stroke from a baseline rate for intravenous thrombolytics of 42.09% to 72.84% in the fifth year of the project . The PRomoting ACute Thrombolysis in Ischemic StrokE (PRACTISE) cluster randomised trial in The Netherlands used the Breakthrough approach – a quality improvement methodology using Plan-Do-Study-Act (PDSA) cycles – to change clinical practice . The PRACTISE trial found that a significantly higher proportion of intervention group ischemic stroke patients presenting within four hours of stroke onset were thrombolysed (44.5%) compared to ischemic stroke patients in the control group (39.3%) . However, patients in the intervention group experienced higher post-care dependency . The PRACTISE trial authors suggested a need for a more intense intervention with continuous process measures to achieve more substantial improvements in thrombolysis rates . Studies of the Breakthrough approach in other settings have produced mixed results [44, 45], which may suggest the need for additional strategies or a more intensive approach in order to achieve sustained change in complex systems. The INSTINCT cluster randomised controlled trial  of 24 acute-care community hospitals in the US used a multilevel, barrier assessment-interactive educational intervention. The intervention included educational elements, assessment of barriers, audit and feedback. Trial results indicated a modest increase in the proportion of admitted stoke patients receiving thrombolysis between pre- and post-intervention periods (1.25% and 2.79% at intervention hospitals, compared to 1.25% and 2.10% at control hospitals using Intention to Treat analysis) but showed non-significant differences between groups. The authors concluded that additional strategies are needed in order to increase acute stroke treatment.
Therefore, a need remains for a rigorous trial of usual approaches versus a comprehensive and intensive strategy for increasing thrombolysis rates in a manner suitable for large scale implementation. The Thrombolysis ImPlementation in Stroke (TIPS) study aims to provide a robust test of a theory and evidence-driven multi-component and multidisciplinary collaborative approach designed to achieve sustained change in complex health systems over multiple sites.
Increase the proportion of all stroke patients receiving thrombolysis at intervention hospitals, compared with control hospitals at follow-up (Primary Outcome);
Maintain best-practice benchmarks of 30% of patients achieving 3-month post-stroke modified Rankin scores of 0 to 1 (little or no disability) at both intervention and control sites (Secondary Outcome);
Ensure that the adverse event rate for major intracranial haemorrhage (parenchymal haematoma) in thrombolysed patients does not rise above best-practice benchmarks of 6%  in either intervention or control hospitals (Secondary Outcome).
Methods and design
Eligible hospitals are those with a Stroke Care Unit or staffing equivalent of a stroke physician and stroke nurse; an Emergency Department and where the hospital is at early stages of thrombolysis implementation. All participating hospitals are required to record every consecutive case of stroke and thrombolysis, including adverse events and patient functional outcomes at three months. Both public and private hospitals are eligible to be included in the sample, as are both teaching and non-teaching hospitals.
Hospitals will be stratified according to their stroke thrombolysis rates at baseline: very low (0% to ≤4.00%), low (>4.00% to ≤10.00%) or moderate (>10.00%). Baseline thrombolysis rate was considered important for the following reasons: A ‘very low’ thrombolysis rate suggests some impediments to thrombolysis, potentially limiting the ability of the site to respond quickly to the intervention. Sites categorised as having a ‘low’ thrombolysis rate may have the greatest potential for achieving change during the study, due to having substantial room for improvement. Conversely, a site with a ‘moderate’ baseline rate of thrombolysis may have limited opportunity to demonstrate further substantial improvement on the outcome measure.
Hospitals will be randomised at one time (effectively achieving allocation concealment) to either the intervention arm or the control arm with a 1:1 allocation ratio, using a computer generated stratified randomisation scheme in StatsDirect. Intervention hospital staff will not be effectively blinded to their allocation, in that they will be aware of their own involvement in intervention activities. There will be minimal interaction with control hospitals during the intervention phase.
Eligible hospitals were identified from National Stroke Foundation (NSF) audit records and state-based clinical SCU networks. Clinical leaders in stroke at each potentially eligible hospital have been contacted by a clinical leader from the research team to verify eligibility and have been invited in person or by telephone to provide in-principle support to participate in the study following discussions with relevant decision makers at each site. A signed memorandum of understanding or equivalent consent agreement between each hospital and the research team has documented agreement by all relevant parties to collaborate for the whole period of the study. Ethical approval for the study has been obtained from relevant human research ethics committees in each state, from each participating hospital and from The University of Newcastle Human Research Ethics Committee.
Information on each patient thrombolysed during the study period will be entered into the secure TIPS database hosted on the National Stroke Foundation (NSF) website. Stroke imaging will be entered via an additional INternational Stroke Perfusion Imaging REgistry (INSPIRE) website designed for imaging data that will be linked to the TIPS database. The TIPS database is only accessible via secure logins and will be maintained by the NSF. The patient data will be entered by the stroke care nurse at each participating hospital as de-identified patient unit record data. Data quality including range checks will be undertaken. The research team will not have access to identifying information about individual clinicians, hospital staff, or data identifying individual patients other than in the normal course of their existing clinical duties. Electronic data files will be securely protected so that only the principal researchers will have access to these data. Staff members from participating departments at each hospital will be asked to complete an anonymous baseline voluntary survey regarding perceived barriers and facilitators to thrombolysis implementation at their hospitals. This survey will be repeated at follow-up.
Numerator: The numerator is the number of cases treated with thrombolytic therapy in each hospital. The following details will be recorded for each thrombolysed case: age, gender, date and time of stroke onset, date and time of arrival at treating hospital, date and time of brain imaging examination, time of tPA treatment, mRS grade before stroke (premorbid mRS), stroke severity (NIHSS Scale), hypertension, diabetes, hyperlipidaemia, smoking status (current or previous), previous diagnosis of stroke, atrial fibrillation (including paroxysmal), congestive heart failure, aspirin at stroke onset, dipyridamole at stroke onset, clopidogrel at stroke onset, anticoagulants (oral) at stroke onset, glucose level before treatment, and mRS at three months post-treatment. Patient imaging data will be uploaded via the INSPIRE database and will include: non-contrast CT (NCCT), perfusion CT (CTP), CT angiography (CTA), and Magnetic Resonance Imaging (MRI).
Denominator: The denominator is the number of cases with a primary discharge diagnosis code of stroke (ICD-10-AM I60.9, I61, I63, I64) that will be extracted from hospital records. The accuracy of ICD coding for all types of stroke is good [48–51], with a NSW tertiary hospital reporting 96% accuracy in stroke discharge diagnosis identified by a random selection of stroke medical records [50, 51]. Another Australian study reported high levels of reliability and adherence to coding standard for ICD-10 codes , with a study of stroke attack rates and case fatality in the Hunter Region reporting 97.5% accuracy between stroke coding and audit . Validation of Australian data sets for stroke diagnosis found 93% coding accuracy levels , and a Canadian study reported 92% coding accuracy using ICD-10 codes for stroke .
The primary outcome is the stroke thrombolysis rate for each hospital, calculated as the number of stroke cases thrombolysed divided by the total number of individuals with a stroke primary discharge diagnosis.
The secondary outcomes include intracranial haemorrhage as defined by the SITS registry and functional outcomes at three months post stroke using the modified Rankin Scale (mRS) . Intracranial haemorrhage events (parenchymal haematoma) will be identified from the TIPS database and will be monitored annually during the intervention and follow-up phases by the research team to ensure that it does not exceed international benchmarks . Sites will be provided with additional support if benchmarks are exceeded. Monitoring of secondary outcomes will not interfere with established governance processes for reporting and monitoring of adverse events in the health system. Functional outcomes will be scored as good (mRS = 0 to 1) or poor (mRS = 2 to 6) by the treating clinician or stroke nurse. The mRS has demonstrated reliability and validity for use by clinicians and stroke nurses . The percentage of tPA-treated stroke cases having a good outcome (mRS <2) at three months post stroke will also be monitored annually by clinical experts in the research team through the post-intervention NSF data collection.
Pen and paper staff surveys will be conducted at all participating hospitals regarding perceived barriers and facilitators to tPA implementation. Survey items include: perceptions of tPA efficacy, local policies about tPA, knowledge of tPA, skills relating to assessment for and administration of tPA, challenges in tPA implementation, training relating to tPA, expertise and confidence in relation to tPA, contingencies relating to tPA, participant’s gender, age, experience, staff role, and estimated rates of tPA administration. For intervention hospitals, engagement with each intervention activity will be assessed, including number and type of staff from each site attending each workshop or meeting. Assessment of baseline overall readiness for change will also be assessed by a member of the research team using a readiness for change checklist based on the work of Warrick et al. (2009), Kasurinen et al. (2002), and the National Institute for Health and Clinical Excellence (2007). The full version of the process measure tools can be obtained from the corresponding author.
Intervention components and activities
Component 1. Situational analysis – clarifying the patient journey
Careful delineation of the phases of care was carried out by the research team on the basis of earlier work at a single site , beginning at the point of seeking emergency medical care (hospital or paramedic). The situational analysis follows the journey of a stroke patient including pre-hospital assessment, triage, clinical assessment, imaging, final clinical assessment, preparation and delivery of thrombolysis. The tasks and staff roles involved along the patient journey are described in Table 1. Examination of how well each of the steps and roles described in the situational analysis will be implemented at a particular site is embedded in a number of intervention activities, including a pre-workshop meeting between the primary change agent and the champions at each site, collaborative workshops involving a team from each site, and site-based working group meetings. In the context of each of these intervention activities, the situational analysis facilitates identification of barriers and facilitators to increasing the number of patients who are appropriately thrombolysed.
Component 2. Change agents - educating, persuading and modelling
Much of the work on organisational change emphasises the importance of champions or leaders in order to increase the rate of adoption of a desired change [58, 59]. In the context of TIPS, change agents will interact with participants to educate, persuade and model desired behaviours throughout the change process. TIPS change agents include primary change agents who liaise with each of the intervention sites and change agents or ‘champions’ located at each site. Stroke nurses from the research team will act as stroke nurse ‘champions’ to monitor and encourage completion of the nurse professional development training (see Component 5). Members of the research team (CL, MP, CB) have been identified as the primary change agents, selected on the basis of meeting criteria for leadership and ability to enact education, persuasion and modelling at a national level . The activities of the primary change agents involve identification of Site Champions and assisting the Site Champions with obtaining endorsement from senior hospital administrators or executives. During the intervention phase, the primary change agent will make regular contact (by email or telephone) to discuss any questions or concerns relating to recent cases of thrombolysis. The primary change agents will also lead education and training aspects of the communal workshops, and attend site workshops. The Site Champions will be the drivers of change at each site, engage with those involved in clinical work flow decisions, and will form a working group comprising key local staff from para-medicine, emergency, stroke care, and imaging. The Site Champions will lead the site working groups in setting goals, identifying actions, and reviewing performance designed to achieve change. The Site Champions are selected on the basis of clinical role (generally lead neurologist and lead stroke nurse).
Component 3. Information-based target setting – persuasion and incentivisation
Target setting has been established as an important motivator and driver of change on an individual and system level . Information-based target setting involves a process of setting overall targets for appropriate and achievable rates of thrombolysis for each site. Importantly, a series of interim targets or successive approximations will be set during the course of the intervention period.
During the pre-workshop meeting, the Site Champions at each intervention hospital will set a preliminary target for increased tPA administration at that hospital with the guidance of the primary change agent. This target will be reviewed at the first communal workshop in collaboration with members of the site-based working group. Targets are framed based on baseline thrombolysis rates and should be challenging but achievable to create a collective expectation of change and experience of achievement. Interim targets will also be set, based on perceived achievable changes in performance in successive three-month periods throughout the intervention phase.
Component 4. Collaborative problem solving - education, modelling and enablement
Collaborative problem solving is central to commonly-used quality improvement strategies [61–63] and aims to harness the knowledge, skills and support of a group. Collaborative problem solving will occur within site working groups during their bi-weekly meetings. Bi-monthly teleconferences between the primary change agent and representatives from each of the intervention site-based working groups will explore experiences of all groups in relation to identifying barriers and implementing solutions. An evidence-based process will be used whereby groups attempt to analyse the problem before attempting to search for a viable solution .
Component 5. Professional development – education, training and restriction
Detailed education and training regarding clinical decision making for thrombolysis will be provided via web-based educational modules with a strong emphasis on achieving accepted levels of competency in key skills such as decision making. Web-based approaches provide highly accessible, flexible and effective approaches to learning, particularly when an interactive case-based approach is used . The TIPS website is available to intervention sites via a unique log-in for each participant.
For clinicians, the website contains didactic training modules with a test at the end of each module and a series of case studies where clinicians test their clinical judgement against a consensus-based logic frame for a set of stroke cases. In each case, the clinician is provided with feedback regarding the appropriateness of the decision and links to further reading if required. Clinicians are asked to continue with the case studies until they have achieved an agreed level of competence in decision making. Lead clinicians are asked to ensure that all relevant staff including emergency physicians and registrars complete the training prior to caring for potential stroke patients. The website provides information and training for paramedics in the identification of potential stroke patients. Both the clinician and paramedic training modules were developed by members of the TIPS research team and colleagues.
NET SMART-Junior, a distance-accessible competency-based learning system covering evidence-based management of acute stroke patients, will serve as the educational platform for standardised nursing education in the TIPS trial. The NET SMART system offers rolling entry/completion times supported by self-learning paced programming. NET SMART-Junior is accredited by the Arizona State University, College of Nursing and Healthcare Innovation (Tempe, Arizona, USA) to deliver continuing education credits internationally for completion of modular units consisting of pre-test/post-test and clinical skills measures. TIPS participants will be asked to complete a selected number of training modules and to discuss their results with their nurse ‘champion’.
The TIPS website also contains scientific literature relating to thrombolysis, and tools and resources such as those from the PAST trial . In addition to training in decision making skills, training in the skills of being a champion will be provided. Change champion training will also occur as a component of the communal workshops in the form of group-based reflective activities, followed by self-directed tasks and opportunities for engaging with ongoing external training or self-directed learning.
Component 6. Performance feedback - persuasion, modelling
Given the strategic importance of providing performance feedback in order to achieve clinical practice change [65, 66], local and comparative feedback will be provided. Sites will be provided with their three-monthly estimated proportion of ischaemic stroke cases who receive thrombolysis, graphed against site targets. Comparative data will be provided, showing each site how it compares to other intervention hospitals in a de-identified format (i.e., other hospital names not disclosed), to create a positive level of competition among peers. This information will be made available to the participants via their unique logins on the TIPS website and also forwarded directly to Site Champions.
Achieving sustainability: Throughout the intervention period, there will be a focus on identifying and enacting strategies to embed changes in practice and ensure ongoing practice is monitored via mandatory recording and reporting of thrombolysis delivery. During every collaborative intervention activity (e.g., workshops, site-based meetings, inter-site teleconferences), sustainability will be discussed in order that a strong focus on sustainability is maintained. Strategies for achieving sustainability will include: i) key steps in the process of care (e.g., time from arrival to handover to stroke team and coverage of imaging services) as part of local standard recording and internal reporting; ii) establishing ongoing communities of practice/clinical networks or including thrombolysis as one of the priority initiatives of existing clinical networks; iii) establishing networks for ongoing mentoring, including formal mentoring relationships that link groups of staff across hospitals; iv) including numbers and proportions of thrombolysed patients in standard reporting to hospital executives and local health districts; v) incorporating the requirement for completion of the TIPS web-based training modules as part of orientation for medical staff in all relevant departments; and vi) establishment of the web-based training as a locally-recognised professional credential required for those involved in the care of stroke patients in the local health district. Site-based working groups will also be encouraged to identify and enact other locally-relevant strategies for achieving sustainability.
Measuring sustainability: Progress in achieving the above sustainability strategies will be monitored during both the intervention and follow-up phases via bi-monthly site teleconferences and post-test staff surveys as part of the study process measures. The degree to which each of the intervention activities (regular site working group meetings, completion of training modules by new staff, entry and monitoring of the number of thrombolysis cases) continues to occur during the follow-up period (Year 4) will also be monitored as part of the study process measures.
From baseline data, it is estimated that participating hospitals will have an average of approximately 150 stroke patients per year, that 5% of stroke patients in the control group will be prescribed tPA, and that the average co-efficient of variation across strata will be approximately 0.4. With 10 hospitals per group, the study will have 80% power with a 5% significance level to detect an absolute difference of 7% to 10% . The subgroup analysis of hospitals with baseline tPA rate of <10% will also have similar power to detect this difference, as, although the number of hospitals per group will be smaller, the baseline tPA rate will also be lower.
Primary outcome: Analysis will involve cluster-level summary data. The difference in post-intervention thrombolysis rates between intervention and control groups will be compared via a two-stage approach to adjust for baseline thrombolysis rates, using a stratified t-test.
Secondary outcomes: The proportion of thrombolysed cases with good outcomes (mRS scores of 0 to 1) and the proportion with intracranial haemorrhage at each hospital will be compared to benchmarks (using a one tailed hypothesis test) and analogous methods to stopping rules for randomised clinical trials. Thresholds beyond which intervention is required will be calculated using one-sided hypothesis tests adjusted for annual checks. There will be intervention thresholds, beyond which quality assurance processes would be instituted, and stopping thresholds, beyond which tPA administration would be suspended until processes are reviewed. Because we are conducting three analyses, we have used the method of O’Brien-Fleming to determine significance levels for each of the three analyses (this method provides more stringent stopping criteria for early analysis). The appropriate significance levels are 0.0006, 0.0151 and 0.0471 for analysis 1, 2 and 3 respectively. Analysis will be intention to treat in that all individuals administered tPA should be included in the study database, and the denominator will be obtained directly from computerised hospital admissions data.
Secondary analyses: Given that there may be limited potential for hospitals with high baseline tPA rates to substantially further increase tPA administration, we will undertake a subgroup analysis of the primary outcome excluding hospitals with a baseline tPA of more than 10%.
The trial has been registered with the Australian New Zealand Clinical Trials Registry: ACTRN12613000939796 and has obtained a UTN number: U1111-1145-6762.
TIPS is one of the few studies to rigorously trial a comprehensive, multi-component and multidisciplinary collaborative approach to improving thrombolysis rates at multiple sites, and builds substantially on the promising work of the PRACTICE trial  and the INSTINCT trial , which both emphasised the need for a more intensive approach, such as that proposed for TIPS. The study findings will occur in the context of a range of challenges and limitations as described below. TIPS has the potential to identify methods by which optimal care can be effectively implemented on a large scale for stroke patients during the acute phase. Trial outcomes will be published whether significant or not.
Key study limitations and challenges
Measurement of primary outcome – tPA rates
The measurements of both the numerator and denominator for the primary outcome are challenging. The study numerator is entered by the participating site, rather than by an independent, blinded auditor, therefore introducing potential bias. The study denominator – medical coding of all stroke cases – is independent and blinded in that medical coders at each hospital are unaware of the project. The denominator is also an approximation in that it includes all stroke cases rather than only ischaemic stroke cases.
Group allocation not able to be blinded
A hospital’s random allocation to intervention or control group cannot be concealed due to the nature of the intervention. Therefore, additional potential bias is introduced.
Engagement of paramedic/ambulance service
The Ambulance Service has been approached to participate in this project. They will provide the ‘in-field’ context for ischaemic stroke identification and management. It is unlikely but possible that ambulance officers participating in the project will cover both control and intervention hospitals and that any changes in their protocols or procedures resulting from this project may have an impact on both control and intervention sites. This cannot be controlled for in the study design.
Other factors that may influence the results of the study, but cannot be controlled for, include policies, guidelines, or any health reforms being rolled out during the intervention period. There may also be new information regarding the tPA drug or other treatments of ischaemic stroke becoming available during the lifespan of this project, thus potentially affecting the results of the study. However, any benefit from these initiatives reasonably can be expected to impact equally on control sites as on intervention sites. Any adverse events associated with this study may also impact the outcomes of the study, especially if they garner media attention. These external factors will be monitored and included in the discussion of study findings.
Notwithstanding these limitations, TIPS has significant rigour given the large cluster RCT design, stratification on baseline tPA rates, and objective outcome measurement.
Changes to this protocol
The TIPS Working Group will report any changes to this Protocol to the TIPS Steering Committee and relevant Human Research Ethics Committees. Changes will also be reported in published papers emerging from this trial.
TIPS has the potential to make a major contribution to the implementation of translational literature, given the combination of a theoretically driven, comprehensive intervention strategy and rigorous outcome measurement.
Thrombolysis ImPlementation in Stroke
Tissue plasminogen activator (also referred to as PLAT)
National Stroke Foundation (Australia)
INternational Stroke Perfusion Imaging REgistry
Safe Implementation of Thrombolysis in Stroke registry (Australia)
Stroke care unit
National Institutes of Health Stroke Scale
National Health and Medical Research Council
Modified rankin score
The Model Of Resource Utilization, Costs, and Outcomes for Stroke
Pre-hospital Acute Stroke Triage
Non-contrast Computed Tomography
Perfusion Computed Tomography
Computed Tomography Angiography
Magnetic resonance imaging
This project is funded by the National Health and Medical Research Council (NHMRC) as a partnership grant (ID569328) and is part-funded by a TRIP fellowship, with collaborative funding from Boehringer Ingelheim and in-kind support from ACI Stroke Care Network/Stroke Services NSW, Victorian Stroke Clinical Network, National Stroke Foundation (NSF) and NSW Cardiovascular Research Network-National Heart Foundation, with infrastructure funding from Hunter Medical Research Institute (HMRI) and The University of Newcastle (UoN). The UoN is the coordinating centre, the TRIP fellowship funds the research activities for the Clinical Champion, Boehringer Ingelheim funds workshops, logistics and educational material, ACI Stroke Clinical Network/Stroke Services NSW and Victorian Stroke Clinical Network provide support for a stroke care coordinator’s time collating and entering tPA audit results, and NSW Cardiovascular Research Network-National Heart Foundation provides support for the involvement of a clinical practice change expert to support quality improvement strategies at each site. The NHMRC partnership grant is covering the cost of the remaining project activities. Although the Industry Partner is a manufacturer of the tPA drug and therefore has a vested interest in increasing tPA rates, their involvement is subject to strict guidelines, including non-involvement in the study design, data collection, analysis and publications. The assistance of Kate Day and Kate Ross in progressing toward launching the study is gratefully acknowledged.
Roles and responsibilities
The TIPS Steering Committee is comprised of Chief Investigators and external advisors. They are responsible for overseeing the design and milestones of the project. The TIPS Working Group is a smaller group of Chief Investigators and people driving the project, including Research Assistants. They are responsible for data monitoring and report to the TIPS Steering Committee. Boehringer Ingelheim has no representation on either of these committees to ensure that study findings are independent of the sponsor and their competing interest.
- Mackay J, Mensah G: The Atlas of Heart Disease and Stroke. 2004, Geneva: WHOGoogle Scholar
- Compressed Mortality File 1999-2009: Underlying cause-of-death 1999–2009. CDC WONDER Online Database, compiled for Compressed Mortality File 1999–2009 Series 20, No. 20. 2012, Accessed August 20, 2012 [http://wonder.cdc.gov/mortSQl.html]Google Scholar
- Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Borden WB, Bravata DM, Dai S, Ford ES, Fox CS: Heart disease and stroke statistics—2013 update a report from the American heart association. Circulation. 2013, 127 (1): e6-e245. 10.1161/CIR.0b013e31828124ad.View ArticlePubMedGoogle Scholar
- Heidenreich PA, Trogdon JG, Khavjou OA, Butler J, Dracup K, Ezekowitz MD, Finkelstein EA, Hong Y, Johnston SC, Khera A: Forecasting the future of cardiovascular disease in the United States a policy statement from the American heart association. Circulation. 2011, 123 (8): 933-944. 10.1161/CIR.0b013e31820a55f5.View ArticlePubMedGoogle Scholar
- Household component summary data table. 2012,http://meps.ahrq.gov/mepsweb/,
- Australian Bureau of Statistics: Cardiovascular Disease in Australia: a snapshot, 2004-05. 2006, Canberra: ABSGoogle Scholar
- Australian Institute of Health and Welfare: Australia's health 2006. The Economic Impact of Stroke in Australia. 2006, Canberra: AIHW, Melbourne: NSF; 2013Google Scholar
- Deloitte Access Economics: The economic impact of stroke in Australia. 2013, Melbourne: NSFGoogle Scholar
- Australian Institute of Health and Welfare: Stroke and its management in Australia: an update. Cardiovascular Disease Series. 2013, Canberra: AIHWGoogle Scholar
- National Stroke Foudation: National Stroke Audit – Acute Services Clinical Audit Report 2011. 2011, Melbourne: NSFGoogle Scholar
- Hankey GJ, Jamrozik K, Broadhurst RJ, Forbes S, Anderson CS: Long-term disability after first-ever stroke and related prognostic factors in the Perth community stroke study, 1989–1990. Stroke. 2002, 33 (4): 1034-1040. 10.1161/01.STR.0000012515.66889.24.View ArticlePubMedGoogle Scholar
- International Stroke Trial Collaborative Group: The International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19,435 patients with acute ischaemic stroke. Lancet. 1997, 349 (9065): 1569-1581.View ArticleGoogle Scholar
- Gilligan A, Thrift A, Sturm J, Dewey H, Macdonell R, Donnan G: Stroke units, tissue plasminogen activator, aspirin and neuroprotection: which stroke intervention could provide the greatest community benefit?. Cerebrovasc Dis. 2005, 20 (4): 239-244. 10.1159/000087705.View ArticlePubMedGoogle Scholar
- Saver J, Fonarow G, Smith E, Reeves M, Grau-Sepulveda M, Pan W, Olson D, Hernandez A, Peterson E, Schwamm L: Time to treatment with intravenous tissue plasminogen activator and outcome from acute ischemic stroke. JAMA. 2013, 309 (23): 2480-2488. 10.1001/jama.2013.6959.View ArticlePubMedGoogle Scholar
- Hacke W, Kaste M, Bluhmki E, Brozman M, Dávalos A, Guidetti D, Larrue V, Lees KR, Medeghri Z, Machnig T: Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med. 2008, 359 (13): 1317-1329. 10.1056/NEJMoa0804656.View ArticlePubMedGoogle Scholar
- Wardlaw JM, Murray V, Berge E, del Zoppo G, Sandercock P, Lindley RL, Cohen G: Recombinant tissue plasminogen activator for acute ischaemic stroke: an updated systematic review and meta-analysis. Lancet. 2012, 379 (9834): 2364-2372. 10.1016/S0140-6736(12)60738-7.View ArticlePubMedPubMed CentralGoogle Scholar
- IST-3 Collaborative Group: Effect of thrombolysis with alteplase within 6 h of acute ischaemic stroke on long-term outcomes (the third International Stroke Trial [IST-3]): 18- month follow-up of a randomised controlled trial. Lancet Neurol. 2013, 12 (8): 768-776.View ArticleGoogle Scholar
- Hacke W, Donnan G, Fieschi C, Kaste M, Von Kummer R, Broderick J, Brott T, Frankel M, Grotta J, Haley E: Association of outcome with early stroke treatment: pooled analysis of ATLANTIS, ECASS, and NINDS rt-PA stroke trials. Lancet. 2004, 363 (9411): 768-774.View ArticlePubMedGoogle Scholar
- Wahlgren N, Ahmed N, Dávalos A, Hacke W, Millán M, Muir K, Roine R, Toni D, Lees K, Investigators S: Thrombolysis with alteplase 3-4.5 h after acute ischaemic stroke (SITS-ISTR): an observational study. Lancet. 2008, 372 (9646): 1303-1309. 10.1016/S0140-6736(08)61339-2.View ArticlePubMedGoogle Scholar
- Saver J, Gornbein J, Grotta J, Liebeskind D, Lutsep H, Schwamm L, Scott P, Starkman S: Number needed to treat to benefit and to harm for intravenous tissue plasminogen activator therapy in the 3- to 4.5-hour window: joint outcome table analysis of the ECASS 3 trial. Stroke. 2009, 40 (7): 2433-2437. 10.1161/STROKEAHA.108.543561.View ArticlePubMedPubMed CentralGoogle Scholar
- del Zoppo G, Saver J, Jauch E, Adams H, American Heart Association Stroke Council: Expansion of the time window for treatment of acute ischemic stroke with intravenous tissue plasminogen activator: a science advisory from the American heart Association/American Stroke Association. Stroke. 2009, 40 (8): 2945-2948. 10.1161/STROKEAHA.109.192535.View ArticlePubMedPubMed CentralGoogle Scholar
- Buchan A, Barber P, Newcommon N, Karbalai H, Demchuk A, Hoyte K, Klein G, Feasby T: Effectiveness of t-PA in acute ischemic stroke: outcome relates to appropriateness. Neurology. 2000, 54 (3): 679-684. 10.1212/WNL.54.3.679.View ArticlePubMedGoogle Scholar
- Saver J: Number needed to treat estimates incorporating effects over the entire range of clinical outcomes: novel derivation method and application to thrombolytic therapy for acute stroke. Arch Neurol. 2004, 61 (7): 1066-1070.View ArticlePubMedGoogle Scholar
- Wardlaw J, Warlow C, Counsell C: Systematic review of evidence on thrombolytic therapy for acute ischaemic stroke. Lancet. 1997, 350 (9078): 607-614. 10.1016/S0140-6736(97)03022-5.View ArticlePubMedGoogle Scholar
- Hoffman JR, Cooper RJ: How is more negative evidence being used to support claims of benefit: the curious case of the third international stroke trial (IST-3). Emerg Med Australas. 2012, 24 (5): 473-476. 10.1111/j.1742-6723.2012.01604.x.View ArticlePubMedGoogle Scholar
- Canadian Stroke Best Practices and Standards Working Group Canadian Best Practice Recommendations for Stroke Care: Canadian best practice recommendations for stroke care. Edited by: Lindsay M, Gubitz G, Bayley M, Phillips S. 2013, Ottawa: Canadian Stroke Network, Fourth
- Party ISW: National Clinical Guideline for Stroke. 2012, London: Royal College of Physicians, 4Google Scholar
- National Stroke Foundation: Clinical guidelines for stroke management. 2010, Melbourne: NSFGoogle Scholar
- Jauch EC, Saver JL, Adams HP, Bruno A, Demaerschalk BM, Khatri P, McMullan PW, Qureshi AI, Rosenfield K, Scott PA: Guidelines for the early management of patients with acute ischemic stroke a guideline for healthcare professionals from the American heart association/American stroke association. Stroke. 2013, 44 (3): 870-947. 10.1161/STR.0b013e318284056a.View ArticlePubMedGoogle Scholar
- Samsa G, Reutter R, Parmigiani G, Ancukiewicz M, Abrahamse P, Lipscomb J, Matchar D: Performing cost-effectiveness analysis by integrating randomized trial data with a comprehensive decision model: application to treatment of acute ischemic stroke. J Clin Epidemiol. 1999, 52 (3): 259-271. 10.1016/S0895-4356(98)00151-6.View ArticlePubMedGoogle Scholar
- Fagan S, Morgenstern L, Petitta A, Ward R, Tilley B, Marler J, Levine S, Broderick J, Kwiatkowski T, Frankel M, Brott T, Walker M: Cost-effectiveness of tissue plasminogen activator for acute ischemic stroke. Neurology. 1998, 50 (4): 883-890. 10.1212/WNL.50.4.883.View ArticlePubMedGoogle Scholar
- Moodie M: Trial application of a Model of Resource Utilization, Costs, and Outcomes for Stroke (MORUCOS) to assist priority setting in stroke. Stroke. 2004, 35 (5): 1041-1046. 10.1161/01.STR.0000125012.36134.89.View ArticlePubMedGoogle Scholar
- Sandercock P, Berge E, Dennis M, Forbes J, Hand P, Kwan J, Lewis S, Lindley R, Neilson A, Wardlaw J: Cost-effectiveness of thrombolysis with recombinant tissue plasminogen activator for acute ischemic stroke assessed by a model based on UK NHS costs. Stroke. 2004, 35 (6): 1490-1497. 10.1161/01.STR.0000126871.98801.6E.View ArticlePubMedGoogle Scholar
- Nguyen-Huynh MN, Johnston SC: Is hospitalization after TIA cost-effective on the basis of treatment with tPA?. Neurology. 2005, 65 (11): 1799-1801. 10.1212/01.wnl.0000187067.93321.fa.View ArticlePubMedGoogle Scholar
- National Health Priority Action Council (NHPAC): National service improvement framework for heart, stroke and vascular disease. 2006, Canberra: Australian Govenement Department of Health and AgeingGoogle Scholar
- Adams HP, Brott TG, Furlan AJ, Gomez CR, Grotta J, Helgason CM, Kwiatkowski T, Lyden PD, Marler JR, Torner J: Guidelines for thrombolytic therapy for acute stroke: a supplement to the guidelines for the management of patients with acute ischemic stroke a statement for healthcare professionals from a special writing group of the stroke council, American heart association. Circulation. 1996, 94 (5): 1167-1174. 10.1161/01.CIR.94.5.1167.View ArticlePubMedGoogle Scholar
- Fang M, Cutler D, Rosen A: Trends in thrombolytic use for ischemic stroke in the United States. J Hosp Med. 2010, 5 (7): 406-409. 10.1002/jhm.689.View ArticlePubMedPubMed CentralGoogle Scholar
- Rudd A, Hoffman A, Grant R, Campbell J, Lowe D, Intercollegiate Working Party for Stroke: Stroke thrombolysis in England, Wales and Northern Ireland: how much do we do and how much do we need?. J Neurol Neurosurg Psychiatry. 2011, 82 (1): 14-19. 10.1136/jnnp.2009.203174.View ArticlePubMedGoogle Scholar
- Improving patient care: the implementation of change in clinical practice. Edited by: Grol R, Wensing M, Eccles M, Davis D. 2005, London: Elsevier: Butterworth-Heinemann
- Quain DA, Parsons MW, Loudfoot AR, Spratt NJ, Evans MK, Russell ML, Royan AT, Moore AG, Miteff F, Hullick C: Improving access to acute stroke therapies: a controlled trial of organised pre-hospital and emergency care. Med J Aust. 2008, 189 (8): 429-433.PubMedGoogle Scholar
- Cadilhac D, Ibrahim J, Pearce D, Ogden K, McNeill J, Davis S, Donnan G, SCOPES Study Group: Multicenter comparison of processes of care between stroke units and conventional care wards in Australia. Stroke. 2004, 35 (5): 1035-1040. 10.1161/01.STR.0000125709.17337.5d.View ArticlePubMedGoogle Scholar
- Schwamm L, Fonarow G, Reeves M, Pan W, Frankel M, Smith E, Ellrodt G, Cannon C, Liang L, Peterson E, LaBresh K: Get with the guidelines – stroke is associated with sustained improvement in care for patients hospitalized with acute stroke or transient ischemic attack circulation. Circulation. 2009, 119: 107-115. 10.1161/CIRCULATIONAHA.108.783688.View ArticlePubMedGoogle Scholar
- Dirks M, Niessen L, van Wijngaarden J, Koudstaal P, Franke C, van Oostenbrugge R, Huijsman R, Lingsma H, Minkman M, Dippel D, PRomoting ACute Thrombolysis in Ischemic StrokE (PRACTISE) Investigators: Promoting thrombolysis in acute ischemic stroke. Stroke. 2011, 42 (5): 1325-1330. 10.1161/STROKEAHA.110.596940.View ArticlePubMedGoogle Scholar
- Schouten LM, Hulscher ME, Everdingen JJ, Huijsman R, Grol RP: Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008, 336 (7659): 1491-1494. 10.1136/bmj.39570.749884.BE.View ArticlePubMedPubMed CentralGoogle Scholar
- Jones K, Piterman L: The effectiveness of the breakthrough series methodology. Aust J Prim Health. 2008, 14 (1): 59-65. 10.1071/PY08008.View ArticleGoogle Scholar
- Scott PA, Meurer WJ, Frederiksen SM, Kalbfleisch JD, Xu Z, Haan MN, Silbergleit R, Morgenstern LB: A multilevel intervention to increase community hospital use of alteplase for acute stroke (INSTINCT): a cluster-randomised controlled trial. Lancet Neurol. 2013, 12 (2): 139-148. 10.1016/S1474-4422(12)70311-3.View ArticlePubMedGoogle Scholar
- Michie S, van Stralen MM, West R: The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011, 6 (1): 42-10.1186/1748-5908-6-42.View ArticlePubMedPubMed CentralGoogle Scholar
- Ghia D, Thomas P, Cordato D, Worthington J, Cappelen-Smith C, Griffith N, Hanna I, Hodgkinson S, McDougall A, Beran R: Validation of emergency and final diagnosis coding in transient ischemic attack: South Western Sydney transient ischemic attack study. Neuroepidemiology. 2010, 25 (1): 53-58.View ArticleGoogle Scholar
- Gattellari M, Worthington J, Jalaludin B, Mohsin M: Stroke unit care in a real-life setting Can results from randomized controlled trials be translated into every-day clinical practice? An observational study of hospital data in a large Australian population. Stroke. 2009, 40 (1): 10-17. 10.1161/STROKEAHA.108.523548.View ArticlePubMedGoogle Scholar
- Wang Y, Levi C, Attia J, D'Este C, Spratt N, Fisher J: Seasonal variation in stroke in the Hunter region, Australia: a 5-year hospital-based study, 1995-2000. Stroke. 2003, 34 (5): 1144-1150. 10.1161/01.STR.0000067703.71251.B6.View ArticlePubMedGoogle Scholar
- Wang Y, Levi C, D’Este C, Attia J, Fisher J: Variation of stroke attack rates in rural, urban, and coalfields areas of the Hunter region, Australia, 1995-2000. J Stroke Cerebrovasc Dis. 2003, 12 (2): 103-110. 10.1053/jscd.2003.12.View ArticlePubMedGoogle Scholar
- Henderson T, Shepheard J, Sundararajan V: Quality of diagnosis and procedure coding in ICD-10 administrative data. Med Care. 2006, 44 (11): 1011-1019. 10.1097/01.mlr.0000228018.48783.34.View ArticlePubMedGoogle Scholar
- Marsden D, Spratt N, Walker R, Barker D, Attia J, Pollack M, Parsons M, Levi C: Trends in stroke attack rates and case fatality in the Hunter region Australia 1996–2008. Cerebrovasc Dis. 2010, 30 (5): 500-507. 10.1159/000319022.View ArticlePubMedGoogle Scholar
- Kokotailo R, Hill M: Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke. 2005, 36 (8): 1776-1781. 10.1161/01.STR.0000174293.17959.a1.View ArticlePubMedGoogle Scholar
- Rankin L: Cerebral vascular accidents in patients over the age of 60. II. Prognosis. Scott Med J. 1957, 2 (5): 200-215.PubMedGoogle Scholar
- Wahlgren N, Ahmed N, Dávalos A, Ford G, Grond M, Hacke W, Hennerici M, Kaste M, Kuelkens S, Larrue V, Lees K, Roine R, Soinne L, Toni D, Vanhooren G, SITS-MOST Investigators: Thrombolysis with alteplase for acute ischaemic stroke in the Safe Implementation of Thrombolysis in Stroke-Monitoring Study (SITS-MOST): an observational study. Lancet. 2007, 369 (9558): 275-282. 10.1016/S0140-6736(07)60149-4.View ArticlePubMedGoogle Scholar
- Banks J, Marotta C: Outcomes validity and reliability of the modified Rankin scale implications for stroke clinical trials: a literature review and synthesis. Stroke. 2007, 38 (3): 1091-1096. 10.1161/01.STR.0000258355.23810.c6.View ArticlePubMedGoogle Scholar
- Valente T, Davis R: Accelerating the diffusion of innovations using opinion leaders. ANNALS Am Acad Polit Soc Sci. 1999, 566: 55-67. 10.1177/0002716299566001005.View ArticleGoogle Scholar
- Katz E: The two-step flow of communication: an up-to-date report on a hypothesis. Publ Opin Q. 1957, 21: 61-78. 10.1086/266687.View ArticleGoogle Scholar
- Strecher V, Seijts G, Kok G, Latham G, Glasgow R, DeVellis B, Meertens R, Bulger D: Goal setting as a strategy for behaviour change. Health Educ Behav. 1995, 22 (3): 190-View ArticleGoogle Scholar
- Kilo C: A framework for collaborative improvement: lessons from the institute for healthcare improvement's breakthrough series. Qual Manag Health Care. 1998, 6 (4): 1-13. 10.1097/00019514-199806040-00001.View ArticlePubMedGoogle Scholar
- Plsek P: Collaborating across organizational boundaries to improve the quality of care. Am J Infect Control. 1997, 25 (2): 85-95. 10.1016/S0196-6553(97)90033-X.View ArticlePubMedGoogle Scholar
- Hirokawa R: Group communication and problem-solving effectiveness: an investigation of group phases. Hum Commun Res. 1983, 9 (4): 291-305. 10.1111/j.1468-2958.1983.tb00700.x.View ArticleGoogle Scholar
- Casebeer L, Engler S, Bennett N, Irvine M, Sulkes D, DesLauriers M, Zhang S: A controlled trial of the effectiveness of internet continuing medical education. BMC Med. 2008, 6: 37-10.1186/1741-7015-6-37.View ArticlePubMedPubMed CentralGoogle Scholar
- Grimshaw J, Thomas R, MacLennan G, Fraser C, Ramsay C, Vale L, Whitty P, Eccles M, Matowe L, Shirran L: Effectiveness and efficiency of guideline dissemination and implementation strategies. 2004Google Scholar
- Kawamoto K, Houlihan C, Balas E, Lobach D: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330 (7494): 765-10.1136/bmj.38398.500764.8F.View ArticlePubMedPubMed CentralGoogle Scholar
- Hayes R, Moulton L: Cluster Randomised Trials. 2009, London: Chapman and HallView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.