Skip to main content
Download PDF
Download ePub

Evaluation of a targeted, theory-informed implementation intervention designed to increase uptake of emergency management recommendations regarding adult patients with mild traumatic brain injury: results of the NET cluster randomised trial

Implementation Science volume 14, Article number: 4 (2019) Cite this article

Abstract

Background

Evidence-based guidelines for management of mild traumatic brain injury (mTBI) in the emergency department (ED) are now widely available; however, clinical practice remains inconsistent with these guidelines. A targeted, theory-informed implementation intervention (Neurotrauma Evidence Translation (NET) intervention) was designed to increase the uptake of three clinical practice recommendations regarding the management of patients who present to Australian EDs with mild head injuries. The intervention involved local stakeholder meetings, identification and training of nursing and medical local opinion leaders, train-the-trainer workshops and standardised education materials and interactive workshops delivered by the opinion leaders to others within their EDs during a 3 month period. This paper reports on the effects of this intervention.

Methods

EDs (clusters) were allocated to receive either access to a clinical practice guideline (control) or the implementation intervention, using minimisation, a method that allocates clusters to groups using an algorithm to minimise differences in predefined factors between the groups. We measured clinical practice outcomes at the patient level using chart audit. The primary outcome was appropriate screening for post-traumatic amnesia (PTA) using a validated tool until a perfect score was achieved (indicating absence of acute cognitive impairment) before the patient was discharged home. Secondary outcomes included appropriate CT scanning and the provision of written patient information upon discharge. Patient health outcomes (anxiety, primary outcome: Hospital Anxiety and Depression Scale) were also assessed using follow-up telephone interviews. Outcomes were assessed by independent auditors and interviewers, blinded to group allocation.

Results

Fourteen EDs were allocated to the intervention and 17 to the control condition; 1943 patients were included in the chart audit. At 2 months follow-up, patients attending intervention EDs (n = 893) compared with control EDs (n = 1050) were more likely to have been appropriately assessed for PTA (adjusted odds ratio (OR) 20.1, 95%CI 6.8 to 59.3; adjusted absolute risk difference (ARD) 14%, 95%CI 8 to 19). The odds of compliance with recommendations for CT scanning and provision of written patient discharge information were small (OR 1.2, 95%CI 0.8 to 1.6; ARD 3.2, 95%CI − 3.7 to 10 and OR 1.2, 95%CI 0.8 to 1.8; ARD 3.1, 95%CI − 3.0 to 9.3 respectively).

A total of 343 patients at ten interventions and 14 control sites participated in follow-up interviews at 4.3 to 10.7 months post-ED presentation. The intervention had a small effect on anxiety levels (adjusted mean difference − 0.52, 95%CI − 1.34 to 0.30; scale 0–21, with higher scores indicating greater anxiety).

Conclusions

Our intervention was effective in improving the uptake of the PTA recommendation; however, it did not appreciably increase the uptake of the other two practice recommendations. Improved screening for PTA may be clinically important as it leads to appropriate periods of observation prior to safe discharge. The estimated intervention effect on anxiety was of limited clinical significance. We were not able to compare characteristics of EDs who declined trial participation with those of participating sites, which may limit the generalizability of the results.

Trial registration

Australian New Zealand Clinical Trials Registry (ACTRN12612001286831), date registered 12 December 2012.

Peer Review reports

Introduction

Traumatic brain injury, caused by external forces such as sports, falls or accidents, is a frequent presentation to emergency departments (EDs) worldwide [1]. The vast majority (80 to 90% depending on the definition) are classified as ‘mild’ severity. People with mild traumatic brain injury (mTBI) are usually managed in the ED and discharged within hours [2]. The challenge for ED clinicians is to identify which patients presenting with a head injury require further management and which patients can safely be sent home [3]. While the majority of people suffering mTBI will make a full recovery within a few weeks or months, approximately 15–25% will go on to subjectively report post-concussion symptoms such as ongoing headaches, memory and concentration problems, and sleep difficulties [4,5,6]. A small minority (approximately 1%) have underlying intracranial haemorrhage and deteriorate quickly, requiring neurosurgical intervention [7].

Several high quality evidence-based clinical practice guidelines are available to guide the care of patients who present to the ED with mTBI [8]. Three key clinical practice recommendations from these guidelines determined as important in an Australian setting [8, 9] are (1) post-traumatic amnesia (PTA) should be prospectively assessed in ED using a validated tool; (2) guideline-developed criteria or clinical decision rules should be used to determine the appropriate use and timing of computed tomography (CT) imaging; (3) verbal and written patient information consisting of advice, education and reassurance should be provided upon discharge from the ED. Despite the availability of guidelines, research undertaken within Australia and internationally has shown that care is often inconsistent with these recommendations [10,11,12,13,14] (see Additional file 1 (Table 1) for further information on the three key recommendations, their relevance to managing this patient group and the evidence underpinning the recommendations).

This gap between guideline recommendations and actual practice is not unique to mTBI, with similar difference identified in many clinical disciplines. We know that the dissemination of guidelines alone is seldom sufficient to change practice [15, 16] and more active strategies aiming to bring about practice change are needed. These implementation strategies may be more effective if they are underpinned by theories of behaviour change and consider the context and determinants of practice (both barriers to and enhancers of the recommended practice) [17, 18]. Implementation studies incorporating the explicit use of theories in the processes of designing and evaluating targeted interventions [18] has been recommended in emergency settings [19, 20], as relatively few implementation studies have been conducted compared with other settings.

As part of a program of research aiming to improve outcomes for patients with mTBI (the Neurotrauma Evidence Translation (NET) program) [21], we developed an implementation intervention to increase the uptake of the three key clinical practice recommendations. To maximise the likelihood of the intervention’s effectiveness, our intervention was informed by evidence and theories of change [22] and designed to target the identified determinants of practice (e.g. address the barriers and enhance the enablers) [23, 24]. The NET-Trial [25] aimed to test the effectiveness of this implementation intervention, compared with the dissemination of a guideline on the management of mTBI patients presenting to ED [26].

Aim and objectives

Our primary objective was to establish whether the intervention increased the percentage of patients for whom a prospective measure of PTA using a validated tool was performed in the ED until a perfect score was achieved (indicating absence of acute cognitive impairment) or the patient was transferred or admitted.

Secondary objectives included establishing whether the intervention increased the percentage of patients for whom two other assessment methods of PTA were performed, for whom CT scanning was appropriately performed; who received written patient information upon discharge from the ED; and who received appropriate care according to outcomes measuring the implementation of multiple (composite) recommendations. In addition, we hypothesised that the provision of appropriate patient information [27] upon discharge from the ED would reduce anxiety and the number of self-reported symptoms. We also investigated the effects of our intervention on post-accident functioning (return to normal activities including work and health-related quality of life (HRQoL)) and head injury-related re-presentations. Finally, we aimed to assess the cost-effectiveness of the intervention and we conducted a process evaluation to aid the interpretation of the trial results. In this paper, we report the effects of the intervention on clinical practice and patient outcomes.

Methods

A protocol for this study has been published (Additional file 1) [25] and a brief overview of the methods follows. We describe deviations from planned methods (Additional file 2) and provide further detail of methods that had not been fully developed at the time of publication of the protocol. A completed CONSORT for cluster randomised trial reporting checklist, which indicates the sections of the paper where each reporting item is addressed, is available in Additional file 3. The trial was registered in the Australian New Zealand Clinical Trials Registry on 12 December 2012 (ACTRN12612001286831).

Ethics statement

The trial protocol was approved by the Alfred Health Human Research Ethics Committee (approval Number 398/12). Following recruitment, additional local ethics and research governance procedures were completed for each site. Details of consent and confidentiality procedures are available in the study protocol (Additional file 1).

Study design

The study design was a cluster randomised trial. Each cluster included an ED with its medical and nursing clinicians and the patients treated with mTBI. A cluster randomised design was primarily chosen because the intervention was targeted at ED staff. Two levels of participation in the study were offered, which we term NET and NET-Plus. In NET, clinical practice outcomes, but not patient outcomes, were measured, while in NET-Plus, both were measured.

Recruitment of EDs and inclusion/exclusion criteria

Recruitment of EDs occurred between February 2013 and October 2013. We approached EDs listed in the Australasian Society for Emergency Medicine ED Directory list of 24-h Australian EDs [28]. EDs were contacted in batches. All non-responding sites were followed up by email and phone. Exclusion criteria were (1) specialised hospitals not routinely treating adults with mTBI; (2) no CT scanner on site; (3) risk of contamination due to two EDs having the same ED Director, or senior influential clinicians working across sites (in which case only one ED was allowed to participate); and (4) sites having involvement in the pilot and/or development of the intervention. Hospitals were included if the ED director provided consent to enter the study (either NET or NET-Plus) on behalf of their staff by returning a completed consent form. Details of the ED recruitment process and consent procedures are outlined in the protocol (Additional file 1).

Identification of patients and inclusion/exclusion criteria

A retrospective chart audit of the ED medical records was conducted to identify eligible patients (see Additional file 4 for details on this process). Patients meeting the following criteria were included: (1) aged 18 or older, (2) presented to the ED within 24 h of injury, (3) sustained an acute blunt head trauma, and (4) had a GCS score of 14 or 15 at presentation [26]. Patients meeting the following criteria were excluded: (1) penetrating injuries and (2) non-traumatic brain-injury such as stroke. Two additional exclusion criteria were added: (3) patient left the ED before being seen or discharged themselves, and (4) the patient medical record was missing, with reasons outlined in Additional file 2. A waiver was granted to undertake the process of retrieving records of patients meeting our inclusion criteria without patient consent.

Recruitment of patients for follow-up and inclusion/exclusion criteria (NET-Plus only)

In hospitals which chose to participate in the NET-Plus study component, eligible patients identified from chart audit were contacted by telephone by an ED staff member and invited to participate in a follow-up telephone interview by psychologists experienced in interviewing individuals with brain trauma. Additional exclusion criteria for the NET-Plus component included (1) not being able to participate in a telephone interview (e.g. we were unable to support patients with hearing-impairments or provide translation services for patients who spoke languages other than English), (2) cognitive impairment from intellectual disability and/or neurological syndrome, and (3) severe substance use disorder and/or major psychiatric disorder requiring hospitalisation. Informed consent from patients to pass their contact details to the NET research team was first sought by the ED staff member. Following consent to share contact details, an information sheet was posted to the patient, which provided a 2-week opt-out option. After 2 weeks without opt-out, consent to participate was presumed. Prior to conducting the interview, the psychologists re-checked inclusion criteria. Patients were able to opt-out of the interview at any time.

Randomisation and allocation concealment

EDs were allocated to intervention or control groups using minimisation, a method that allocates clusters to groups using an algorithm to minimise differences in predefined factors between the groups [29]. Minimisation was implemented in the package minim [30]. Pure minimisation is completely deterministic; however, the algorithm we implemented included a random element. The allocation of EDs to intervention groups was undertaken externally to preclude any potential influence in the allocation by trial staff, study investigators, or study participants (i.e. ED directors). A statistician independent of the study implemented the minimisation in two batches. The statistician was only provided with ED identification codes and minimisation variables and was instructed to randomly sort the order in which the EDs would be entered into the minimisation package. The minimisation factors included (1) existence of a protocol for appropriate PTA assessment in mTBI patients, (2) size (annual presentation rate 2012), (3) rurality, and (4) level of participation (NET or NET-Plus).

Blinding

Due to the nature of the intervention, it was not possible to blind ED staff members to group allocation. To limit the possibility of selection and detection bias, chart auditors were independent of the hospital and blinded to ED group allocation. In addition, medical records staff who retrieved the records, patient interviewers and the statistician who performed the analyses were blinded to group allocation.

Intervention

The method of development of the intervention has been reported elsewhere [22]. In brief, prior to designing the intervention, we conducted interviews to identify the clinical and organisational factors that may influence the implementation of the three recommended practices [23, 24]. The content of the intervention was designed to target the important factors identified through the interviews. Both the interviews and the intervention design process were guided by two theoretical frameworks in a complementary manner [22]. The first of these, the Theoretical Domains Framework (TDF), is grounded in psychological theories of clinical behaviour change [31]. The second, the Model of Diffusion of Innovations in Service Organisations, was developed from an organisational perspective [32]. Next, intervention components were identified and operationalised. Behaviour change techniques that were most likely to bring about change for each clinical practice were identified, using sources that link techniques to the theoretical domains of the TDF [33,34,35]. In addition, the literature was consulted to identify intervention components that might be effective in targeting or taking into account organisational factors that were identified through the interviews [32, 36,37,38,39]. Finally, evidence on the effectiveness of interventions designed to improve healthcare delivery [40, 41] and information derived from the interviews regarding practicalities and feasibility of proposed intervention components was considered. Table 1 presents an overview of the delivery of intervention components, and Additional file 5 provides further details on the content and rationale for including each component.

Table 1 Delivery of the intervention
Full size table

Control

Control EDs received the guideline and data collection reminders only (components 1 and 2, Table 1). They were offered the full intervention following the conclusion of the trial.

Outcomes

The clinical practice and patient outcomes are described in Table 2. These represent a subset of all outcomes measured in the trial; effects of the intervention for the other outcomes (proxy measures of clinical practice and predictors of clinical practice (Additional file 1)) will be reported in a separate publication. Clinical practice outcomes include those which measure implementation of single and multiple (composite) recommendations.

Table 2 Clinical practice and patient outcomes
Full size table

Data collection processes

We collected baseline cluster characteristics including type of hospital (public or private), rurality and whether the site had a protocol for appropriate PTA assessment in mTBI patients via telephone with ED Directors or delegates. Clinical practice outcomes, patient characteristics and re-presentations were measured through retrospective chart audit. The follow-up period was 2 months in length and began post-delivery of the last intervention component at each site (ranging between October 2014 and February 2015). During follow-up, notices were implemented at all hospitals to remind staff of the importance of recording decisions in medical records for mTBI patients. Patient outcomes were collected via telephone interview.

Data quality assurance

Chart auditors were trained with the aim of maximising consistency in applying inclusion and exclusion criteria and collecting data from medical records. The auditors received a data collection manual with instructions regarding data entry in the web-based database (including a data dictionary), and weekly phone meetings were held to discuss questions. In addition, data were downloaded twice during each hospital audit and data checks were run to identify inconsistencies and errors, which were discussed with chart auditors the same day to enable correction of data entries.

Researchers conducting patient interviews received training and supervision to ensure standardised delivery of interview questions. Patient outcome data were entered directly into the web-based database.

The database was designed to minimise errors through real-time checks.

Sample size

To detect an absolute increase of 20% in the rate of appropriate PTA screening (equivalent to an odds ratio of 3.9, log odds 1.3) (assuming a control group rate of 10%, an intra-cluster correlation (ICC) of 0.18, coefficient of variation of cluster size of 0.47, an average of 30 patient participants per ED, and a two-sided 5% significance level) with approximately 80% power, we required 15 EDs per intervention group. A total of 30 EDs would provide 900 patient participants for whom ED staff management would be assessed. Allowing for 10% attrition, we planned to initially recruit 34 EDs. Rationale and empirical support for the sample size parameters and justification for the target difference we wished to detect between groups is provided in the trial protocol (Additional file 1). The sample size calculations were undertaken using the module clustersampsi [42], implemented in the statistical package Stata (StataCorp LP, USA) [43].

Effectiveness analyses

The effectiveness of the intervention for the clinical practice and patient outcomes was estimated with marginal modelling using generalised estimating equations (GEEs). These models appropriately account for the correlation of responses of individuals within EDs. An exchangeable correlation structure was specified, whereby responses from the same ED were assumed to be equally correlated [44]. We used robust variance estimation which yields valid standard errors even if the within-cluster correlation has been incorrectly specified [45, 46]. For binary outcomes, a logit link was used. For continuous outcomes, model specification tests were undertaken to determine the probability distribution and link function.

All models included adjustment for minimisation factors (see ‘Randomisation and allocation concealment’ section) and additionally (unless otherwise noted) for pre-specified confounders that included patients’ age, sex, and whether they presented after hours. All confounders were included in the models even when no baseline imbalance existed. Our primary effectiveness analysis was the model (as described above) that estimated the intervention effect on the primary outcome, appropriate PTA screening.

Estimates of intervention effect from the models with binary outcomes yielded odds ratios. To aid interpretability, we also provided estimates of absolute risk differences [47], computed from marginal probabilities estimated from the fitted logistic models [44]. Analysis additions and deviations from the protocol are outlined in Additional file 2.

Results

Participation of EDs

Fifty-three expression of interest forms were received, and 50 information meetings held. Subsequently, 34 ED Directors (or delegates) provided written consent to participate in the trial. Three sites declined after or during completion of ethics documentation, which left 31 sites for completion of baseline characteristics and randomisation. Fourteen sites were allocated through minimisation to the intervention group and 17 sites to the control group. Twenty-seven sites consented to participate in NET-Plus, three of which (all intervention sites) did not recruit patients for follow-up interviews. Figure 1 shows the flow of sites through the trial.

Fig. 1
figure 1

Participant flow diagram

Full size image

Baseline characteristics of EDs and clinicians

Participating EDs were primarily public hospitals, and the majority were based in urban areas. Less than a quarter of EDs had a protocol for appropriate PTA assessment. Groups were well balanced across the demographic measures (Table 3).

Table 3 Baseline demographic characteristics of EDs
Full size table

Demographic and clinical characteristics of included patients

NET-Trial patients

A total of 1943 patients were identified from medical records for inclusion in the study (Table 4, column 2 and 3). The demographic and clinical characteristics of the patients were similar between groups, although intervention patients had more frequently experienced other injuries and had a higher mean age.

Table 4 Patient characteristics
Full size table

NET-Plus patients

In total, 343 patients participated in the patient follow-up study (Table 4, column 4 and 5). Compared to the intervention group, the control group included a slightly higher proportion of males, more with loss of consciousness and fewer with headache recorded in their notes.

Effects of the intervention on clinical practice outcomes

Clinical practice outcomes

Outcomes measuring the implementation of single clinical recommendations

Patients from EDs in the intervention group compared to those in the control group were more likely to have been appropriately assessed for PTA (primary outcome; adjusted OR 20.1, 95%CI 6.8 to 59.3, p <  0.001; which converts to an adjusted absolute risk difference (ARD) of 14%, 95%CI 8 to 19, Table 5). However, the percentage of patients who were screened appropriately in both groups was small (1% and 13% in the control and intervention groups respectively). Patients from intervention EDs were more likely to have had at least one administration of the validated PTA tool (PTA screening-tool; adjusted OR 19.7, 95%CI 6.6 to 58.1, p <  0.001) and to have had an assessment of PTA where the clinician used clinical questions (but no tool) (memory-clinical assessment; adjusted OR 1.6, 95%CI 1.2 to 2.1, p = 0.001).

Table 5 Estimated effects of the intervention on clinical practice outcomes
Full size table

The difference between groups in the odds of compliance with recommendations for CT scanning were small and not statistically significant (Table 5). For both outcomes (CT scan-clinical criteria, and CT scan (all)), the odds were 1.2 times higher in the intervention compared with the control group (95%CI 0.8 to 1.6 and 0.9 to 1.6 respectively). In both groups, around 70% of patients for whom risk factors were noted in the medical record received a scan that was justified by those symptoms.

Similarly, the difference between groups in the odds of compliance with the recommendation for the provision of written patient discharge information was small (adjusted OR 1.2, 95%CI 0.8 to 1.8). In both groups, only around 20% of patients received written patient information upon discharge from the ED.

Outcomes measuring the implementation of multiple (composite) recommendations

Patients from EDs randomised to the intervention compared to the control group were more likely to have had safe discharge, both based on PTA and INFO (whole cohort; adjusted OR 27.6, 95%CI 6.9 to 110.5, p <  0.001) and based on PTA, CT and INFO (subset of cohort; adjusted OR 1.8, 95%CI 1.1 to 3.0, p = 0.022). In both groups, however, percentages of patients who received appropriate care according to our composite indicators of clinical practice were very low. This was caused by the low baseline rates for PTA and INFO predominantly.

Effects of the intervention on patient outcomes

In total, 343 patients were interviewed at follow-up (Tables 4 and 6). The mean number of days between their ED presentation and the follow-up interview was 210 days (SD 38.5 days; IQR 181–239). The intervention had a small effect on anxiety, with the observed difference being − 0.52 (95%CI − 1.34 to 0.30, p = 0.216, Table 6) in favour of the intervention. Rates of post-concussion symptoms were low in both groups, and the intervention had only a small effect on RPQ-13 scores, which were 1.15 (95%CI − 2.77 to 0.48) lower for intervention patients, and RPQ-3 scores, which were 1.10 (95%CI − 0.48 to 0.28) lower in intervention patients. The percentage of patients who had not returned to normal activities was 19% and 13% in the control and intervention groups respectively (adjusted OR 0.67, 95%CI 0.28 to 1.61; which converts to an adjusted ARD of − 4.6% (95%CI − 16.2 to 7.0). There was uncertainty of the impact of the intervention on SF6D HRQoL scores which were 0.03 (95%CI 0.00 to 0.06) higher for those in the intervention group. The confidence interval included both important [48, 49] and trivial differences. The odds of re-presentation for those in the intervention group were nearly twice those in the control group (adjusted OR 1.92, 95%CI 1.08 to 3.40, p = 0.026, which converts to an adjusted ARD of 2.1%, 95%CI 0.3 to 3.8), a small difference of uncertain clinical significance.

Table 6 Effects of the intervention on patient outcomes
Full size table

Sensitivity analyses: effect estimates from models adjusting for minimisation criteria only

For clinical practice outcomes, the effect estimates from models in which there was only adjustment for the minimisation factors (see Additional file 2) were not appreciably different compared with the models that in addition adjusted for pre-specified confounders (Additional file 6). The exception to this was the primary outcome ‘appropriate PTA screening’, where the OR of 20.1 (95%CI 6.8 to 59.3) from the full model reduced to 15.6 (95%CI 5.0 to 48.8) for the model that only included the minimisation factors. This difference was influenced by the imbalance in age at baseline, where patients in the intervention group were on average older, and appropriate PTA screening was more likely to occur in younger patients. For patient outcomes, no meaningful differences were observed between the effect estimates obtained from models with and without adjustment for pre-specified confounders.

Intra-cluster correlations (ICCs) for the primary outcomes

The ICC for our primary clinical practice outcome (appropriate PTA screening) was 0.12 (95%CI 0.06 to 0.19). However, estimates of ICCs differed for the two groups, with the ICC in the intervention group (0.06 (95%CI 0.00 to 0.11)) being smaller than the control (0.20 (95%CI 0.08 to 0.32)), potentially suggesting that clinical practice for PTA screening may have become more consistent across intervention EDs (Additional file 6—ICCs for clinical practice outcomes). The difference in prevalence rates between the groups may also provide a part explanation for the differences in estimated ICCs [50]. The ICC for our primary patient outcome (anxiety) was 0.02 (95%CI 0.01 to 0.07; Additional file 6).

Discussion

We conducted a trial of a targeted, theory-informed implementation intervention to increase the uptake of clinical practice recommendations for the management of patients presenting to Australian EDs with mTBI. Results suggest that our intervention improved management, increasing the percentage of patients being appropriately assessed for PTA and of ‘safe discharge’ (based on both composite scores). The observed improvement in our composite measures mainly reflects the improvement in PTA as the intervention did not appreciably increase the uptake of the other two practice recommendations. The impact of the intervention on patient outcomes was generally in favour of the intervention group, but estimated effects were small and of limited clinical significance. Anxiety levels at follow-up in both groups were low (intervention mean 3.4, SD 3.58; control mean 4.3, SD 4.01), and the intervention had only a small effect in favour of intervention patients (adjusted mean difference − 0.52, 95%CI − 1.34 to 0.30; scale 0–21). The latter is perhaps unsurprising since our intervention did not improve the provision of patient information, which was the mechanism by which we hypothesised anxiety would improve.

While the observed effect for our primary outcome was smaller than the 20% difference in absolute improvement, we powered our trial to detect (Additional file 1), the effect was in fact larger than that observed in many trials testing similar interventions [41]. Further, the confidence bounds suggest that the true intervention effect could plausibly be as small as an 8% improvement, or as large as a 19% improvement, with the latter magnitude consistent with that which we set out to detect. This improvement means that more patients received care in concordance with best clinical practice; they had a record of PTA duration (which is important for diagnosis and management) [51], and fewer patients were sent home in unsafe conditions (i.e., while still experiencing acute but temporary cognitive impairment) [52, 53]. The intervention effect was somewhat larger for the ‘PTA screening tool’. This demonstrates that PTA screening was started but not maintained until the patient had a perfect score before the patient was determined safe for discharge. EDs may find it difficult to repeat assessment under time and resource pressure. However, this is a crucial aspect of PTA assessment.

As is the case with every new intervention, ultimately, health service providers and fundholders would need to decide what size of improvement would be important enough in their setting to justify any increase in costs associated with adopting the intervention. The economic evaluation that was conducted alongside this trial considered the trade-off between the net costs of the implementation intervention and improvements in clinical practice and health outcomes. The authors conclude that, as delivered in the trial, the balance of costs and outcomes from the implementation intervention is unlikely to be acceptable to providers and fundholders. Full results and further reflections on this can be found in Mortimer et al. [54].

While very few EDs in both groups were screening for PTA, there may have been less room for improvement for CT scanning. CT rates were 44% in the control group at follow-up. Comparison between studies is complicated due to differences in definitions, methods, study population and the fact that the criteria that justified a scan in our study differ from published studies. However, this potentially indicates there was only modest room for improvement, as the percentage of CT head scans in this patient group that would have been required by applying various head rules (calculated by dividing the number of mTBI patients in whom the decision rule was positive divided by the total number of mTBI patients) has been estimated to range roughly between 50 and 70% ([55,56,57], although lower (42% [57], 43% [58]) and higher (79 and 96%) [57] rates have also been reported). This does not apply however to the provision of written discharge information, with no more than 20% of patient records in both groups including documentation of patients receiving written materials, although these latter rates may have been influenced by incomplete recording (see study strengths and limitations).

Many factors may explain why the intervention was effective in increasing the uptake of PTA screening, but not the clinical recommendations related to appropriate CT scanning and provision of patient information on discharge. For example, it may have been the case that the content of the intervention (e.g. the components we selected) did not address all identified barriers for these practices. It could also be that the intervention was not implemented as intended (e.g. content surrounding PTA assessment received more attention in the local training workshops provided within the EDs by the opinion leaders as it was a new or less familiar practice). Our process evaluation will provide insight on potential explanations.

What this study adds to the literature

To our knowledge, few studies have evaluated theory-informed, targeted interventions in an ED setting. A cluster trial that included 12 matched pairs of community hospitals [59] concluded that their implementation intervention failed to significantly increase alteplase use in patients with ischaemic stroke. Although their intervention addressed local barriers in each intervention site, the intervention was similar to that of the NET trial in that it was designed to alter systems and behaviour at an institutional level and individual staff level, focusing on change in the ED setting. The intervention was based on behaviour change theory and adapted from previous experience in the development of alteplase delivery systems.

In terms of contributing to the body of knowledge relating to the implementation of guideline recommendations, the NET trial will inform research examining the effectiveness of organisational and professional interventions in emergency practice settings in increasing uptake of research evidence [60], as well as—in any setting—the effectiveness of multi-faceted interventions versus single-component interventions [61], the effectiveness of targeted versus non-targeted interventions [18] and of theory-informed interventions versus other interventions [18] in increasing the uptake of recommendations.

Study strengths and limitations

This study has limitations. First, the majority of EDs invited to participate in the trial declined participation. This may therefore limit the generalisability of the results since the characteristics of the non-participating EDs may have differed, and these characteristics may influence the effectiveness of the intervention.

As we were conducting a retrospective audit to select patients for inclusion in the trial, we relied on discharge coding in the medical record systems to identify patients. Previous research has shown coding in TBI populations is likely to be incomplete and/or inaccurate [62,63,64] particularly when other injuries were involved. Therefore, it is likely we have missed mTBI patients in our audits. Where possible, we included text searches in triage notes using head injury-related terms to identify patients with non-head injury-related codes. The proportion of sites where this was not possible was similar across groups. The comparability of the patient characteristics demonstrates that the identification processes were implemented similarly by group.

Although previous studies have indicated incomplete information in patient records [65], we collected data from medical records retrospectively. Therefore, our trial outcomes were dependent on the recording practices and clinical information available in the medical records. Previous studies have noted this may particularly be an issue for practices such as providing advice [66, 67]. Indeed, in over 55% of patients, no information on receipt of patient information at discharge was recorded. For patients who had records of written patient information upon discharge in their files, we were unable to determine whether the information provided was in fact the intervention booklet [27]. Several alternative patient information sheets are available for EDs, and these may not include information such as reassurance and the importance of gradual return to activities. In addition, due to missing information, we were not able to study appropriate denial of CT head, as this would require recorded evidence in the notes of the absence of all possible criteria justifying a scan.

We intended to improve recording in the medical record systems by implementing a data collection reminder in all participating EDs. In addition, we asked study coordinators to remind their staff of the importance of including full information in the clinical record during the trial catchment period. However, it is unlikely that this would have led to differences in incomplete reporting between groups.

The duration of the follow-up of the study may also have been a limitation. We collected chart audit data over the 2 months directly following a 3-month local intervention delivery period in each site. It could be the case that this period was too short for the intervention to be fully embedded in routine practice. Conversely, it is also conceivable that the observed intervention effect fades out over time. Therefore, it would have been informative to study sustainability by including a later time point for repeat chart audit [68]. This was not feasible within the timeframe of the trial. In addition, although originally planned to take place at 3 to 5 months post-injury, delays in recruitment of participants by ED staff meant that clinical follow-up in the NET-Plus trial did not take place until an average of 7 months post-injury and there was fairly wide range in time post-injury at which participants were followed up (control mean 209.1 days, SD 35.9 and intervention mean 212.7, SD 42.6). This may have impacted on rates of reporting of symptoms. Also, the intervention may have had an early clinically important effect on patient outcomes that we could not assess.

This study also has some clear strengths, such as the process used to minimise selection bias in the allocation of EDs to the intervention groups through our implementation of the minimisation method. This included (i) using a minimisation algorithm that had a random element so that the allocation was not fully deterministic, (ii) having a statistician independent of the trial implement the process using batches of EDs, and (iii) randomly sorting the order in which EDs were entered into the minimisation program. Additional strengths are that we used a systematic process to design the intervention in order to maximise the likelihood of effectiveness, that we included objective measures of practice and that we included a process evaluation as well as an economic evaluation. In addition, we used independent chart auditors and only one chart auditor (who also was an experienced ED nurse) selected patient records for retrieval by medical records departments of participating sites. Several measures were in place to assure consistent data entry between chart auditors, such as training, phone meetings on a weekly basis to discuss any questions, circulation of decisions taken and real-time data downloads and checks based on algorithms. Finally, the EDs we recruited were distributed across the country.

Conclusions

We report the results of a large, nationwide trial of a targeted, theory-informed implementation intervention in emergency care settings where, to date, relatively few trials have been carried out. The intervention had an important impact on appropriate PTA assessment, but did not have an appreciable impact on appropriate CT scanning and written patient information on discharge at 2 months follow-up. Further, the impact of the intervention on patient outcomes was either clinically uncertain or not clinically important. Future evaluations may focus on modifying the developed intervention to bring about larger improvement and longevity of the effects of the intervention.

Abbreviations

CT:

Computed tomography

ED:

Emergency department

GCS:

Glasgow Coma Scale

HRQoL:

Health-related quality of life

ICC:

Intra-cluster correlation

INFO:

Provision of written patient information upon discharge

mTBI:

Mild traumatic brain injury

NET:

Neurotrauma Evidence Translation

PTA:

Post-traumatic amnesia

RPQ:

Rivermead Post-Concussion Symptoms Questionnaire

SF-12:

12-item Short Form Health Survey

SF6D:

Short-Form Six-Dimension

References

  1. Maas AI, Stocchetti N, Bullock R. Moderate and severe traumatic brain injury in adults. Lancet Neurol. 2008;7(8):728–41.

    Article  PubMed  Google Scholar 

  2. Thurman D, Guerrero J. Trends in hospitalization associated with traumatic brain injury. JAMA. 1999;282(10):954–7.

    Article  CAS  PubMed  Google Scholar 

  3. Kristman VL, Borg J, Godbolt AK, Salmi LR, Cancelliere C, Carroll LJ, et al. Methodological issues and research recommendations for prognosis after mild traumatic brain injury: results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil. 2014;95(3):S265–77.

    Article  PubMed  Google Scholar 

  4. Ponsford J, Willmott C, Rothwell A, Cameron P, Kelly AM, Nelms R, et al. Factors influencing outcome following mild traumatic brain injury in adults. J Int Neuropsychol Soc. 2000;6(5):568–79.

    Article  CAS  PubMed  Google Scholar 

  5. Theadom A, Cropley M, Parmar P, Barker-Collo S, Starkey N, Jones K, et al. Sleep difficulties one year following mild traumatic brain injury in a population-based study. Sleep Med. 2015;16(8):926–32.

    Article  PubMed  Google Scholar 

  6. Donker-Cools BH, Wind H, Frings-Dresen MH. Prognostic factors of return to work after traumatic or non-traumatic acquired brain injury. Disabil Rehabil. 2015;3:1–9.

    Google Scholar 

  7. Stiell IG, Clement CM, Rowe BH, Schull MJ, Brison R, Cass D, et al. Comparison of the Canadian CT Head Rule and the New Orleans Criteria in patients with minor head injury. JAMA. 2005;294(12):1511–8.

    Article  CAS  PubMed  Google Scholar 

  8. Tavender EJ, Bosch M, Green S, O'Connor D, Pitt V, Phillips K, et al. Quality and consistency of guidelines for the management of mild traumatic brain injury in the emergency department. Acad Emerg Med. 2011;18(8):880–9.

    Article  PubMed  Google Scholar 

  9. Bosch M, Tavender E, Bragge P, Gruen R, Green S. How to define 'best practice' for use in Knowledge Translation research: a practical, stepped and interactive process. J Eva Clin Pract. 2013;19(5):763–8 [Epub ahead of print 2012 Apr 9].

    Google Scholar 

  10. Bazarian JJ, McClung J, Cheng YT, Flesher W, Schneider SM. Emergency department management of mild traumatic brain injury in the USA. Emerg Med J. 2005;22(7):473–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Campbell. Are head injury guidelines being met? Clin Governance An Int J. 2005;10(1):2.

    Article  Google Scholar 

  12. Heskestad B, Baardsen R, Helseth E, Ingebrigtsen T. Guideline compliance in management of minimal, mild, and moderate head injury: high frequency of noncompliance among individual physicians despite strong guideline support from clinical leaders. J Trauma. 2008;65(6):1309–13.

    Article  PubMed  Google Scholar 

  13. Kerr J, Swann IJ, Pentland B. A survey of information given to head-injured patients on direct discharge from emergency departments in Scotland. Emerg Med J. 2007;24(5):330–2.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Melnick ER, Szlezak CM, Bentley SK, Dziura JD, Kotlyar S, Post LA. CT overuse for mild traumatic brain injury. Jt Comm J Qual Patient Saf. 2012;38(11):483–9.

    Article  PubMed  Google Scholar 

  15. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–65.

    Article  CAS  PubMed  Google Scholar 

  16. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225–30.

    Article  PubMed  Google Scholar 

  17. Walshe K. Understanding what works - and why - in quality improvement: the need for theory-driven evaluation. Int J Qual Health Care. 2007;19(2):57–9.

    Article  PubMed  Google Scholar 

  18. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015;4:CD005470.

    Google Scholar 

  19. Boudreaux ED, Cydulka R, Bock B, Borrelli B, Bernstein SL. Conceptual models of health behavior: research in the emergency care settings. Acad Emerg Med. 2009;16(11):1120–3.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Compton S, Lang E, Richardson TM, Hess E, Green J, Meurer W, et al. Knowledge translation consensus conference: research methods. Acad Emerg Med. 2007;14(11):991–5.

    Article  PubMed  Google Scholar 

  21. Green SE, Bosch M, McKenzie JE, O'Connor DA, Tavender EJ, Bragge P, et al. Improving the care of people with traumatic brain injury through the Neurotrauma Evidence Translation (NET) program: protocol for a program of research. Implement Sci. 2012;7:74.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Tavender EJ, Bosch M, Gruen RL, Green SE, Michie S, Brennan SE, et al. Developing a targeted, theory-informed implementation intervention using two theoretical frameworks to address health professional and organisational factors: a case study to improve the management of mild traumatic brain injury in the emergency department. Implement Sci. 2015;10(1):74.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Bosch M, Tavender EJ, Brennan SE, Knott J, Gruen RL, Green SE. The many organisational factors relevant to planning change in emergency care departments: a qualitative study to inform a cluster randomised controlled trial aiming to improve the management of patients with mild traumatic brain injuries. PLoS One. 2016;11(2):e0148091.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Tavender EJ, Bosch M, Gruen RL, Green SE, Knott J, Francis JJ, et al. Understanding practice: the factors that influence management of mild traumatic brain injury in the emergency department--a qualitative study using the Theoretical Domains Framework. Implement Sci. 2014;9:8.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Bosch M, McKenzie J, Mortimer D, Tavender E, Francis J, Brennan S, et al. Implementing evidence-based recommended practices for the management of patients with mild traumatic brain injuries in Australian emergency care departments: study protocol for a cluster randomised controlled trial. Trials. 2014;15(1):281.

    Article  PubMed  PubMed Central  Google Scholar 

  26. NSW Institute of Trauma and Injury Management. Initial management of closed head injury in adults, second edition. North Sydney: NSW Ministry of Health; 2011.

  27. Ponsford J, Willmott C, Rothwell A, Cameron P, Kelly AM, Nelms R, et al. Impact of early intervention on outcome following mild head injury in adults. J Neurol Neurosurg Psychiatry. 2002;73(3):330–2.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Australasian Society for Emergency Medicine (ASEM). Directory of Emergency Departments of Australia and New Zealand; 2009.

  29. Scott NW, McPherson GC, Ramsay CR, Campbell MK. The method of minimization for allocation to clinical trials a review. Control Clin Trials. 2002;23(6):662–74.

    Article  PubMed  Google Scholar 

  30. Evans S RP, Day S. Minim: allocation by minimisation in clinical trials Available: http://www-users.york.ac.uk/~mb55/guide/minim.htm

  31. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;4(1):26–33.

    Article  Google Scholar 

  32. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Cane J, Richardson M, Johnston M, Ladha R, Michie S. From lists of behaviour change techniques (BCTs) to structured hierarchies: comparison of two methods of developing a hierarchy of BCTs. Br J Health Psychol. 2015;20(1):130–50.

    Article  PubMed  Google Scholar 

  34. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81–95.

    Article  PubMed  Google Scholar 

  35. Michie S, Johnston M, Francis J, Hardeman W, Eccles M. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008;57(4):660–80.

    Article  Google Scholar 

  36. Colquhoun H, Leeman J, Michie S, Lokker C, Bragge P, Hempel S, et al. Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems and policies. Implement Sci. 2014;9:51.

    PubMed  PubMed Central  Google Scholar 

  37. Leeman J, Baernholdt M, Sandelowski M. Developing a theory-based taxonomy of methods for implementing change in practice. J Adv Nurs. 2007;58(2):191–200.

    Article  PubMed  Google Scholar 

  38. Mazza D, Bairstow P, Buchan H, Chakraborty SP, Van Hecke O, Grech C, Kunnamo I. Refining a taxonomy for guideline implementation: results of an exercise in abstract classification. Implement Sci. 2013;8:32.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57.

    Article  PubMed  Google Scholar 

  40. The Cochrane Library. EPOC reviews, available from https://epoc.cochrane.org/epoc-reviews.

  41. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Hemming K, Marsh J. A menu-driven facility for sample-size calculations in cluster randomized controlled trials. Stata J. 2013;13(1):114–35.

    Article  Google Scholar 

  43. StataCorp. Stata Statistical Software: Release 14. College Station: StataCorp LP; 2015.

    Google Scholar 

  44. Ukoumunne OC, Gulliford MC, Chinn S, Sterne JA, Burney PG. Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review. Health Technol Assess. 1999;3(5):iii–92.

    CAS  PubMed  Google Scholar 

  45. Fitzmaurice GM, Laird NM, Ware JH. Marginal models: generalized estimating equations (GEE). In: Balding DJ, Cressie NAC, Fisher NI, Johnstone IM, Kadane JB, Molenberghs G, et al., editors. Applied longitudinal analysis. Hoboken, New Jersey: John Wiley; 2004.

    Google Scholar 

  46. Hanley JA, Negassa A, Edwardes MD, Forrester JE. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol. 2003;157(4):364–75.

    Article  PubMed  Google Scholar 

  47. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–37.

    Article  PubMed  Google Scholar 

  48. Walters SJ, Brazier JE. Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res. 2005;14(6):1523–32.

    Article  PubMed  Google Scholar 

  49. Luo N, Johnson J, Coons SJ. Using instrument-defined health state transitions to estimate minimally important differences for four preference-based health-related quality of life instruments. Med Care. 2010;48(4):365–71.

    Article  PubMed  Google Scholar 

  50. Gulliford MC, Adams G, Ukoumunne OC, Latinovic R, Chinn S, Campbell MJ. Intraclass correlation coefficient and outcome prevalence are associated in clustered binary data. J Clin Epidemiol. 2005;58(3):246–51.

    Article  CAS  PubMed  Google Scholar 

  51. Carroll LJ, Cassidy JD, Holm L, Kraus J, Coronado VG. Methodological issues and research recommendations for mild traumatic brain injury: the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. J Rehab Med. 2004;(43 Suppl):113–125.

  52. Meares S, Shores EA, Taylor AJ, Lammel A, Batchelor J. Validation of the Abbreviated Westmead Post-traumatic Amnesia Scale: a brief measure to identify acute cognitive impairment in mild traumatic brain injury. Brain Inj. 2011;25(12):1198–205.

    Article  PubMed  Google Scholar 

  53. Nell V, Yates DW, Kruger J. An extended Glasgow Coma Scale (GCS-E) with enhanced sensitivity to mild brain injury. Arch Phys Med Rehabil. 2000;81(5):614–7.

    Article  CAS  PubMed  Google Scholar 

  54. Mortimer D, Bosch M, McKenzie J, Turner S, Chau M, Ponsford J, et al. Economic evaluation of the NET intervention versus guideline dissemination for management of mild head injury in hospital emergency departments. Implement Sci. 2018;13:147. https://doi.org/10.1186/s13012-018-0834-6.

  55. Stein SC, Fabbri A, Servadei F, Glick HA. A critical comparison of clinical decision instruments for computed tomographic scanning in mild closed traumatic brain injury in adolescents and adults. Ann Emerg Med. 2009;53(2):180–8.

    Article  PubMed  Google Scholar 

  56. Tan DW, Lim AME, Ong DY, Peng LL, Chan YH, Ibrahim I, et al. Computed tomography of the head for adult patients with minor head injury: are clinical decision rules a necessary evil? Singap Med J. 2018;59(4):199–204.

    Article  Google Scholar 

  57. Foks KA, van den Brand CL, Lingsma HF, van der Naalt J, Jacobs B, de Jong E, et al. External validation of computed tomography decision rules for minor head injury: prospective, multicentre cohort study in the Netherlands. BMJ. 2018;362:k3527.

    Article  PubMed  PubMed Central  Google Scholar 

  58. DeAngelis J, Lou V, Li T, Tran H, Bremjit P, McCann M, Crane P, et al. Head CT for minor head injury presenting to the emergency department in the era of choosing wisely. West J Emerg Med. 2017;18(5):821–9.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Scott PA, Meurer WJ, Frederiksen SM, Kalbfleisch JD, Xu Z, Haan MN, et al. A multilevel intervention to increase community hospital use of alteplase for acute stroke (INSTINCT): a cluster-randomised controlled trial. Lancet Neurol. 2013;12(2):139–48.

    Article  PubMed  Google Scholar 

  60. Curran. Organisational and professional interventions to promote the uptake of evidence in emergency care: effects on professional practice and health outcomes (protocol). Cochrane Database of Syst Rev. 2009;(2).

  61. Squires JE, Sullivan K, Eccles MP, Worswick J, Grimshaw JM. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:152.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Chan V, Thurairajah P, Colantonio A. Defining traumatic brain injury in children and youth using international classification of diseases version 10 codes: a systematic review protocol. Syst Rev. 2013;2:102.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Bazarian JJ, Veazie P, Mookerjee S, Lerner EB. Accuracy of mild traumatic brain injury case ascertainment using ICD-9 codes. Acad Emerg Med. 2006;13(1):31–8.

    Article  PubMed  Google Scholar 

  64. Chen AY, Colantonio A. Defining neurotrauma in administrative data using the international classification of diseases tenth revision. Emerg Themes in Epidemiol. 2011;8(1):4.

    Article  Google Scholar 

  65. Zegers M, de Bruijne MC, Spreeuwenberg P, Wagner C, Groenewegen PP, van der Wal G. Quality of patient record keeping: an indicator of the quality of care? BMJ Qual Saf. 2011;20(4):314–8.

    Article  PubMed  Google Scholar 

  66. Hrisos S, Eccles MP, Francis JJ, Dickinson HO, Kaner EF, Beyer F, et al. Are there valid proxy measures of clinical behaviour? A systematic review. Implement Sci. 2009;4:37.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Eccles MP, Hrisos S, Francis JJ, Stamp E, Johnston M, Hawthorne G, et al. Instrument development, data collection, and characteristics of practices, staff, and measures in the Improving Quality of Care in Diabetes (iQuaD) Study. Implement Sci. 2011;6:61.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Tricco AC, Ashoor HM, Cardoso R, MacDonald H, Cogo E, Kastner M, et al. Sustainability of knowledge translation interventions in healthcare decision-making: a scoping review. Implement Sci. 2016;11(1):55.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Valente TW, Pumpuang P. Identifying opinion leaders to promote behavior change. Health Educ Behav. 2007;34(6):881–96.

    Article  PubMed  Google Scholar 

  70. Whelan-Goodinson R, Ponsford J, Schonberger M. Validity of the Hospital Anxiety and Depression Scale to assess depression and anxiety following traumatic brain injury as compared with the Structured Clinical Interview for DSM-IV. J Affect Disord. 2009;114(1–3):94–102.

    Article  PubMed  Google Scholar 

  71. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–70.

    Article  CAS  PubMed  Google Scholar 

  72. King NS, Crawford S, Wenden FJ, Moss NE, Wade DT. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol. 1995;242(9):587–92.

    Article  CAS  PubMed  Google Scholar 

  73. Brazier JE, Rowen D, Hanmer J. Revised SF-6D scoring programmes: a summary of improvements. PRO Newsletter. 2008;40:14–5.

    Google Scholar 

  74. Ware J Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220–33.

    Article  PubMed  Google Scholar 

  75. Austin PC. Absolute risk reductions, relative risks, relative risk reductions, and numbers needed to treat can be obtained from a logistic regression model. J Clin Epidemiol. 2010;63(1):2–6.

    Article  PubMed  Google Scholar 

  76. Brazier JE, Roberts J. The estimation of a preference-based measure of health from the SF-12. Med Care. 2004;42(9):851–9.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We are grateful to all the hospitals who participated in our study: Albury-Wodonga Health (Wodonga campus) Hospital; Alfred Health operating through Sandringham Hospital; Armadale Hospital; Bairnsdale Regional Health Service; Bunbury Regional Hospital; Bundaberg Hospital; Cairns and Hinterland Hospital and Health Service (Cairns Hospital); Coffs Harbour Base Hospital; Central Adelaide Local Health Network Incorporated, Operating as The Queen Elizabeth Hospital; Eastern Health through Box Hill Hospital; Fremantle Hospital and Health Service; Gold Coast Hospital and Health Service; Gosford Hospital; Hunter New England Local Health District acting through Maitland Hospital and Tamworth Base Hospital; Knox Private Hospital; Mackay Hospital and Health Service; Metro North Hospital and Health Service acting through Royal Brisbane and Women’s Hospital; Metro South Hospital and Health Service via the Queen Elizabeth II Jubilee Hospital and Redland Hospital; Mildura Base Hospital; Noarlunga Hospital; Northeast Health Wangaratta Hospital; Northern Sydney Local Health District through Hornsby Hospital; St. John of God (Ballarat) Hospital; St. Vincent’s Hospital (Sydney) Ltd.; Sunshine Hospital; The Canberra Hospital; The Prince Charles Hospital; Warrnambool Base Hospital; Western Sydney Local Health District through Blacktown Hospital.

In addition, we thank Ornella Clavisi for management support as NET-Program Manager, Prof. Andrew Forbes for conducting the randomisation process, Dr. Matthew Page for assessing fidelity of the intervention during both Train-the-Trainer days, Laura Varanelli, Loyal Pattuwage, Madeleine Hill and Huey Ming Seah for providing research assistance. We thank our chart auditors Angela Weber, Nathan Farrow, Crystal Seah, Natalie Lott, and Sue Boucher; we thank Prof. Jane Fisher and Dr. Sara Holton for conducting process evaluation interviews and Dr. Sylvia Nguyen and research assistants at the Monash Epworth Rehabilitation Research Centre for conducting patient follow-up interviews. We thank Research Path for providing data management services and their contributions to the design of the customised features and functionalities of their web-based clinical trial management system.

We acknowledge everyone who has been part of the NET Program as Steering Group Members: Dr. Mark Bayley, Dr. Heather Buchan, Prof. Peter Cameron, Ms. Ornella Clavisi, Dr. David Cooksley, Prof. Jamie Cooper, Prof. Niki Ellis, A/Prof. Mark Fitzgerald, Prof. Jill Francis, Prof. Sally Green, Prof. Jeremy Grimshaw, Prof. Russell Gruen, A/Prof. Claire Harris, Ms. Sue Huckson, Prof. Andrew Kaye, Prof. Fary Khan, A/Prof. Jonathan Knott, Prof. John Lavis, Prof. Shawn Marshall, Mr. Steve McDonald, Prof. Susan Michie, Prof. Peter Morley, Dr. Andrew Morokoff, Dr. Andrew Pearce, Prof. Jennie Ponsford, Prof. Jeffrey V Rosenfeld, Mr. Nick Rushworth, Dr. Lisa Sherry.

Funding

This trial was part of a 5-year grant (the Neurotrauma Evidence Translation (NET) Program), funded by the Victorian Transport Accident Commission (TAC), Australia (https://www.tac.vic.gov.au/). RLG is supported by a Practitioner Fellowship from the Australian National Health and Medical Research Council. DAO is supported by an Australian NHMRC Public Health Fellowship (606726). JMG holds a Canada Research Chair in Health Knowledge and Transfer. EJT is supported by an Australian Postgraduate Award, Australian National Health and Medical Research Council Fellowship. JEM holds an NHMRC Australian Public Health Fellowship (1072366). All funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. JEM, ST, SEG, MB and DM had full access to all the data in the study, and the corresponding authors had final responsibility for the decision to submit for publication.

Availability of data and materials

Data are available upon request. Initial queries should be submitted to Alfred Hospital Ethics Committee (research@alfred.org.au). Publication of data files is not possible as ethics approvals and consent do not permit data to be stored publicly.

Author information

Authors and Affiliations

  1. Department of Surgery, Monash University, Melbourne, Australia

    Marije Bosch, Marisa Chau, Emma J. Tavender & Jeffrey V. Rosenfeld

  2. National Trauma Research Institute, Alfred Hospital and Monash University, Melbourne, Australia

    Marije Bosch, Marisa Chau, Emma J. Tavender, Russell L. Gruen & Jeffrey V. Rosenfeld

  3. Faculty of Economics and Business, University of Groningen, Groningen, The Netherlands

    Marije Bosch

  4. School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia

    Joanne E. McKenzie, Simon Turner, Sue E. Brennan, Denise A. O’Connor & Sally E. Green

  5. Monash-Epworth Rehabilitation Research Centre, Epworth Hospital, Melbourne, Australia

    Jennie L. Ponsford

  6. School of Psychological Sciences, Monash University, Melbourne, Australia

    Jennie L. Ponsford

  7. Melbourne Medical School, The University of Melbourne, Melbourne, Australia

    Jonathan C. Knott

  8. Department of Emergency Medicine, Royal Melbourne Hospital, Melbourne, Australia

    Jonathan C. Knott

  9. Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore

    Russell L. Gruen

  10. School of Health Sciences, City University of London, London, UK

    Jill J. Francis

  11. MedSTAR Emergency Medical Retrieval Service, Adelaide, Australia

    Andrew Pearce

  12. Royal Adelaide Hospital Emergency Department, Adelaide, Australia

    Andrew Pearce

  13. Centre for Health Economics, Monash Business School, Monash University, Melbourne, Australia

    Duncan Mortimer

  14. Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada

    Jeremy M. Grimshaw

  15. Department of Medicine, University of Ottawa, Ottawa, ON, Canada

    Jeremy M. Grimshaw

  16. Department of Neurosurgery, The Alfred Hospital, Melbourne, Australia

    Jeffrey V. Rosenfeld

  17. Department of Psychology, Macquarie University, Sydney, Australia

    Susanne Meares

  18. Emergency Department, Westmead Hospital, Sydney, Australia

    Tracy Smyth

  19. Department of Clinical, Educational and Health Psychology, University College London, London, UK

    Susan Michie

Authors
  1. Marije Bosch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joanne E. McKenzie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennie L. Ponsford
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Simon Turner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marisa Chau
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Emma J. Tavender
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jonathan C. Knott
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Russell L. Gruen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jill J. Francis
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sue E. Brennan
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Andrew Pearce
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Denise A. O’Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Duncan Mortimer
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Jeremy M. Grimshaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Jeffrey V. Rosenfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Susanne Meares
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Tracy Smyth
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Susan Michie
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Sally E. Green
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RLG and SEG were the lead investigators of the funding application, provided general oversight and input in the study design and contributed to revisions of the manuscript. MB (trial manager) co-led the design of the trial, designed the data collection instruments, contributed to data collection and data cleaning, contributed to the intervention design, wrote the first draft of the manuscript and prepared the revised versions. JEM (project statistician) co-led the design of the trial and was responsible for all statistical aspects, supervised the data cleaning and analyses, wrote the sections ‘randomisation and allocation concealment’, ‘sample size’, ‘effectiveness analysis’ and ‘deviations from study protocol’ file and contributed extensively to revisions of the manuscript. ST performed the data cleaning and analyses and contributed to revisions of the manuscript. DM (project health economist) was responsible for the design of the economic evaluation, wrote the ‘economic evaluation’ section and contributed to revisions of the manuscript. JJF, SEB, EJT, SMi, DAO, SMe and TS contributed to intervention design and contributed to revisions of the manuscript. MC contributed to the data collection and to the revisions of the manuscript. JLP contributed to the design of the intervention and the design of the patient follow-up study, supervised the patient follow-up study and contributed to revisions of the manuscript. JVR contributed to the design of the patient follow-up study and contributed to the revisions of the manuscript. JCK and AP contributed to the intervention design, piloting of data collection tools and contributed to the revisions of the manuscript. JMG provided input in the study design and critical review of drafts and contributed to the revisions. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Marije Bosch or Sally E. Green.

Ethics declarations

Ethics approval and consent to participate

The trial protocol was approved by the Alfred Health Human Research Ethics Committee (approval Number 398/12). Following recruitment, additional local ethics and research governance procedures were completed for each site. Consent procedures conformed with recent guidance regarding ethical issues in CRTs as described in the trial protocol [25].

Consent for publication

Not applicable

Competing interests

All authors declare they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Trial protocol. (PDF 1390 kb)

Additional file 2:

Deviations from study protocol. (PDF 282 kb)

Additional file 3:

CONSORT checklist. (PDF 301 kb)

Additional file 4:

Patient identification protocol. (PDF 205 kb)

Additional file 5:

Overview of NET-Trial intervention and rationale for selection of components. (PDF 447 kb)

Additional file 6:

Extra tables. ICCs and estimated effects of the intervention on clinical practice and patient outcomes, adjusting for the minimisation criteria only. (PDF 303 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bosch, M., McKenzie, J.E., Ponsford, J.L. et al. Evaluation of a targeted, theory-informed implementation intervention designed to increase uptake of emergency management recommendations regarding adult patients with mild traumatic brain injury: results of the NET cluster randomised trial. Implementation Sci 14, 4 (2019). https://doi.org/10.1186/s13012-018-0841-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-018-0841-7

Keywords

Implementation Science

ISSN: 1748-5908