- Commentary
- Open access
- Published:
The use of segmented regression in analysing interrupted time series studies: an example in pre-hospital ambulance care
Implementation Science volume 9, Article number: 77 (2014)
Abstract
Background
An interrupted time series design is a powerful quasi-experimental approach for evaluating effects of interventions introduced at a specific point in time. To utilize the strength of this design, a modification to standard regression analysis, such as segmented regression, is required. In segmented regression analysis, the change in intercept and/or slope from pre- to post-intervention is estimated and used to test causal hypotheses about the intervention. We illustrate segmented regression using data from a previously published study that evaluated the effectiveness of a collaborative intervention to improve quality in pre-hospital ambulance care for acute myocardial infarction (AMI) and stroke. In the original analysis, a standard regression model was used with time as a continuous variable. We contrast the results from this standard regression analysis with those from segmented regression analysis. We discuss the limitations of the former and advantages of the latter, as well as the challenges of using segmented regression in analysing complex quality improvement interventions.
Findings
Based on the estimated change in intercept and slope from pre- to post-intervention using segmented regression, we found insufficient evidence of a statistically significant effect on quality of care for stroke, although potential clinically important effects for AMI cannot be ruled out.
Conclusions
Segmented regression analysis is the recommended approach for analysing data from an interrupted time series study. Several modifications to the basic segmented regression analysis approach are available to deal with challenges arising in the evaluation of complex quality improvement interventions.
Background
An Interrupted Time Series (ITS) study is a powerful quasi-experimental design for evaluating effects of interventions when random assignment is not feasible [1]. In an ITS study, a series of observations on the same outcome before and after the introduction of an intervention are used to test immediate and gradual effects of the intervention. A major strength of this design is its ability to distinguish the effect of the intervention from secular change, that is, change that would have happened even in the absence of the intervention. Estimating the intervention effect is done by comparing the trend in the outcome after the intervention to the existing trend in the pre-intervention period, and is achieved through modifications to the standard regression analysis. In a basic segmented regression analysis [2–4], the time period is divided into pre- and post-intervention segments, and separate intercepts and slopes are estimated in each segment. Statistical tests of changes in intercepts and slopes pre- to post-intervention are carried out. By making a few simple changes to the data set-up and model specification, segmented regression analysis can easily be implemented in standard statistical software packages. An additional adjustment is usually required to account for serial autocorrelation, which arises because observations taken over time are usually correlated. Technical details for data preparation, model specification, and adjustment for autocorrelation are presented elsewhere [2, 4]. Several examples of the use of segmented regression analyses in studies of quality improvement interventions have been published [5–7].
Illustration
To illustrate the segmented regression analysis approach, we analysed data from a previously published study [8] that used an ITS design to evaluate the effectiveness of a collaborative intervention to improve quality in pre-hospital ambulance care for acute myocardial infarction (AMI) and stroke at 11 publicly funded ambulance organizations in England. A series of weekly measurements (the percentage of patients with a pre-hospital diagnosis of AMI and stroke who received a defined care bundle) between January 2010 and February 2012 was used to measure the impact of the collaborative intervention. The six-month pre-intervention period was defined as January to June 2010. The authors used logistic regression analysis of the data at each site, with the outcome being delivery of the care bundle and the predictor being time, modeled in two ways: first as a continuous variable across the entire study period, and then as a dichotomous indicator representing pre- and post-intervention periods. The estimated odds ratios (ORs) from the models that specified time as a continuous variable were combined across sites using fixed effects meta-analysis. The authors concluded that, over all sites, the collaborative intervention led to statistically significant improvements in ambulance care for AMI (OR 1.04 per month, 95% Confidence Interval [CI] 1.04, 1.04) and stroke (OR 1.06, 95% CI 1.05, 1.07).
We used the plots of weekly data aggregated across sites provided in the additional file [9] to re-analyse the data using segmented regression analysis. The details of our results are presented in Table 1 and displayed graphically in Figures 1 and 2. Before the intervention, the increase in AMI performance was OR = 1.02 per month; after the intervention, there was an additional increase of OR = 1.04 (95% CI 0.98 to 1.10) per month, which was not statistically significantly different from the pre-intervention trend (p = 0.20). Over the entire 86-week intervention period, the estimated increase in AMI performance is given by OR = 3.2 (a 220% relative increase); in the absence of the intervention, we would have expected a 45% increase (OR = 1.45). After accounting for the secular trend, the additional improvement associated with the intervention would, if it were real, likely be clinically important. For stroke, the pre-intervention increase in performance was OR = 1.05 per month; after the intervention, there was an additional effect of OR = 1.02 (95% CI 0.97 to 1.07) per month, which was not statistically significantly different from the pre-intervention trend (p = 0.52). Over the entire 86-week intervention period, the estimated increase in stroke performance is given by OR = 4 (that is, a 300% increase in odds). But even if the intervention had not been introduced, we would have expected a relative increase of 190% (OR = 2.9).
Discussion
A challenge in the use of the ITS design to evaluate complex quality improvement interventions is that the intervention may consist of several different components introduced at different times. For example, in this study, the authors’ preliminary investigations suggested that the two most effective interventions may be provider prompts and individualized feedback, while education and passive information dissemination did not appear to bring about change. One way to estimate the effects of different intervention components is to add multiple 'interruptions’ to the time series, but this requires a sufficient number of time points between interventions to allow their independent effects to be estimated [4]. In some studies, the intervention may need to be phased in or introduced gradually over a period of time, before being fully implemented. Thus, there may be a time lag from the initial introduction of an intervention to when its full effects can be observed. One way to allow for this in the segmented regression analysis is to fit a model with three segments, corresponding to the pre-implementation, implementation, and post-implementation periods. An alternative is to fit the model excluding the time points corresponding to the phase-in period [4, 10]. Although the authors did not specify a phase-in period in their original analyses, we explored the effect of allowing for a phased introduction by censoring the first three months of observation after the start of the intervention. The results from this analysis are presented in Table 2, and our conclusions about the effect of the collaborative intervention remain unchanged.
In ITS studies, it is not uncommon to have different participating sites contributing data. In our re-analyses of these data, we used a single time series of data aggregated across all sites. An analysis of aggregated data is likely to have less power than a multilevel logistic regression analysis of the time series from the individual sites. Gebski et al. [3] describe how to conduct segmented regression analysis when there are multiple sites and different intervention start times. One approach is to conduct separate segmented regression analyses at each site, and then estimate the overall effect by pooling the estimates of intervention effect across sites using inverse variance weights in a meta-analytical model [11]. Another approach is to fit a single model to the data from all sites and account for heterogeneity across sites by incorporating random effects for the sites.
Sensitivity analyses may be conducted to consider the effect of outliers on the results. Outliers may be censored from the analyses or modeled explicitly via dummy variables [2]. Alternatively, the effect of outliers may be reduced by using moving averages. Yet another approach is to combine data points prior to analysis, by using, for example, bi-weekly instead of weekly measurements. Although we did not explore these issues in our re-analyses, variability in outcomes over time can substantially affect power in an ITS study.
Conclusion
In conclusion, our results demonstrate the importance of using segmented regression analysis in an ITS study. When a standard regression analysis is used with time modeled as a single continuous variable, an estimate is obtained for the slope over time, but it is impossible to distinguish the effect of the intervention from the underlying secular trend and to make causal claims about the effects of the intervention. Based on our re-analyses of these data, we conclude that the quality improvement collaborative resulted in no statistically significant improvements in the quality of AMI and stroke care, but that potential clinically important effects for AMI cannot be ruled out.
References
Shadish WR, Cook TD, Campbell DT: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. 2002, Boston: Houghton Mifflin
Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D: Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002, 27 (4): 299-309. 10.1046/j.1365-2710.2002.00430.x.
Gebski V, Ellingson K, Edwards J, Jernigan J, Kleinbaum D: Modelling interrupted time series to evaluate prevention and control of infection in healthcare. Epidemiol Infect. 2012, 140 (12): 2131-2141. 10.1017/S0950268812000179.
Penfold RB, Zhang F: Use of interrupted time series analysis in evaluating health care quality improvements. Academic Pediatrics. 2013, 13: S38-S44. 10.1016/j.acap.2013.08.002.
Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE: Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. Int J Tech Assess Health Care. 2003, 19 (4): 613-623.
Ansari F, Gray K, Nathwani D, Phillips G, Ogston S, Ramsay C, Davey P: Outcomes of an intervention to improve hospital antibiotic prescribing: interrupted time series with segmented regression analysis. J. Antimicrob. Chemother. 2003, 52 (5): 842-848. 10.1093/jac/dkg459.
Foy R, Ramsay CR, Grimshaw JM, Penney GC, Vale L, Thomson A, Greer IA: The impact of guidelines on mild hypertension in pregnancy: time series analysis. BJOG. 2004, 111 (8): 765-770. 10.1111/j.1471-0528.2004.00212.x.
Siriwardena AN, Shaw D, Essam N, Togher FJ, Davy Z, Spaight A, Dewey M, ASCQI Core Group: The effect of a national quality improvement collaborative on prehospital care for acute myocardial infarction and stroke in England. Implement Sci. 2014, 9 (1): 17-10.1186/1748-5908-9-17.
Implementation Science, Additional files.http://www.implementationscience.com/content/9/1/17/additional,
Serumaga B, Ross-Degnan D, Avery AJ, Elliott RA, Majumdar SR, Zhang F, Soumerai SB: Effect of pay for performance on the management and outcomes of hypertension in the United Kingdom: interrupted time series study. BMJ. 2011, 342: d108-10.1136/bmj.d108.
Normand SL: Tutorial in biostatistics: Meta-analysis: formulating, evaluating, combining, and reporting. Stat Med. 1999, 18 (3): 321-359. 10.1002/(SICI)1097-0258(19990215)18:3<321::AID-SIM28>3.0.CO;2-P.
Acknowledgements
We would like to thank the referees, especially Dr. Robert Penfold, for valuable suggestions that have led to improvements to this manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
JMG conceived of this commentary, participated in discussions, and helped to draft the manuscript. MT participated in discussions, carried out the statistical analyses, and wrote the first draft of the manuscript. JEM participated in discussions and helped to draft the manuscript. CRR participated in discussions and helped to draft the manuscript. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Taljaard, M., McKenzie, J.E., Ramsay, C.R. et al. The use of segmented regression in analysing interrupted time series studies: an example in pre-hospital ambulance care. Implementation Sci 9, 77 (2014). https://doi.org/10.1186/1748-5908-9-77
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1748-5908-9-77
Comments
View archived comments (1)