- Open Access
Time-series and risk-adjusted control charts
© Groves et al; licensee BioMed Central Ltd. 2013
- Published: 19 April 2013
- Propensity Score Match
- Risk Adjustment
- Statistical Process Control
- Quality Improvement Initiative
- Sequential Probability Ratio Test
There are several statistical process control (SPC) methods used in industry that can be applied in healthcare. However, as noted in the earlier discussion by Toulany et al. regarding quasi-experimental designs for quality improvement research, several considerations must be taken into account when adapting these methods for the complex, high-risk healthcare arena. Industrial methods should be adjusted for (a) heterogeneity at the patient level, including illness type, individualized care, and demographics, (b) heterogeneity at the process level, including geographical and longitudinal clinical care variation, (c) lack of pre-existing standards of comparison for new products or processes, and (d) the critical difference between statistical variation and acceptable clinical risk. Potential methods for successful adaptation of industrial SPC methods for healthcare monitoring and improvement include (a) converting periodic data into cumulative charts to increase detection of trends and (b) addressing heterogeneity through risk adjustment, using a prediction model or propensity score matching. These adjustments tend to inflate Type I errors, however, due to repeated measurements. Thus, the Sequential Probability Ratio Testing (SPRT) method may be of particular use . SPRT uses the more commonly available retrospective control data, accounts for repeated measurements, utilizes risk adjustment, and incorporates both alpha and beta error into the formal framework [2, 3]. The upper control limit is the desired odds ratio, as determined by the hypothesis.
Industrial SPC methods assume process homogeneity and that the outcome rate from the population establishes the threshold for detecting changes in the process. Using these techniques to analyze processes in healthcare often requires addressing the risk and complexities inherent in healthcare in order to obtain meaningful results. As with any data-driven project, the clinical question and limitations of the available data drive the selection of the patient cohort, SPC method, risk adjustment framework, alerting thresholds, and the interpretation of clinical significance. However, regardless of the SPC method used and the risk-adjustment framework, it is important to realize that performance of the risk adjustment model drives the overall result; thus understanding the strengths and weaknesses of each particular model is critical to clinical interpretations. In addition, detecting adverse outcomes over a long period requires recalibrating the model over time to adjust for systematic changes in clinical care. Finally, all signals detected using these methods require root cause analyses (RCA) and sensitivity analyses as they are hypothesis-generating, not confirming.
There are four key requirements and recommendations for making greater use of these SPC methods. First, additional infrastructure for collecting valid and reliable data is needed. The data required to drive these methods is facilitated by the structured data entry and collection from the EHR. This may include building basic EHR infrastructure; greater interoperability between EHRs; restructuring of EHRs, including templates for capturing specific variants; and greater use of natural language processing to capture potentially relevant details from free text and dictated notes. Second, collaboration between statisticians, registry experts, healthcare informaticists, and clinicians is required to address the complexity and heterogeneity inherent in data registries. The expertise of each is needed to extract, risk-adjust, and interpret the data required for answering the right question in an understandable way. Third, the limited amount of funds available to improve data infrastructure, support expertise, and collect and maintain the necessary data may necessitate restructuring of the payment system, potentially at both the state and national level. Because all payers, as well as patients and healthcare facilities, benefit from improved care, new methods of sharing savings and reforming payments could support advancement of these methods. Finally, as SPC methods continue to evolve and the infrastructure needed to support them grows, use of key techniques such as time-series and risk-adjusted control charts should be integrated into the clinical and graduate education of healthcare professionals, assuring their availability and accessibility for healthcare improvement.
- Li L, Kulldorff M: A conditional maximized sequential probability ratio test for pharmacovigilance. Stat Med. 2010, 29: 284-295.PubMedGoogle Scholar
- Matheny ME, Normand SL, Gross TP, Marinac-Dabic D, Loyo-Berrios N, Vidi VD, Donnelly S, Resnic FS: Evaluation of an automated safety surveillance system using risk adjusted sequential probability ratio testing. BMC Med Inform Decis Mak. 2011, 11: 75-10.1186/1472-6947-11-75.PubMed CentralView ArticlePubMedGoogle Scholar
- Spiegelhalter D, Grigg O, Kinsman R, Treasure T: Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care. 2003, 15: 7-13. 10.1093/intqhc/15.1.7.View ArticlePubMedGoogle Scholar
- Matheny ME, Arora N, Ohno-Machado L, Resnic FS: Rare adverse event monitoring of medical devices with the use of an automated surveillance tool. AMIA Annu Symp Proc. 2007, 518-22.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.