The Minimum Quality Criteria Set (QI-MQCS) for critical appraisal: advancing the science of quality improvement
Implementation Science volume 10, Article number: A19 (2015)
Effective learning across related scientific investigations through evidence synthesis is critical to promoting evidence-based approaches to healthcare. Synthesis of findings from quality improvement intervention (QII) publications, however, poses challenges. We aimed to develop a critical appraisal instrument (the Minimum Quality Criteria Set or QI-MQCS) to promote identification, dissemination and implementation of findings from high quality QII evaluations.
We convened a 9 person expert panel to guide QII evidence synthesis methods development through a one year iterative telephone, survey and in-person panel process. We developed and empirically tested electronic search and screening methods for identifying QII publications, and a critical appraisal instrument. Finally, we iteratively tested and improved QI-MQCS psychometric properties based on review of 54 electronically searched and systematically screened QII articles.
Panelists agreed QI-MQCS should focus on QII specific domains, not evaluation design criteria. The 16 QI-MQCS domains address Organizational Motivation, Intervention Rationale, Intervention Description, Organizational Characteristics, Implementation, Study Design, Comparator Description, Data Sources, Timing, Adherence / Fidelity, Health Outcomes, Organizational Readiness, Penetration / Reach, Sustainability, Spread, and Limitations. The median inter-rater agreement for QI-MQCS items was kappa 0.57 (83% agreement). Items discriminated between studies in terms of quality (median criteria met 67%). Internal consistency measures indicated coherence without excessive conceptual overlap (Cronbach's alpha = 0.60, absolute mean inter-item correlation = 0.19). The critical appraisal instrument is accompanied by a user manual detailing What to consider, Where to look, and How to rate.
The QI-MQCS had acceptable psychometric properties for critical appraisal, and can support systematic review of diverse QII evaluations. It is a ready-to-use critical appraisal tool accompanied by a user manual and empirically tested forms and methods.
About this article
Cite this article
Rubenstein, L.V., Hempel, S., Liu, J.L. et al. The Minimum Quality Criteria Set (QI-MQCS) for critical appraisal: advancing the science of quality improvement. Implementation Sci 10 (Suppl 1), A19 (2015). https://doi.org/10.1186/1748-5908-10-S1-A19
- Psychometric Property
- Evidence Synthesis
- Organizational Readiness
- Quality Improvement Intervention
- Intervention Description