Table 4 Characteristics of included references: Methodological approaches to QI selection, practice test, and implementation
From: Methods for the guideline-based development of quality indicators--a systematic review
| Â | QI selection | Additional QI development elements | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
Reference | Panel method | Criteria for panel members | Panel members listed a | Selected indicators listed a | Sources transparent 1 | LoE b | Rating criteria | Practice test | Implementation strategy | Patient participation |
Method papers | Â | Â | Â | Â | Â | Â | Â | Â | Â | Â |
ÄZQ (2009) | Unclear | Unclear | - | - | - | Yes | Yes Importance for the healthcare system, clarity, improvability, risk for adverse effect, evidence base, grade of recommendation | Proposed | No | No |
AHCPR (1995) | No | No panel method | - | - | - | No | Unclear | Not mentioned | No | No |
AHRQ (1995) | No | No panel method | - | - | - | No | No | Included | Yes Development of data collection software, audit and feedback | No |
AQUA (2010) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise | - | - | - | Yes | Yes Relevance, clarity, feasibility | Included | Yes Development/upgrading of data collection software | QI selection |
Baker and Fraser (1995) | No | No panel method | - | - | - | No | Unclear | Not mentioned | Yes Local development, ownership | No |
Bergman (1999) | No | No panel method | - | - | - | Yes | Unclear | Proposed | Yes Involving key stakeholders | No |
Califf et al. (2002) | No | No panel method | - | - | - | Yes | Unclear | Not mentioned | Yes Education and feedback | No |
Campbell et al. (2002) | Other | Unclear | - | - | - | No | Unclear | Not mentioned | No | No |
Graham et al. (2009) | Other | No | - | - | - | No | Yes Grade of recommendation, level of evidence, measurability, improvability | Included | Yes Audit and feedback | No |
Spertus et al. (2005) | No | No panel method | - | - | - | No | Yes Useful in improving patient outcomes, measure design, measure implementation, overall assessment | Not mentioned | No | No |
Topic papers | Â | Â | Â | Â | Â | Â | Â | Â | Â | Â |
Bonow et al. (2005) | Other | No | Yes | Yes | Yes | Yes | Yes Useful in improving patient outcomes, measure design, measure implementation, overall assessment | Not mentioned | Yes Defining challenges to implementation for each QI | No |
Burge et al. (2007) | Modified RAND/UCLA | Yes Members of specialist societies | Yes | Yes | In part | No | No | Proposed | No | No |
Campbell et al. (1999) | Modified RAND/UCLA | Yes Clinical expertise, members of specialist societies | No | Yes | In part | Yes | No | Not mentioned | Yes | No |
Desch et al. (2008) | Other | Yes Members of specialist societies, methodological expertise | Yes | Yes | Yes | No | No | Not mentioned | Yes Integration in nationwide quality-improvement programs | No |
Draskovic et al. (2008) | Modified RAND/UCLA | Yes Clinical expertise | No | Yes | Yes | No | Yes Face validity | Included | Yes Including the informal caregivers' perspective | No |
Estes et al. (2008) | Other | No | Yes | Yes | Yes | Yes | Yes Useful to improve patient outcomes, measure design, measure implementation, overall assessment | Not mentioned | Yes Defining challenges to implementation for each QI | No |
Forbes et al. (1997) | No | No panel method | No panel method | Yes | Yes | No | No | Included | Yes Pilot testing | No |
Giesen et al. (2007) | Other | Unclear | No | Yes | In part | No | Yes Relevance, utility for evaluation of care | Included | No | No |
Hadorn et al. (1996) | Unclear | No | No | Yes | In part | No | Unclear | Not mentioned | No | No |
Hardy and Hadley (1995) | Unclear | Unclear | No | No | Yes | No | No | Not mentioned | No | No |
Hermanides et al. (2008) | Other | Yes Clinical expertise | Yes | Yes | Yes | Yes | Yes Appropriateness | Included | No | No |
Hermens et al. (2006) | Modified RAND/UCLA | Yes Clinical expertise | Yes | Yes | Yes | No | Yes Professional quality, organisational quality, patient-oriented quality | Included | Yes Practice test | QI selection |
James et al. (1997) | Other | Yes Clinical expertise | No | Yes | Yes | Yes | Yes Educational appropriateness, clinical importance, measurement feasibility | Not mentioned | No | No |
Kongnyuy and van den Broek (2008) | Other | Yes Clinical expertise, laypersons | No | Yes | In part | No | No | Planned | Yes Involving all grades of health professionals during the whole development process | QI selection |
Krumholz et al. (2006) | Other | Yes Clinical expertise, methodological expertise members of specialist societies | Yes | Yes | Yes | Yes | Yes Useful in improving patient outcomes, measure design, measure implementation, overall assessment | Not mentioned | Yes Defining challenges to implementation for each QI | No |
Lee et al. (2003) | Other | Yes Clinical expertise | Yes | Yes | In part | No | Yes Meaningfulness, usefulness, potential for improvement, impact on patient outcomes, feasibility of data collection | Not mentioned | No | No |
Maclean et al. (2004) | Modified RAND/UCLA | Yes linical expertise, methodological expertise members of specialist societies | Yes | Yes | No | Yes | Unclear | Not mentioned | No | No |
Martirosyan et al. (2008) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise members of specialist societies | No | Yes | In part | No | Unclear | Included | No | No |
Mourad et al. (2007) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise | No | Yes | Yes | Yes | Unclear | Proposed | Yes Practice test | No |
Nijkrake et al. (2009) | Other | Yes Clinical expertise, methodological expertise | No | No | Yes | Yes | Yes Relevance (effectiveness, efficiency, acceptability, measurability) | Included | Yes Training in the correct use of the respective guideline | No |
Ouwens et al. (2007) | Modified RAND/UCLA | Yes Clinical expertise | No | Yes | In part | No | Yes Clinically relevant to patients' health benefits and/or to the continuity and coordination of care | Included | Yes Practice test | QI selection |
Ouwens et al. (2010) | Other | Yes Patient representatives | No | Yes | In part | No | Unclear | Included | Yes Patient participation | QI selection |
Radtke et al. (2009) | Other | Yes Clinical expertise, methodological expertise, patients | No | Yes | In part | No | Yes Inclusion in the research literature, measurable under routine conditions, inclusion in a certain high-quality guideline, reproducibility, validity, clinical relevance, sensitivity to change | Included | No | No |
Redberg et al. (2009) | Other | Yes Clinical expertise, methodological expertise membership in specialist societies | Yes | Yes | Yes | Yes | Yes Useful in improving patient outcomes, measure design, measure implementation, overall assessment | Not mentioned | No | No |
Schouten et al. (2005) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise | No | Yes | Yes | Yes | Yes Clinical relevance to the patient's health benefit, relevance to reducing antimicrobial resistance, relevance to cost effectiveness | Included | No | No |
Sugarman et al. (2003) | Other | Yes Clinical expertise, membership in specialist societies | No | No | Yes | Yes | Yes Clinical importance, feasibility of measurement, level of evidence | Included | No | No |
Thomas et al. (2007) | Unclear | Yes Clinical expertise, methodological expertise, membership in specialist societies | Yes | Yes | Yes | Yes | Yes Evidence based, interpretable, actionable, clinically meaningful, valid, reliable, feasible | Not mentioned | Yes Defining challenges to implementation for each QI | No |
Tu et al. (2008) | Other | Yes Clinical expertise, methodological expertise, membership in specialist societies | Yes | Yes | In part | No | Yes Usefulness in improving patient outcomes, feasibility of data collection, reliability, validity | Not mentioned | Yes Pay for performance, collaboration with national and local initiatives, use of standard tools, presentation at scientific meetings, availability online | No |
van den Boogaard et al. (2010) | Modified RAND/UCLA | Yes Clinical expertise | Yes | Yes | Yes | Yes | Yes Health gain, overall efficacy | Proposed | No | No |
van Hulst et al. (2009) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise | No | Yes | In part | Yes | No | Not mentioned | Yes Using understandable and measurable QIs | No |
Wang et al. (2006) | Other | Yes Membership in specialist societies | No | Yes | In part | Yes | Yes Validity, feasibility | Not mentioned | No | No |
Yazdany et al. (2009) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise | Yes | Yes | No | Yes | Yes Evidence base, validity, feasibility | Proposed | Yes Assess the technical characteristics of developed QIs | No |
Method + topic papers | Â | Â | Â | Â | Â | Â | Â | Â | Â | Â |
Advani et al. (2003) | No | No panel method | No panel method | No | Yes | No | No | Included | No | No |
Duffy et al. (2005) | Unclear | Unclear | No | Yes | Yes | Yes | Unclear | Planned | Yes Integration in health plan performance measurement, quality monitoring and accreditation programs, integration of needed data elements in medical information systems | No |
Golden et al. (2008) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise, laypersons | No | No | In part | No | Yes Meaningfulness, quality gap, improvability, feasibility of data collection | Included | Yes Transparency during the development process, providing the data collection tool, submission to a national performance measurement program | QI selection |
Hutchinson et al. (2003) | Other | Yes Clinical expertise | No | Yes | In part | Yes | No | Not mentioned | No | No |
Laclair et al. (2001) | Other | Yes Clinical expertise, methodological expertise | No | No | Yes | Yes | No | Included | No | No |
Wollersheim et al. (2007) | Modified RAND/UCLA | Yes Clinical expertise, methodological expertise | No | Yes | In part | Unclear | No | Included | Yes Periodic audits | No |