Skip to main content

Table 1 Summary of methodological quality, psychometric quality and usability, ranked by global COSMIN reliability score across implementation outcome

From: Implementation outcome instruments for use in physical healthcare settings: a systematic review

Reference Implementation outcome
Name of measurement instrument or instrument description 		COSMIN Usability* (number of items) ConPsy score (/22)
Reliability Validity Responsiveness
  Acceptability (number of instruments = 33): the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable or satisfactory. Commonly used terms: satisfaction with various aspects of the innovation (e.g. content, complexity, comfort, delivery and credibility)
Shaw et al. [23] The Mind the Gap Scale—adolescent version Excellent Excellent Not assessed Good (22) 7
The Mind the Gap Scale—parent version Excellent Excellent Not assessed Good (27) 7
Dow et al. [24] The Person-Centred Health Care for Older Adults (PCHCOA) Survey Excellent Excellent Not assessed Good (31) 3
Dykes et al. [25] The Impact of Health Information Technology (I-HIT) Scale Excellent Good Not assessed Good (29) 6
Brehaut et al. [26] Ottawa acceptability of decision rules instrument (OADRI) Excellent Poor Not assessed Good (12) 3
Tomotaki et al. [27] Evidence-Based Practice Questionnaire (EBPQ-J)—Japanese version Good Fair Not assessed Good (18) 4
Upton et al. [28] Evidence-Based Practice Questionnaire (EBPQ) Fair Fair Not assessed Good (24) 7
Bhor et Mason [29] A scale to assess attitudes of healthcare administrators towards the use of e-mail communication between patients and physicians Fair Fair Not assessed Good (20) 4
Phansalkar et al. [30] Instrument for assessing clinicians’ perceptions about use of computerised protocols Fair Fair Not assessed Good (35) 3
Oliveira et al. [31] CARDIOSATIS-Team scale Fair Poor Not assessed Good (11) 6
Wu et al. [32] Healthcare professionals’ intention to use an adverse event reporting system Fair Poor Not assessed Good (17) 6
Melas et al. [33] The Evidence-Based Practice Attitude Scale (EBPAS)—Greek version Fair Poor Not assessed Good (15) 5
Brouwers et al. [34] Practitioner Feedback Questionnaire Fair Poor Not assessed Good (18) 4
Baker et al. [35] The Attitudes Related to Trauma-Informed Care (ARTIC-45) Fair Poor Not assessed Good (45) 4
The Attitudes Related to Trauma-Informed Care (ARTIC-35) Scale Fair Poor Not assessed Good (35) 4
The Attitudes Related to Trauma-Informed Care (ARTIC-10) Scale—short version Poor Poor Not assessed Good (10) 2
Vanneste et al. [36] A survey measuring acceptance of BelRAI, a web-based system enabling person-centred recording and data sharing across care settings Fair Poor Not assessed Good (31) 2
Bakas et al. [37] A rating form measuring the satisfaction of the Telephone Assessment and Skill-Building Kit (TASK) intervention Poor Excellent Not assessed Excellent (9) 3
McConnell et al. [38] Diffusion of Innovation in Long-Term Care (DOI-LTC) measurement battery-version for certified nursing assistants Poor Excellent Not assessed Good (40) 2
Diffusion of Innovation in Long-Term Care (DOI-LTC) measurement battery-version for licensed nurses Poor Excellent Not assessed Adequate (50) 2
Atkinson [39] A Questionnaire to Measure Perceived Attributes of eHealth Innovations Poor Good Not assessed Good (24) 4
Gagnon et al. [40] A questionnaire based on the Technology Acceptance Model (TAM) Poor Fair Not assessed Good (33) 3
Ferrando et al. [41] A questionnaire to measure convenience and satisfaction with a new internet-based tool for oral anticoagulation therapy telecontrol Poor Fair Not assessed Good (10) 2
Wilkinson et al. [42] A survey measuring attitudes towards biomedical HIV prevention Poor Fair Not assessed Good (14) 2
Adu et al. [43] A questionnaire measuring pharmacists and physician’s attitudes to antibiotic policies Poor Poor Poor Good (12) 2
Abetz et al. [44] Cancer Therapy Satisfaction Questionnaire (CTSQ) Poor Poor Poor Good (21) 2
Blumenthal et al. [45] Physiotherapy Mobile Acceptance Questionnaire (PTMAQ) Poor Poor Not assessed Good (30) 9
Weiner et al. [46]** Acceptability of Intervention Measure (AIM) Poor Poor Not assessed Excellent (4) 8
Unni et al. [47] A survey measuring satisfaction with electronic health records Poor Poor Not assessed Good (21) 6
Aggelidis et al. [48] End user computing satisfaction (EUCS) survey Poor Poor Not assessed Good (49) 6
El-Den et al. [49] Perinatal Depression (PND) Attitudes and Screening Acceptability Questionnaire (PASAQ) Poor Poor Not assessed Good (24) 5
Kramer et al. [50] A generic questionnaire to detect physicians’ willingness to implement complex medical interventions Poor Poor Not assessed Good (41) 5
Frandes et al. [51] An instrument assessing mobile technology acceptability in diabetes self-management Poor Poor Not assessed Good (29) 4
Rasoulzadeh et al. [52] A questionnaire measuring acceptance of creating a nurses’ health monitoring system Poor Poor Not assessed Good (12) 3
Sockolow et al. [53] Electronic Health Record Nurse Satisfaction (EHRNS) survey Poor Poor Not assessed Good (21) 3
Johnston et al. [54] A questionnaire assessing physicians’ attitudes towards the computerisation of clinical practice Poor Poor Not assessed Good (20) 2
Bernhardsson et al. [55] Evidence-Based Practice (EBP) Questionnaire—Swedish version Poor Poor Not assessed Good (31) 2
Yildiz et al. [56] Evidence-Based Practice Attitude Scale (EBPAS-50)—Turkish version Poor Poor Not assessed Adequate (50) 0
Bevier et al. [57] Questionnaire of three scoring items for current treatment satisfaction and factors of both clinical trial participation motivations and technology acceptance model Poor Not assessed Not assessed Good (34) 0
Wolf et al. [58] Evidence-Based Practice Attitude Scale (EBPAS) Not assessed Excellent Not assessed Good (15) 5
Steed et al. [59] Acceptability of Continuous Glucose Monitoring Devices (ACGMD) questionnaire Not assessed Fair Not assessed Adequate (64) 2
  Appropriateness (number of instruments = 7): the perceived fit, relevance or compatibility of the innovation or evidence-based practice for a given practice setting, provider or consumer and/or perceived fit of the innovation to address a particular issue or problem. Commonly used terms: perceived fit, relevance, compatibility, suitability, usefulness, and practicability
Diego et al. [60] A questionnaire to measure the attitude of anesthesiologists and residents regarding the use of the checklist in the perioperative period Fair Good Not assessed Excellent (7) 2
Park et al. [61] A questionnaire measuring motivational factors for using wearable healthcare devices Fair Fair Not assessed Good (28) 8
Razmak et al. [62] A techno-humanist model for e-health adoption of innovative technology Fair Fair Not assessed Good (32) 1
Joice et al. [63] Perceived usefulness of a stroke workbook-based intervention measure Poor Fair Not assessed Good (15) 6
Xiao et al. [64] Baylor EHR UX survey Poor Fair Not assessed Good (29) 3
Weiner et al. [46]** Intervention Appropriateness Measure (IAM) Poor Poor Not assessed Excellent (4) 8
King et al. [65] The Portal Survey on Satisfaction and Impact on Care Poor Poor Not assessed Good (38) 5
  Adoption (number of instruments = 4): the intention, initial decision or action to try or employ an innovation or evidence-based practice. Commonly used terms: uptake, utilisation, initial implementation and intention to try
Nydegger et al. [66] Strength of Implementation Intentions Scale (SIIS) for condom use Fair Fair Not assessed Good (22) 4
Everson et al. [67] American Hospital Association IT (AHA-IT) Supplement Survey Fair Poor Not assessed Good (28) 6
Malo et al. [68] A questionnaire evaluating nurses’ intention to use an electronic medical charting system Poor Poor Not assessed Good (46) 3
Kaltenbrunner et al. [69] Lean in Healthcare Questionnaire (LiHcQ) Unable to score*** Fair Not assessed Good (16) 7
  Feasibility (number of instruments = 4): the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting. Commonly used terms: actual fit or utility, suitability for everyday use and practicability
Garcia-Smith et al. [70] Instrument to test the Clinical Information Systems Success Model (CISSM) Excellent Poor Not assessed Good (26) 2
Schnall et al. [71] Technology Acceptance Survey Fair Poor Not assessed Excellent (8) 3
Windsor et al. (2013) The Smoking Cessation and Reduction in Pregnancy Treatment (SCRIPT) Adoption Scale Poor Poor Not assessed Good (28) 4
Weiner et al. [46]** Feasibility of Intervention Measure (FIM) Poor Poor Not assessed Excellent (4) 8
  Penetration (number of instruments = 4): the integration of a practice within a service setting and its subsystems. Commonly used terms: level of institutionalisation, spread and service access
Grooten et al. [72] The Scaling Integrated Care in Context (SCIROCCO) tool Good Good Not assessed Good (12) 5
Slaghuis et al. [73] A measurement instrument for spread of quality improvement in healthcare Fair Poor Not assessed Good (18) 4
Flanagan et al. [74] The Prevention and Control of Antimicrobial resistance (PACAR) scale Poor Fair Not assessed Good (16) 6
Jaana et al. [75] A measure of clinical information technology sophistication in hospitals Poor Not assessed Not assessed Adequate (68) 3
  Sustainability (number of instruments = 3): the extent to which a newly implemented treatment is maintained or institutionalised within a service setting’s ongoing, stable operations. Commonly used terms: maintenance, continuation, durability, incorporation, integration, institutionalisation, sustained use and routinisation
Finch et al. [76] Normalisation Measure Development Questionnaire (NoMAD) Fair Fair Not assessed Good (23) 7
Elf et al. [77] Normalisation Measure Development Questionnaire (S-NoMAD)—Swedish version Fair Poor Not assessed Good (20) 7
Slaghuis et al. [78] A measurement instrument for sustainability of work practices in long-term care—short version Poor Poor Not assessed Good (30) 7
A measurement instrument for sustainability of work practices in long-term care—long version Poor Poor Not assessed Good (40) 6
Barab et al. [79] The Levels of Institutionalization (LoIn) scales Poor Poor Not assessed Good (30) 4
  1. *1. Minimal: > 100 items; 2. adequate: 50–99 items; 3. good: 10–49 items; 4. excellent: < 10 items (Lewis et al. [13])
  2. **One analysis was reported where the 3 scales were included in the same model
  3. ***Test-retest reliability procedure not reported therefore unable to score
  4. Underlined rows indicate instruments with multiple scales
Back to article page