Skip to main content

Table 1 Summary of methodological quality, psychometric quality and usability, ranked by global COSMIN reliability score across implementation outcome

From: Implementation outcome instruments for use in physical healthcare settings: a systematic review

Reference

Implementation outcome

Name of measurement instrument or instrument description

COSMIN

Usability* (number of items)

ConPsy score (/22)

Reliability

Validity

Responsiveness

 

Acceptability (number of instruments = 33): the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable or satisfactory. Commonly used terms: satisfaction with various aspects of the innovation (e.g. content, complexity, comfort, delivery and credibility)

Shaw et al. [23]

The Mind the Gap Scale—adolescent version

Excellent

Excellent

Not assessed

Good (22)

7

The Mind the Gap Scale—parent version

Excellent

Excellent

Not assessed

Good (27)

7

Dow et al. [24]

The Person-Centred Health Care for Older Adults (PCHCOA) Survey

Excellent

Excellent

Not assessed

Good (31)

3

Dykes et al. [25]

The Impact of Health Information Technology (I-HIT) Scale

Excellent

Good

Not assessed

Good (29)

6

Brehaut et al. [26]

Ottawa acceptability of decision rules instrument (OADRI)

Excellent

Poor

Not assessed

Good (12)

3

Tomotaki et al. [27]

Evidence-Based Practice Questionnaire (EBPQ-J)—Japanese version

Good

Fair

Not assessed

Good (18)

4

Upton et al. [28]

Evidence-Based Practice Questionnaire (EBPQ)

Fair

Fair

Not assessed

Good (24)

7

Bhor et Mason [29]

A scale to assess attitudes of healthcare administrators towards the use of e-mail communication between patients and physicians

Fair

Fair

Not assessed

Good (20)

4

Phansalkar et al. [30]

Instrument for assessing clinicians’ perceptions about use of computerised protocols

Fair

Fair

Not assessed

Good (35)

3

Oliveira et al. [31]

CARDIOSATIS-Team scale

Fair

Poor

Not assessed

Good (11)

6

Wu et al. [32]

Healthcare professionals’ intention to use an adverse event reporting system

Fair

Poor

Not assessed

Good (17)

6

Melas et al. [33]

The Evidence-Based Practice Attitude Scale (EBPAS)—Greek version

Fair

Poor

Not assessed

Good (15)

5

Brouwers et al. [34]

Practitioner Feedback Questionnaire

Fair

Poor

Not assessed

Good (18)

4

Baker et al. [35]

The Attitudes Related to Trauma-Informed Care (ARTIC-45)

Fair

Poor

Not assessed

Good (45)

4

The Attitudes Related to Trauma-Informed Care (ARTIC-35) Scale

Fair

Poor

Not assessed

Good (35)

4

The Attitudes Related to Trauma-Informed Care (ARTIC-10) Scale—short version

Poor

Poor

Not assessed

Good (10)

2

Vanneste et al. [36]

A survey measuring acceptance of BelRAI, a web-based system enabling person-centred recording and data sharing across care settings

Fair

Poor

Not assessed

Good (31)

2

Bakas et al. [37]

A rating form measuring the satisfaction of the Telephone Assessment and Skill-Building Kit (TASK) intervention

Poor

Excellent

Not assessed

Excellent (9)

3

McConnell et al. [38]

Diffusion of Innovation in Long-Term Care (DOI-LTC) measurement battery-version for certified nursing assistants

Poor

Excellent

Not assessed

Good (40)

2

Diffusion of Innovation in Long-Term Care (DOI-LTC) measurement battery-version for licensed nurses

Poor

Excellent

Not assessed

Adequate (50)

2

Atkinson [39]

A Questionnaire to Measure Perceived Attributes of eHealth Innovations

Poor

Good

Not assessed

Good (24)

4

Gagnon et al. [40]

A questionnaire based on the Technology Acceptance Model (TAM)

Poor

Fair

Not assessed

Good (33)

3

Ferrando et al. [41]

A questionnaire to measure convenience and satisfaction with a new internet-based tool for oral anticoagulation therapy telecontrol

Poor

Fair

Not assessed

Good (10)

2

Wilkinson et al. [42]

A survey measuring attitudes towards biomedical HIV prevention

Poor

Fair

Not assessed

Good (14)

2

Adu et al. [43]

A questionnaire measuring pharmacists and physician’s attitudes to antibiotic policies

Poor

Poor

Poor

Good (12)

2

Abetz et al. [44]

Cancer Therapy Satisfaction Questionnaire (CTSQ)

Poor

Poor

Poor

Good (21)

2

Blumenthal et al. [45]

Physiotherapy Mobile Acceptance Questionnaire (PTMAQ)

Poor

Poor

Not assessed

Good (30)

9

Weiner et al. [46]**

Acceptability of Intervention Measure (AIM)

Poor

Poor

Not assessed

Excellent (4)

8

Unni et al. [47]

A survey measuring satisfaction with electronic health records

Poor

Poor

Not assessed

Good (21)

6

Aggelidis et al. [48]

End user computing satisfaction (EUCS) survey

Poor

Poor

Not assessed

Good (49)

6

El-Den et al. [49]

Perinatal Depression (PND) Attitudes and Screening Acceptability Questionnaire (PASAQ)

Poor

Poor

Not assessed

Good (24)

5

Kramer et al. [50]

A generic questionnaire to detect physicians’ willingness to implement complex medical interventions

Poor

Poor

Not assessed

Good (41)

5

Frandes et al. [51]

An instrument assessing mobile technology acceptability in diabetes self-management

Poor

Poor

Not assessed

Good (29)

4

Rasoulzadeh et al. [52]

A questionnaire measuring acceptance of creating a nurses’ health monitoring system

Poor

Poor

Not assessed

Good (12)

3

Sockolow et al. [53]

Electronic Health Record Nurse Satisfaction (EHRNS) survey

Poor

Poor

Not assessed

Good (21)

3

Johnston et al. [54]

A questionnaire assessing physicians’ attitudes towards the computerisation of clinical practice

Poor

Poor

Not assessed

Good (20)

2

Bernhardsson et al. [55]

Evidence-Based Practice (EBP) Questionnaire—Swedish version

Poor

Poor

Not assessed

Good (31)

2

Yildiz et al. [56]

Evidence-Based Practice Attitude Scale (EBPAS-50)—Turkish version

Poor

Poor

Not assessed

Adequate (50)

0

Bevier et al. [57]

Questionnaire of three scoring items for current treatment satisfaction and factors of both clinical trial participation motivations and technology acceptance model

Poor

Not assessed

Not assessed

Good (34)

0

Wolf et al. [58]

Evidence-Based Practice Attitude Scale (EBPAS)

Not assessed

Excellent

Not assessed

Good (15)

5

Steed et al. [59]

Acceptability of Continuous Glucose Monitoring Devices (ACGMD) questionnaire

Not assessed

Fair

Not assessed

Adequate (64)

2

 

Appropriateness (number of instruments = 7): the perceived fit, relevance or compatibility of the innovation or evidence-based practice for a given practice setting, provider or consumer and/or perceived fit of the innovation to address a particular issue or problem. Commonly used terms: perceived fit, relevance, compatibility, suitability, usefulness, and practicability

Diego et al. [60]

A questionnaire to measure the attitude of anesthesiologists and residents regarding the use of the checklist in the perioperative period

Fair

Good

Not assessed

Excellent (7)

2

Park et al. [61]

A questionnaire measuring motivational factors for using wearable healthcare devices

Fair

Fair

Not assessed

Good (28)

8

Razmak et al. [62]

A techno-humanist model for e-health adoption of innovative technology

Fair

Fair

Not assessed

Good (32)

1

Joice et al. [63]

Perceived usefulness of a stroke workbook-based intervention measure

Poor

Fair

Not assessed

Good (15)

6

Xiao et al. [64]

Baylor EHR UX survey

Poor

Fair

Not assessed

Good (29)

3

Weiner et al. [46]**

Intervention Appropriateness Measure (IAM)

Poor

Poor

Not assessed

Excellent (4)

8

King et al. [65]

The Portal Survey on Satisfaction and Impact on Care

Poor

Poor

Not assessed

Good (38)

5

 

Adoption (number of instruments = 4): the intention, initial decision or action to try or employ an innovation or evidence-based practice. Commonly used terms: uptake, utilisation, initial implementation and intention to try

Nydegger et al. [66]

Strength of Implementation Intentions Scale (SIIS) for condom use

Fair

Fair

Not assessed

Good (22)

4

Everson et al. [67]

American Hospital Association IT (AHA-IT) Supplement Survey

Fair

Poor

Not assessed

Good (28)

6

Malo et al. [68]

A questionnaire evaluating nurses’ intention to use an electronic medical charting system

Poor

Poor

Not assessed

Good (46)

3

Kaltenbrunner et al. [69]

Lean in Healthcare Questionnaire (LiHcQ)

Unable to score***

Fair

Not assessed

Good (16)

7

 

Feasibility (number of instruments = 4): the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting. Commonly used terms: actual fit or utility, suitability for everyday use and practicability

Garcia-Smith et al. [70]

Instrument to test the Clinical Information Systems Success Model (CISSM)

Excellent

Poor

Not assessed

Good (26)

2

Schnall et al. [71]

Technology Acceptance Survey

Fair

Poor

Not assessed

Excellent (8)

3

Windsor et al. (2013)

The Smoking Cessation and Reduction in Pregnancy Treatment (SCRIPT) Adoption Scale

Poor

Poor

Not assessed

Good (28)

4

Weiner et al. [46]**

Feasibility of Intervention Measure (FIM)

Poor

Poor

Not assessed

Excellent (4)

8

 

Penetration (number of instruments = 4): the integration of a practice within a service setting and its subsystems. Commonly used terms: level of institutionalisation, spread and service access

Grooten et al. [72]

The Scaling Integrated Care in Context (SCIROCCO) tool

Good

Good

Not assessed

Good (12)

5

Slaghuis et al. [73]

A measurement instrument for spread of quality improvement in healthcare

Fair

Poor

Not assessed

Good (18)

4

Flanagan et al. [74]

The Prevention and Control of Antimicrobial resistance (PACAR) scale

Poor

Fair

Not assessed

Good (16)

6

Jaana et al. [75]

A measure of clinical information technology sophistication in hospitals

Poor

Not assessed

Not assessed

Adequate (68)

3

 

Sustainability (number of instruments = 3): the extent to which a newly implemented treatment is maintained or institutionalised within a service setting’s ongoing, stable operations. Commonly used terms: maintenance, continuation, durability, incorporation, integration, institutionalisation, sustained use and routinisation

Finch et al. [76]

Normalisation Measure Development Questionnaire (NoMAD)

Fair

Fair

Not assessed

Good (23)

7

Elf et al. [77]

Normalisation Measure Development Questionnaire (S-NoMAD)—Swedish version

Fair

Poor

Not assessed

Good (20)

7

Slaghuis et al. [78]

A measurement instrument for sustainability of work practices in long-term care—short version

Poor

Poor

Not assessed

Good (30)

7

A measurement instrument for sustainability of work practices in long-term care—long version

Poor

Poor

Not assessed

Good (40)

6

Barab et al. [79]

The Levels of Institutionalization (LoIn) scales

Poor

Poor

Not assessed

Good (30)

4

  1. *1. Minimal: > 100 items; 2. adequate: 50–99 items; 3. good: 10–49 items; 4. excellent: < 10 items (Lewis et al. [13])
  2. **One analysis was reported where the 3 scales were included in the same model
  3. ***Test-retest reliability procedure not reported therefore unable to score
  4. Underlined rows indicate instruments with multiple scales