Skip to main content

Table 1 Characteristics of measures of sustainability and sustainability determinants and overall psychometric and pragmatic quality scores

From: Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review

Measure name

Alternate name

Designed explicitly for  Sustainability

Other constructs covered

Theory/framework

Target population involved in development

Setting/s developed and validated in

Country validated in and languages available

EBI and setting designed to assessd

User perspective

Total PAPERS’ scoresa

Measures of sustainment

 Provider Report of Sustainment Scale (PRESS) [32]

None

Yes

None

No

Yes

Substance use disorder treatment, child welfare, education, and specialty mental health

Country:

USA

Language:

English

General EBI and setting

Executives and frontline staff

35

 Stages of implementation completion (SIC) – Original [33,34,35]

None

No

Implementation process/stages

Unclear

Yes

Multiple

Country:

USA

Language:

English

General EBI and setting

Researcher/purveyor

14

Measures of sustainability determinants

 Assessment of Barriers to Implementation and Sustainability in Schools (ABISS) [36]

None

Yes

None

Framework proposed by the authors.

No

School

Country:

USA

Language

English

Specific EBI: School-wide Positive Behavior Support (SWPBS)

Specific setting: school

Executive staff, frontline staff, and external personnel who have knowledge of a school's SWPBS systems.

26

 Advanced Level Tier Interventions Treatment Utilization and Durability (ALTITUDE) [37]

None

Yes

None

No

Yes

School

Country:

USA

Language:

English

Specific EBI: School-wide Positive Behavior Support (SWPBS) implementation at Tiers 1, 2, and 3

Specific setting: schools

Executive staff, frontline staff, and external personnel who have knowledge of a school's SWPBS systems.

21

 A measurement instrument for sustainability of work practices in in long-term care - long version [38]

None

Yes

None

Framework proposed by the authors.

Yes

Nursing homes, elderly homes, home care and care for disabled

Country:

Netherlands

Language:

Dutch

English

General EBI and setting

Executive and frontline staff

26

 A measurement instrument for sustainability of work practices in in long-term care - short version [38]

None

Yes

None

Framework proposed by the authors.

Yes

Nursing homes, elderly homes, home care and care for disabled

Country:

Netherlands

Language:

Dutch

English

General EBI and setting

Executive and frontline staff

22

 Change Process Capability Questionnaire (CPCQ) [39]

None

No

Change management

Framework proposed by the authors on organisational change

Yes

Medical groups treating patients with depression

Country:

USA

Language:

English

Specific EBI: Depression care

Specific: Clinical/healthcare setting

Executive staff

19

 Clinical Sustainability Assessment Tool (CSAT) [40]

None

Yes

None

Unclear

Yes

Multiple clinical settings

Country:

USA

Language:

English

General EBI

Specific setting: Clinical/healthcare setting

Executive and frontline staff

28

 Faith-Based Organization Health Integration Inventory (FBO-HII) [41]

None

Yes

None

No

Yes

African American Churches

Country:

USA

Language:

English

Specific EBI: Health promotion activities

Specific setting: Faith based organisations

Executive staff

17

 General Organizational Index (GOI) [42, 43]

None

No

Quality of implementation.

No

Yes

Community mental health centers

Country:

Norway

USA

Language:

English

Norwegian

General EBI and setting

Researcher/purveyor

24

 Levels of Institutionalization (Loln) [44, 45]

None

Yes

None

Level of Institutionalization Matrix framework [46]

Yes

Organisations that operate health promotion programs - public schools, county health departments, and non-profit health agencies

Hospitals and home health agencies

Country:

USA

Language:

English

Specific EBI:

Health promotion programs

General setting

Executive staff

25

 National Health Service (NHS) Sustainability Model and Guide [47]

The National Health Service (NHS) Institute for Innovation and Improvement Sustainability Model

Yes

None

A broader sustainability model proposed by the authors

Yes

Health care settings

Country:

UK

Language:

English

General EBI

Specific setting: clinical settings

Executive and frontline staff

17

 The Normalisation Measure Development questionnaire (NoMAD) [48,49,50,51,52,53]

None

Yes

None

Normalization Process Theory [54]

Yes

Multiple clinical and community settings

Country:

UK

USA

Netherlands

Canada

Sweden

Brazil

Language:

Albanian

Danish

Dutch

English

French

German

Italian

Norwegian

Portuguese

Spanish

Swedish

General EBI and setting

Executive and frontline staff

22b

 New South Wales Sustainability Checklist [55]

None

Yes

None

None

Unclear

Health promotion programs

Country:

Australia

Language:

English

Specific EBI: Health promotion workers

General setting

Executive and frontline staff

16

 Office of Adolescent Health (OAH) Sustainability Assessment [56]c

None

Yes

None

The OPA Framework for Program Sustainability

Unclear

Adolescent health promotion and disease prevention

Country:

USA

Language:

English

General EBI and setting

Executive and frontline staff

16

 Office of Population Affairs (OPA) Sustainability Assessment Tool [57]c

None

Yes

None

The OPA Framework for Program Sustainability

Unclear

Adolescent health promotion and disease prevention

Country:

USA

Language:

English

General EBI and setting

Executive and frontline staff

18

 Prevention Program Assessment [58]

None

No

Organizational readiness for practice or program

Multiple theories

Yes

Public health departments, community organizations

Country:

USA

Language:

English

Specific EBI: Chronic disease prevention programs

General setting

Executive and frontline staff

27

 Program Sustainability Assessment Tool (PSAT) [10]

None

Yes

None

Public Health Program Capacity for Sustainability framework [13]

Yes

State and community settings

Country:

USA

Language:

English

Specific EBI: public health practice and programs

General setting

Executive and frontline staff

26

 Program Sustainability Assessment Tool (PSAT) – adapted for elementary setting [59]

None

Yes

None

Public Health Program Capacity for Sustainability framework [13]

Yes

School

Country:

Australia

Language:

English

Specific EBI: public health practice and programs

Specific setting: school

Executive and frontline staff

19

 Program Sustainability Index [60]

None

Yes

None

Framework of sustainability proposed by the original authors.

Yes

Community settings

Country:

USA

Language:

English

Specific EBI: Community-based programs

General setting

Frontline staff

23

 School-wide Universal Behavior Sustainability Index- School Teams (SUBSIST) [61,62,63,64,65]

None

Yes

None

Model of sustainability of school-based practices proposed by the original authors.

No

School

Country:

USA

Language:

English

Specific EBI: School-wide Positive Behavior Support (SWPBS) implementation at Tiers 1, 2, and 3

Specific setting: schools

Executive staff, frontline staff and external personnel who have knowledge of a school's SWPBS systems.

29

 Sustainability Formative Self-Assessment Tool [66]

None

Yes

None

The Georgia Health Policy Center Sustainability Framework

Unclear

Community rural health settings

Country:

USA

Language:

English

General EBI and setting

Executive and frontline staff

15

 Sustainable Implementation Scale (SIS) [67]

None

Yes

None

A range of frameworks covered by current systematic reviews.

No

National sites

Country: Sweden

Language:

English

Swedish

Specific EBI: Community mental

health services

General setting

Researcher/purveyor

23

 Sustained Implementation Support Scale [68]

None

Yes

None

No

Yes

Mix

Country:

Australia,

Barbados,

Belgium, Canada,

Germany,

Ireland,

Netherlands,

New Zealand, Romania,

UK,

United Arab Emirates,

USA

Language:

English

General EBI and setting

Executive and frontline staff

23

 Sustaining Innovation Through Education (SITE): Extended Behavioral [69]

None

Yes

None

Author-derived framework

Yes

Schools

Country

Australia

Language

English

General EBI

Specific setting: schools

Executive and frontline staff

18

Sustaining  Innovation Through Education (SITE): short behavioral [69]

None

Yes

None

Author-derived framework

Yes

Schools

Country

Australia

Language

English

General EBI 

Specific setting: schools

Executive and frontline staff

19

 Sustainment Leadership Scale [70]

None

Yes

None

Exploration, Preparation, Implementation, Sustainment (EPIS) framework [71]

Unclear

Community-based organizations

Country:

USA

Language:

English

General EBI and setting

Frontline staff

24

 Sustainment Measurement System Scale (SMSS) [72]*

None

Yes

None

Consolidated Framework for Implementation Research [73]

Yes

Substance Abuse and Mental Health Services

Country:

USA

Language:

English

Specific EBI: prevention programs

General setting

Executive and frontline staff

28

  1. *Includes a sub-scale that assesses sustainability as an outcome and seven sub-scales assessing determinants of sustainability
  2. a Higher total PAPERS score reflects a higher level of psychometric and pragmatic quality, with the highest possible score being 56
  3. bTotal psychometric PAPERS rating for the Portuguese version of the NoMAD is 2, for the Swedish version (S-NoMAD) is 7, and for the Dutch version is 6, out of a possible 36. As the psychometric properties of translated measures may differ, the PAPERS ratings for these three translated versions of the NoMAD were scored separately
  4. cThe Office of Adolescent Health (OAH) and Office of Population Affairs (OPA) amalgamated. However, both measures developed by these divisions are still publicly available and contain different content, thus were assessed separately
  5. dThis is based on the intended focus of the measure and may differ to how the measure was originally developed and evaluated, which may reflect a more narrow focus than intended