Skip to main content

Table 2 Operationalization and scoring criteria for RE-AIM components

From: Evaluating the implementation of the PACE Steps to Success Programme in long-term care facilities in seven countries according to the RE-AIM framework

Component1SourceTiming of collectionMeasureScoring criteria
ReachAttendance lists from training steps 1-6Month 3–18Mean attendance rate (number of care staff attending a training session divided by the total number staff working in the LTCF) on all 6 PACE steps< 30% = low
30–69% = medium
≥ 70% = high
Effectiveness1----
AdoptionReport from PACE coordinator at end of consolidation periodMonth 12Proportion of residents with Looking and Thinking Ahead document (number of residents with document completed divided by the number of PACE beds)< 40% = low
40–79% = medium
≥ 80% = high
ImplementationDiaries country trainersMonth 1–121. Score for fidelity (0-8 points)
(a) Number of training steps delivered (0-6 points, 1 point per step)
(b) If 6 training steps delivered: delivered in right order from 1 to 6? (yes = 1 point)
(c) If 6 training steps delivered: delivered within 8 months? (yes = 1 point)
Fidelity ≤ 4, satisfaction 0–8 = low
Fidelity > 4, satisfaction < 4 = Low
Fidelity ≥ 5, satisfaction 4–5.9 = medium
Fidelity 5–6, satisfaction ≥ 4 = medium
Fidelity ≥ 7, satisfaction ≥ 6 = high
Evaluation questionnaireMonth 82. Score for satisfaction of care staff members (0-8 points)
(a) Mean score for care staff members’ satisfaction with trainer’s teaching competences (not at all = 0 points, a little = 1 point, somewhat = 2 points, quite a lot = 3 points, very much = 4 points)
(b) Mean score for care staff members’ overall evaluation of complete PACE Programme (very poor = 0 points, poor = 1 point, fair = 2 points, good = 3 points, very good = 4 points)
(Intention to) Maintenance2Interview with managerMonth 13–151. LTCF manager’s intention to continue with (elements of) PACE in the future (no = 0 points, yes = 1 point)Manager = 0, care staff < 5 = Low
Manager = 1, care staff < 4 = low
Manager = 0, care staff ≥ 5 = medium
Manager = 1, care staff 4–5.9 = medium
Manager = 1, care staff ≥ 6 = high
Evaluation questionnaireMonth 82. Care staff members’ intention and recommendation to work with PACE (0-8 points):
(a) Mean number of PACE steps/documents they intend to use in the future (0-7 points, 1 point for each PACE step/document)
(b) Mean score for recommending PACE to other LTCFs (no = 0 points, yes = 1 point)
  1. 1The RE-AIM component Effectiveness has not formed part the process evaluation study, because it has already been investigated in the effectiveness study of the PACE cluster randomized controlled trial.
  2. 2Because of the limited duration of the study, we measured intention to maintenance instead of actual maintenance.