Table 2 Process evaluation domains, research questions and data collection
Process evaluation domains | Research questions | Core information sought | Data type | Data source | Record kept |
|---|---|---|---|---|---|
Domain 1: Implementation | 1. How was the intervention implemented in each agency? Including: | Intervention components selected and how agencies asked that they be tailored | Updates | SPIRIT staff implementing the intervention | Fieldnotes and memos - data indexed in NVivo |
a. What components were delivered? | What was delivered in each agency, including: which components; delivery format; provider recruitment and preparation; learning materials; the fidelity with which essential elements were delivered; any changes to the plan, and follow-up activities | Structured observation | Intervention sessions | Completed checklists - data input in spreadsheet | |
b. To what extent were the essential elements implemented? | Email information | SPIRIT trial coordinator's records | Collated spreadsheet data on email delivery | ||
Knowledge brokering records | SPIRIT staff delivering brokered services | Brokered service assessment form | |||
Domain 2: Participation and response | 2. How did people interact with the intervention? What were their levels of participation and satisfaction? | Session participation and responses, including: roles of attendees, proportion of invitees who attended; nature of participation (types and extent of interaction) | Pre-session sign-in and consent process | Participants attending each session | Completed sign-in/consent sheets -no.s and roles input in spreadsheet |
Semi-structured and structured observation | Intervention sessions | Completed checklist, fieldnotes and memos - data indexed in NVivo | |||
3. What effects that are not captured by the outcome measures did the intervention have (including unexpected effects)? | Participants' evaluation of intervention sessions | Self-reported evaluation feedback | Participants attending each session | Completed feedback forms - data input in spreadsheet | |
Informal conversations after sessions | Liaison Person and ad hoc participants | Fieldnotes - data indexed in NVivo | |||
How participants and the organisational system responded to the intervention overall (including unexpected effects) | Interviews | Purposively sampled participants, Liaison Person and CEO | Audio recordings, transcripts and memos - data indexed in NVivo | ||
Interviews, meetings and informal conversations | SPIRIT intervention staff and providers | Fieldnotes and memos - data indexed in NVivo | |||
Domain 3: Context | 4. What was the context of the agencies in which the intervention was implemented? | Immediate characteristics of session delivery context (site, facilities, etc.) | Structured observation | Intervention sessions | Completed checklist fieldnotes data input in spreadsheet |
Organisational context: (i) agency culture, (ii) agenda-setting & prioritisation, (iii) leadership styles & perceptions of leaders, (iv) how research & other information is valued, accessed & used, (v) barriers and enablers to using research, (vi) other contextual factors that may affect outcomes | Semi-structured observation | Intervention sessions | Audio recordings, fieldnotes and memos - data indexed in NVivo | ||
Interviews | Purposively sampled participants | Audio recordings, transcripts and memos - data indexed in NVivo | |||
Interviews, meetings and informal conversations | SPIRIT staff implementing the intervention | Audio recordings, fieldnotes and memos - data indexed in NVivo | |||
Across domains | 5. How might the relationships between the program, the people and the context in each agency have shaped variations in these effects? | Analytic synthesis of above data | |||
6. What lessons can we derive from this study that might be relevant for other interventions and settings? | |||||