Skip to main content

Table 4 Aims of the EwQI external evaluation (with related tasks identified by the external evaluation team)

From: Developing the protocol for the evaluation of the health foundation's 'engaging with quality initiative' – an emergent approach

Aim one: to work with award holders on the development and implementation of their evaluation plans

Tasks

- work with the project teams to support their self-evaluations, including data identification and validation.

- assess the experiences of the users as 'active partners' in the projects, seeking to establish, for example, their role in defining outcome measures and their contribution to the design and implementation of improvement interventions and to governance arrangements.

- consider how the counterfactual for each project can be addressed to assess how much change was attributable to the project, and how much to secular activity.

Aim two: to synthesise the data and findings from project level evaluations

Task

- synthesise the data and findings from project level evaluations using a modified form of logic modeling within an overall framework informed by realist evaluation and develop a logic model for the initiative as a whole.

Aim three: to assess increases in clinical engagement in quality improvement

Tasks

- gauge current clinical engagement through an examination of the documentary evidence, using the projects' original proposals and other evidence made available to us by the projects.

- following this, conduct interviews with project team members and key informants in order to explore the state of affairs in the quality improvement context of each project before it has had a chance to influence that setting.

- assess the change achieved during the life of the initiative by supporting each project in designing, implementing and analysing a survey of relevant clinicians.

- in the final year of the initiative, conduct a web-based Delphi survey to identify how clinicians can best be engaged in quality improvement initiatives.

Aim four: to measure the effectiveness of the award scheme (during its life) in leveraging external commitment to clinical leadership of quality improvement

Tasks

The results of the project surveys and the Delphi will be used to support a workshop with representatives from each project on leveraging external commitment, identifying barriers, facilitators, processes, and outcomes.

Aim five: to evaluate the increase in competency and infrastructure for quality improvement in the professional bodies involved in the EwQI

Tasks

- conduct in-depth interviews with each relevant professional body focusing on the issues identified by Leatherman and Sutherland, viz: standard setting, development of quality measures, data collection and analysis, peer review and the design, based on evidence, of interventions to predictably improve patient care.

- look at what the professional bodies involved in the EwQI have done. How effectively have they involved users? Have they promoted more effective use of audit and of audit data?

Aim six: to assess the policy influence and cost consequences of the initiative

Tasks

1. Influence of the EwQI

- evaluate the projects' legacy plans

- ask the project teams to identify the impact their work has had on the development and implementation of other quality initiatives, such as, for example, the development of a relevant NSF.

2. Cost consequences

- work with the projects to explore what data they can provide to estimate costs.

- provide further advice on these requirements to the project teams

- collect data throughout the EwQI on the 'central' costs of the initiative