Table 3 Other evaluations of individual CLAHRCs

Ariss, 2012 [27]

Early internal evaluation of CLAHRC South Yorkshire (only executive summary publicly available).

Described as a developmental evaluation approach combined with realist evaluation and a utilisation focus to guide the evaluation activities.

Informed by 27 interviews with theme leads, programme managers, members of the core team and the executive board, conducted in 2011.

Analysis of quarterly theme activity reports covering the period from October 2008 to March 2010 (53 in total).

Implications and opportunities are reported under nine interlinked headings:

1. Changing landscape

2.Participation, involvement and engagement

3.Public and patient involvement

4.Priority setting

5.Addressing inequalities

6.Capacity building

7.Governance and programme processes

8.Funding and value

9.Outcomes and impact

On impact, the importance of leaving a ‘footprint’ or evidence of the legacy of CLAHRC SY is recognised. However, there is a danger that collaboration and other successes could remain invisible or unattributed and impacts occur further ‘downstream’ and beyond the CLAHRC funding envelope.

Caldwell, 2012 [28]

How national-level understanding of the aims and objectives of the CLAHRCs translated into local practice in North West London.

Uses a variation of Goffman’s frame analysis to trace the development of the initial national CLAHRC policy to its implementation at three levels.

Data collection and analysis were qualitative through interviews (n = 21), document analysis and observation (hours not specified).

Interviews conducted at two different time periods but by different interviewers for slightly different purposes but are an acknowledged limitation.

Analysis at the macro (national policy), meso (national programme) and micro (North West London) levels showed a significant common understanding of the aims and objectives of the policy and programme. Local level implementation in North West London was also consistent with these.

Chew, 2013 [29]

Explore the enactment of full-time intermediary roles in bridging the knowledge-translation gap.

A qualitative case study in an anonymised CLAHRC exploring the formalised intermediary roles of seven ‘knowledge brokers’ enacted in different partner organisations.

Data collection included individual interviews, a focus group with all intermediaries and 118 h of workplace ethnographic observation.

Structural issues around professional boundaries, organisational norms and career pathways may make such roles difficult to sustain in the long term.

Despite the intuitive appeal of intermediary roles as a knowledge-translation solution, organisations should think carefully about how best to realise them if they are to achieve their potential in a sustainable manner.

Cooke, 2015 [30]

Explores how one CLAHRC used collaborative priority setting between researchers and end users to shape its research agenda and project development.

Mixed methods

Semi-structured interviews (n = 28) with CLAHRC researchers and partners. Field notes from a workshop of key stakeholders.

Documentary analysis of CLAHRC internal reports and annual reports from the first two and half years (2008–10).

Dedicated CLAHRC resources, including the use of ‘matched funding’, increased the potential for engagement across academic and practice boundaries.

Fitzgerald, 2015 [31]

Explores how the design and initiation of a CLAHRC impacts its modes of operation and knowledge mobilisation.

Longitudinal case study of one CLAHRC in first 3 years of existence (2008–2011).

Documentary analysis of minutes from board and project team meetings combined with observation and participation in project meetings of two implementation project teams.

Setting up translational networks is insufficient in itself. To leverage benefit attention must be paid to devising a structure which integrates research production and use and facilitates lateral cross-disciplinary and cross-organisational communication

Knowledge mobilisation extends beyond knowledge translation to include the negotiated utilisation of knowledge—a balanced power form of collaboration.

Gerrish, 2014 [32]

Evaluation of an initiative undertaken by NIHR CLAHRC South Yorkshire to increase KT capacity among clinical and academic nurses from partner organisations through a secondment model.

Qualitative evaluation using focus group and individual interviews with 14 clinical and academic secondees and five managers from partner organisations to explore contribution secondees made to KT projects.

Qualitative content analysis used to identify criteria for success.

Six criteria for judging the success of the secondments at individual, team and organisation level were identified: KT skills development, effective workload management, team working, achieving KT objectives, enhanced care delivery and enhanced education delivery.

Hosting teams should provide mentorship support to secondees and be flexible to accommodate secondees’ needs as team members. Ongoing support of managers from seconding organisations is needed to maximise the benefits to individual secondees and the organisation.

Heaton, 2015, 2016 [33, 34]

To delineate the mechanisms by which, and circumstances in which, some projects carried out under the programme achieved success in knowledge translation while others were frustrated.

Longitudinal case study of one CLAHRC over 5 years of existence (2008–12).

Phase 1 mapping to develop programme theories involving 77 semi-structured interviews with 54 stakeholders, combined with documentary analysis.

Phase 2 exploration of programme theories via in-depth case studies of four CLAHRC projects. Twenty-eight semi-structured interviews with project teams.

Identification of five rules based on nine associated mechanisms for promoting knowledge translation through collaborations based on principles of co-production (active agents, equality of partners, reciprocity and mutuality, transformative and facilitated).

1. Base research on co-production through closer collaboration

2. Establish small strategic teams led by strong facilitative leaders

3. Harness and develop respective assets

4. Promote relational adaptive capacity

5. End user is king

Howe, 2013 [35]

To identify whether an assessment framework can provide information for collaborative leaders about how the collaborative approach is implemented, what and where additional support may be required and identify potential peer exemplars.

Assessment of extent to which 17 projects engaged with eight promoted collaborative methods (2010–2011).

Review of 17 individual formal ‘end of project’ meetings. Two to five participants in each meeting and included use of two non-linear scales (0–6) to assess the relative priority given to ‘engagement’ and ‘results’.

Uptake of collaborative methods was variable across projects with no project engaging with all methods, but all engaging with some.

Jordan, 2014 [36]

Explores the nature of the research team–service user relationship in collaborative health research conducted by CLAHRC for Nottinghamshire, Derbyshire, and Lincolnshire (NDL).

Utilises data from a proposed internal evaluation of the CLAHRC-NDL [61] (no data presented) focused on the CLAHRC as a developing organisation and exploring members’ experiences of the CLAHRC.

The study ‘involves’ 46 semi-structured interviews from across one CLAHRC’s membership. The authors state that they intentionally prioritise the service user voice but unclear how many (if any) services users are included in the dataset.

There can be a disparity between initial expectations and actual experiences of involvement for service users. Therefore, as structured via ‘The Three Rs’ (Roles, Relations and Responsibilities), aspects of the relationship are evaluated (e.g. motivation, altruism, satisfaction, transparency, scope, feedback, communication, time). Regarding the inclusion of service users in health research teams, a careful consideration of ‘The Three Rs’ is required to ensure expectations match experiences.

Kislov, 2012 [37]

Explored intra- and inter-organisational boundaries on the implementation of service improvement within and across primary healthcare settings and on the development of multi-professional and multi-organisational communities of practice in the Chronic Kidney Disease theme of the CLAHRC Greater Manchester (GM).

Qualitative embedded case study design, encompassing 20 semi structured interviews with practice doctors, nurses, managers and members of the CLAHRC facilitation team.

Data also derived from 20 h of direct observation, conducted predominantly at learning sessions and practice meetings and from documentary analysis.

The study showed that in spite of epistemic and status differences, professional boundaries between general practitioners, practice nurses and practice managers co-located in the same practice over a relatively long period of time could be successfully bridged, leading to the formation of multiprofessional communities of practice.

While knowledge circulated relatively easily within these communities of practice, barriers to knowledge sharing emerged at the boundary separating them from other groups existing in the same primary care setting. The strongest boundaries lay between individual general practices, with inter-organisational knowledge sharing and collaboration between them remaining unequally developed across different areas due to historical factors, competition and strong organisational identification.

Kislov, 2014 [38]

Exploration of the discontinuity of knowledge sharing across different groups co-located within the collaborative research partnership, CLAHRC GM.

Qualitative single case study involving a purposive sample of 45 research participants drawn from both core and peripheral membership of the four domains of CLAHRC GM.

Interviews were supplemented by direct observation (69 h) of various boundary encounters (e.g. implementation team meetings, learning sessions, practice visits, etc.). Document analysis of (e.g. reports, meeting minutes, presentations, leaflets, etc.) was also carried out.

The structure of the CLAHRC institutionalised the pre-existing gap between the activities of research and implementation strands underpinned by political (conflicting goals and incentives) and epistemic (conflicting attitudes to evidence) factors. This prevented an open conflict between the strands, but at the same time removed the need to renegotiate the boundary and develop a shared practice.

Collaboration within the CLAHRC is a complex, dynamic system of practices, boundaries, and boundary bridges with the potential for both continuity and discontinuity in knowledge sharing. Differences between communities of practice give rise to discontinuities in knowledge sharing. This in turn highlights the role of fragmented organisational structure, divergent meanings and identities, and marginalised boundary bridges in the (re)production, legitimisation, and protection of boundaries.

Kislov, 2016 [39]

Explored what strategies knowledge brokering professionals deploy to alleviate the challenges associated with fulfilling a hybrid role in CLAHRC GM and examines the implications of these strategies for theoretical understanding of knowledge brokering in a broader organisational and institutional context.

Qualitative embedded case study design involving 57 research participants drawn from three projects and the management team to represent different sectors (primary, community and secondary care) and occupational groups (doctors, nurses, care coordinators, managers, etc.5).

Semi-structured interviews were conducted at two points and were supplemented by direct observation (14 h) of team meetings, educational sessions and practice visits, as well as by numerous informal face-to-face conversations with participants.

Formally designated knowledge brokers mitigate the constraining power of context by transferring some of their knowledge brokering functions to managers and clinicians; by conforming to the local ways of doing things; and by complementing (and even replacing) the situated processes of knowledge brokering with the supply of knowledge and skills to clinicians wishing to achieve their organisational performance objectives. These strategies reveal how, through use of knowledge brokers, macro-level institutional arrangements exert influence on the dynamics of knowledge processes unfolding in practice, how the formalised and emergent elements of knowledge brokering as a collectively enacted phenomenon are intertwined, and how the professional expertise and authority of hybrids can become an impediment to their knowledge brokering function.

Initiatives deploying designated boundary spanning roles could possibly benefit from diversifying the pool of knowledge brokers to include managers, quasi-managerial professionals and professionals with formal managerial responsibilities, and supporting the formation of links between knowledge brokers working at different levels.

Kislov, 2017 [40]

Explored how investment in boundary spanning roles, processes and practices changed over time in CLAHRC GM.

Qualitative longitudinal case study involving 88 participants.

Semi-structured interviews (30 to 95 min in duration) were conducted (in two rounds (2009–2010 and 2012–2013). CLAHRC facilitators and managers remained in their posts for the second round of data collection and were interviewed twice. However, as different general practices participated in CLAHRC projects in 2009–2010 and 2012–2013, the sample significantly differed between the two rounds, but remained comparable in terms of the professional and organisational groups represented.

A focus group was conducted with all facilitators at the end of 2013 to discuss the development of legitimacy over time.

Using a Bourdieusian lens, three main themes emerged: (1) changes in the distribution of economic, cultural and social capital mobilised by boundary spanners; (2) implications of these changes for the relationships between the intersecting fields; and (3) effects on the social trajectories of boundary spanners.

The legitimation of boundary spanning roles and practices is a highly transformative, collective and political process that increases the capital endowments and authority of individual boundary spanning agents but may lead to the erosion of the very same roles and practices that were being legitimised

Marston, 2013 [41]

Investigated how PPI was put into practice and how patient and professional roles developed over time.

A 4-year ethnographic study, using participant observation of PPI activities run by CLAHRC Northwest London (NWL) (160 h) and in-depth interviews (n = 89), 45 with patient participants (i.e., patients and service users involved in CLAHRC improvement projects) and 44 with health-care professionals involved in implementing PPI.

At first, health professionals demanded evidence of PPI effects of the type typical in clinical practice, such as cost-effectiveness data, treating PPI as a discrete intervention to improve a specific health outcome. They often spoke about effect in linear terms, and measured success using indicators such as successful participant recruitment and retention or tangible non-health outputs (e.g. leaflets co-designed with patients), rather than changes in health outcomes.

Patients talked about their own contributions in collective and utilitarian terms: they were reluctant to attribute success to individuals, emphasising the role of the team. For them, effect meant timely (and rapid) implementation of incremental changes in health care, which were then sustained and improved upon through collaborative relationships between patients, clinicians, researchers, and others.

Over time staff focus shifted towards creating environments conducive to patient collaboration, and less on calculating the effect of individual contributions. PPI success increasingly described in terms of collaborative relationships between diverse patients and professionals, and acknowledged the importance of unpredictable positive effects of patient innovations.

Martin, 2013 [42]

Explored the way in which CLAHRC for Leicester, Northamptonshire and Rutland (LNR) was put into practice, to understand the theories of change on which its structures and activities are premised, and the degree to which these are realised.

Longitudinal, mixed-methods using interviews, observations and documentary analysis.

Twenty-seven interviews (conducted 2010–2011) with core CLAHRC staff including the executive group, theme leads, deputy leads and managers responsible for the programme of research and implementation.

Supplemented by ‘extensive’ observational work involving attendance at key CLAHRC internal meetings and externally-oriented events.

Documentary analysis of minutes of meetings, strategy documents and externally oriented publicity materials were used not as a data source but rather to sensitise the researchers to key issues in the CLAHRC and inform interviews.

The breadth of CLAHRCs’ missions seems crucial to mobilise the diverse stakeholders needed to succeed, but also produces disagreement about what the prime goal of the CLAHRC should be. A process of consensus building is necessary to instil a common vision among CLAHRC members, but deep-seated institutional divisions continue to orient them in divergent directions, which may need to be overcome through other means.

Reed, 2018 [44]

Consolidation of cross-project learning from the first 5 years of the CLAHRC NWL (2008–2013).

Learning from 22 evidence translation projects to used develop theory and a conceptual framework.

Authors acted as auto-ethnographers drawing on own experiences in running and researching the CLAHRC NWL programme.

Analytical auto-ethnography combined with documentary analysis (including project proposals, progress and final project reports, posters and presentations) and a non-systematic literature review.

Results were interpreted using complexity theory and ‘simple rules’ that enhanced project progress were identified.

Three strategic principles, (1) ‘act scientifically and pragmatically’—knowledge of existing evidence is only one part of the effort required to achieve sustainable improvements in care in complex systems; (2) ‘embrace complexity’—evidence-based interventions only work if supporting or dependent practices and processes of care are working sufficiently well; (3) ‘engage and empower’—evidence translation and system navigation requires commitment and insights from staff and patients with experience of the local system, and changes need to align with their motivations and concerns.

Twelve associated ‘simple rules’ also presented to provide actionable guidance to support evidence translation and improvement in complex systems.

Renedo, 2015 [43]

Examined how PPI was organised and enacted in practice in CLAHRC NWL.

Uses data drawn from the larger ethnographic study by Marston [41].

Draws on interviews with 20 ‘patient participants’—patients or carers involved in CLAHRC improvement projects conducted between September 2010 and November 2012 and supplemented with 132 h of observation of PPI activities run by the CLAHRC.

Patients used four elements of organisational culture as resources to help them collaborate with healthcare professionals. The four elements were (1) organisational emphasis on non-hierarchical, multidisciplinary collaboration; (2) organisational staff ability to model desired behaviours of recognition and respect; (3) commitment to rapid action, including quick translation of research into practice; and (4) the constant data collection and reflection process facilitated by improvement methods.

Spyridonidis, 2015 [46]

To explore the relationship between knowledge translation and leadership in a CLAHRC aiming to bridge the gap between research and practice.

Undertaken as part of a longitudinal 5-year exploration of the development of a CLAHRC.

Data for study derived from interviews with the clinical leaders (n = 36) of CLAHRC projects at two time points.

Supplemented with documentary analysis of internal CLAHRC reports

Relationship between leadership and KT shifted over time from an authoritarian top-down leadership with set outcome measures for KT performance to one of distributed leadership that better accommodated the diverse range of CLAHRC stakeholders.

Knowledge translation viewed as an on-going process informed by interactions between individuals and groups, underpinned by pre-existing individual and group experiences and values.

Spyridonidis, 2015 [45]

Explored the organisational development of the CLAHRC with a focus on a new hybrid physician–manager role, working within this new organisational form.

Longitudinal qualitative study in CLAHRC NWL.

Data derived from interviews with physicians who had taken on a hybrid physician–manager role (n = 62) in CLAHRC projects. Physicians were interviewed twice, at the beginning of their project and at the end over an 18-month time period (total n = 124 interviews).

The study also draws on interviews with CLAHRC senior members conducted as part of the larger study.

Three differing responses were found to taking on a hybrid physician–manager role (the sceptics, the innovators and the late majority), with identity emerging as a mitigating factor for negotiating potentially conflicting roles.

Waterman, 2015 [47]

To explore how CLAHRC GM knowledge transfer associates facilitated the implementation of evidence-based healthcare across several commissioning and provider health care agencies.

A prospective co-operative inquiry with eight knowledge transfer associates responsible for the facilitating the implementation of evidence based practices in six projects in primary- and community-care.

Twenty semi-structured interviews with other team members to gain their perspectives of the facilitation role and process.

Facilitation is context dependent and ‘one size does not fits all’.

Facilitators need tailored support and education to enhance their capacity to support the process of implementation.

Wright, 2013 [48]

Sought to understand the experiences of nurses and allied health professionals acting as first-time knowledge brokers and those of their mentors in CLAHRC NDL.

Exploratory study using interviews with 17 knowledge brokers and 5 mentors to elicit their experiences as first-time knowledge brokers, attempting to bridge the research-practice gap within CLAHRC NDL.

Supplemented with data from documents (reflective diaries and final reports) produced as part of the programme.

Four themes described their experiences: expectations, pragmatics, emotional reactions and outcomes.

Knowledge brokering roles had multi-level benefits. However, there is a lack of support and recognition for these roles at an organisational level, making these activities difficult to sustain in the long term.