A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar
Implementation Science volume 13, Article number: 138 (2018)
Facilitation is a promising implementation intervention, which requires theory-informed evaluation. This paper presents an exemplar of a multi-country realist process evaluation that was embedded in the first international randomised controlled trial evaluating two types of facilitation for implementing urinary continence care recommendations. We aimed to uncover what worked (and did not work), for whom, how, why and in what circumstances during the process of implementing the facilitation interventions in practice.
This realist process evaluation included theory formulation, theory testing and refining. Data were collected in 24 care home sites across four European countries. Data were collected over four time points using multiple qualitative methods: observation (372 h), interviews with staff (n = 357), residents (n = 152), next of kin (n = 109) and other stakeholders (n = 128), supplemented by facilitator activity logs. A combined inductive and deductive data analysis process focused on realist theory refinement and testing.
The content and approach of the two facilitation programmes prompted variable opportunities to align and realign support with the needs and expectations of facilitators and homes. This influenced their level of confidence in fulfilling the facilitator role and ability to deliver the intervention as planned. The success of intervention implementation was largely dependent on whether sites prioritised their involvement in both the study and the facilitation programme. In contexts where the study was prioritised (including release of resources) and where managers and staff support was sustained, this prompted collective engagement (as an attitude and action). Internal facilitators’ (IF) personal characteristics and abilities, including personal and formal authority, in combination with a supportive environment prompted by managers triggered the potential for learning over time. Learning over time resulted in a sense of confidence and personal growth, and enactment of the facilitation role, which resulted in practice changes.
The scale and multi-country nature of this study provided a novel context to conduct one of the few trial embedded realist-informed process evaluations. In addition to providing an explanatory account of implementation processes, a conceptual platform for future facilitation research is presented. Finally, a realist-informed process evaluation framework is outlined, which could inform future research of this nature.
Current controlled trials ISRCTN11598502.
The challenges of ensuring practice is informed by the best research evidence are well rehearsed. While facilitation as a role and process is shown to be a promising approach to enabling evidence-informed practice [1,2,3], there is a need for theory-informed evaluations of facilitation as an implementation strategy . In this paper, we report on a realist-informed process evaluation, which was embedded in the first cross Europe randomised controlled trial (RCT) to evaluate two approaches to facilitating urinary incontinence recommendations in care home settings . This study was novel in scale with a four cross-country setting, and as an exemplar of a realist process evaluation within a large scale international trial. The purpose was to provide a theory-driven explanation of the response to facilitation interventions as they were being implemented in practice.
Methodological guidance reinforces the importance of process evaluations in designing and evaluating complex interventions [6, 7]. Moore et al.’s process evaluation framework identifies the importance of paying attention to what is implemented, the mechanisms responsible for impact and the effect that context can have on implementation. The Standards for Reporting Implementation studies (StaRI)  also focus attention on the importance of reporting underpinning intervention mechanisms and the influence of the implementation context. The guidance and reporting standards both resonate with the idea of a realist-informed inquiry, which pays attention to mechanisms, context and outcomes [9, 10]. Realist inquiry is particularly helpful in providing a theory-driven explanation of how interventions and programmes, which by their nature are complex, work contingently within the context of their implementation.
There has been a lively debate about the notion of realist randomised controlled trials [11,12,13,14]. The debate centres on whether RCTs are ‘inimical to realist enquiry’ (, p1). Whilst RCTs and realist inquiry share some of the same language, i.e., mechanisms and contexts, there is disagreement about the meaning of those terms because of fundamentally different ontological perspectives, and a difference of opinion about whether this matters. In this research, we conducted a randomised controlled trial, which involved a process evaluation that was realist informed. In this way, we were able to remain faithful to the foundations of realist research as developed by Pawson and Tilley  and reap the benefits of a theory-informed approach to evaluation, whilst preserving the strengths of an RCT design.
As one of the first published examples of a realist process evaluation [15,16,17], we provide details about how we approached this evaluation, before presenting realist contingent explanations about how people responded to the facilitation interventions as they were being implemented. Finally, we offer a framework to help guide the conduct of future realist process evaluations.
Our realist process evaluation enquiry, rather than identifying cause and effect relationships, aimed to uncover what worked (and did not work), for whom, how, why and in what circumstances whilst implementing and evaluating two types of facilitation interventions. See Seers et al.  for the trial protocol, and Seers et al.  for trial outcome findings.
We followed the stages of realist evaluation including theory formulation, theory testing and refining. A fundamental assumption of realist inquiry is that ‘programmes are complex interventions introduced into complex systems’ (:p33) including that programmes are theories. Therefore, realist theories typically combine elements of substantive theory with stakeholders’ theories—i.e. their ideas about how programmes may work. Recognising that interventions work differently in different circumstances rather than identifying linear cause and effect relationships through secessionist logic (x causes y: often illustrated through logic models), realist enquiry is concerned with identifying the underlying generative mechanisms about how interventions work [or not]. Dalkin et al.  suggest that a mechanism is both the resource that an intervention provides and recipients’ reasoning and response to it. They also conceptualise mechanisms as operating on an activation continuum rather than as an ‘on-off switch’. Therefore, realist theories are those that define the underlying causal mechanisms through which outcomes occur, and the contexts in which those mechanisms are triggered or activated, which are often expressed as context (C) + mechanism (M) = outcome (O).
The trial had three intervention arms: standard dissemination, type A and type B facilitation, which were derived from the Promoting Action on Research Implementation in Health Services (PARIHS) framework [5, 18, 20, 21]. As the starting point for theory formulation, we undertook a concept mining exercise in which we identified the main elements of the interventions and PARIHS that might explain how the interventions could work in practice, and what might influence implementation. We also incorporated the geographical, policy and practice contexts of the international study into this process.
This process resulted in a sizable list of concepts and ideas, which we clustered into meaningful units. Consistent with the focus of realist evaluation on engaging with stakeholders, a workshop was held with 30 participants at an international knowledge utilisation colloquium. These stakeholders had a strong interest in implementation research, and some also had expertise in care home research. During the workshop, we asked participants to share ideas, i.e. personal theories, about how and why standard dissemination and facilitation interventions might work (or not) within care home settings. Following the workshop, participants and study team’s ideas were combined. These were then shared with participants at the colloquium the following year (Fig. 1 and Table 1).
At this second workshop, a number of hypotheses which threaded together the ideas into theories were developed jointly by participants and study team, which are framed here as ‘if-then’ statements [22, 23] (Table 2).
Multiple qualitative methods were used to test these realist theories:
Semi-structured interviews: audio-recorded at baseline/pre-intervention, 6 (T1), 12 (T2), 18 (T3) and 24 (T4) months post the intervention initiation. Country-based research fellows undertook interviews in their native language using a consistent approach. Interviews were guided by a schedule that was developed from the realist theories and tailored to data collection time points. Key informants included site managers, nursing staff, facilitators, residents and next of kin, and relevant external stakeholders such as regional directors.
Non-participant observations of health care and implementation activities were undertaken at least three times across data collection points in each intervention site using a consistent approach involving piloting the observation protocol. Data were recorded in field notes using Spradley’s nine dimensions of observation (space, actors, activities, objects, acts, events, time, goals and feelings) as a guide . We focused on situations where residents were assisted with the management of urinary incontinence and implementation activities in each site. Observation of care necessitated an unobtrusive, sensitive approach and with consent.
Site and country reports were kept and included history and/or events affecting the care of older people: current demographics, legislation and political agenda; payment and organisation of nursing homes, staffing, resident turnover, and any new routines.
Facilitator activity logs completed by the IFs included activities, purpose, time spent, others involved, resources used, comments on what went well and what went less well.
The amount of data collected within each site depended on how conducive the home context was to data collection visits. This accounts for a variation in data collected (Table 3).
Interview and observation data were transcribed in full and managed in Atlas Ti 6.2 and NVivo 9.
A combined inductive and deductive content analysis approach was used. Data were first analysed within country, within site and within data set, per data collection point. Coding was undertaken within countries by country research teams (CM/CH, TN/TvdZ, PS/CM, ACE) to enable within country reliability checking. Country level coding was then shared at cross-country meetings, which involved a wider group of investigators (JRM, KS, GH, BMc). The starting point for analysis was the framework concepts (Tables 1 and 2). Sub-categories and categories that were developed from interview data were then used to analyse observation texts. Afterwards, sub-categories and categories were formed into themes, a process that was undertaken by country research fellows (CM/CH, TN/TvdZ, PS/CM, ACE) and country principal investigators (JRM, KC, BMc, LW). At this point, themes were translated to English including supporting quotations, for the purpose of country level, and then cross-country analysis (Fig. 2).
Cross-country analysis was managed through monthly teleconference and six monthly face to face meetings and began after the 6-month follow-up. These meetings involved research fellows (CM/CH, TN/TvdZ, PS/CM, ACE), country principal investigators (JRM, KS, KC, BMc, LW) and wider FIRE team members (GH, ALK, AT). Involving different investigators at each stage provided opportunities for challenge and cross checking of both analysis processes and emerging findings. At this stage, the development and refining of context-mechanism-outcome threads was undertaken. This involved searching for context, mechanism, outcome elements and patterns from across the themes through a deliberative and inductive process.
Findings from the trial showed no significant difference between study arms; all study arms improved on the primary outcome (documented compliance with continence recommendations) over time in all countries, but this was not statistically significant . The 12-month type A and the 24-month type B facilitation interventions did not have different levels of impact on documented compliance with recommendations. Both facilitation groups showed significantly better documentation in three outcomes: cognitive impairment, depression and incontinence-associated dermatitis between baseline and 24 months, although these were based on small numbers .
Findings from the process evaluation are expressed as realist CMO configurations. Where we observed a difference between responses to type A and B facilitation, this is highlighted; however, findings surfaced similar issues irrespective of the type of facilitation approach.
Aligning to needs and expectations
The content and approach of the two facilitation programmes (context) prompted variable opportunities to align and realign support (mechanism) with the needs and expectations of IFs and homes. This influenced their level of confidence in fulfilling the facilitator role, challenged an ability to deliver the intervention as planned, a compromise to intervention content exposure, and a continuum of engagement from sustained-partial-no engagement (outcomes).
The initial theories prompted us to examine issues of fit between, and in combination with, the type of facilitation programme, and the needs of the individuals and homes, and the nature of the support provided by the programme and external facilitators (EF). Findings show that alignment of these characteristics was important for the confidence of the IF to enact the facilitation role as intended, and therefore, the level of engagement there was in the programme in general. Factors that affected the fidelity of the intervention are summarised in Table 4.
The responses to the type of facilitation, formed at the initial residential development programme, were important precursors to how well aligned and relevant the approach was perceived to be by the individual and to the home. The IF went through a process of sense making. Whilst a number of IFs expressed that they had been empowered by the residential experience and the enthusiasm of the EFs, there were differences in the IFs as to the extent they felt aligned with the facilitation approach, and theoretically, practically and emotionally equipped to enact the role. Additionally, whilst in both types of programmes, IFs were unsure about how they were going to translate what they learnt into practice, this perception appeared to be particularly evident in the numbers of accounts reported by those experiencing the Type B residential programme, for example:
Yes initially I thought, Jesus…with all these creative methods, where will this lead to, but I did experience it personally and how illuminating it was. Nevertheless I constantly wondered how am I going to do this on my unit with those persons... (IF type B, baseline, site 1 NL), and, after the residential, I was exhausted. For five days I just sat there, demolished, and like ‘where do I start.’ (IF, site 5, Swe, T1, type B).
Following the residential programme, support for the IFs switched to teleconferences, which whilst welcomed by most IFs, participants presented two challenges. The first, engaging in the group dynamics of a teleconference, including for most, in a language that was not their own:
The monthly teleconference meetings were very tiring because all was in English using telephone, so you do not see the others. We did not know the people either because we entered the project later…some people dominated the conversations…They had lots of questions…They had the advantage of the language (IF type A, 12 months, site 5, NL)
The second challenge was a feeling that there was a lack of opportunity to tailor support to their particular needs in real time, which meant they lacked confidence to act on advice that was provided in the monthly teleconferences:
...every time I heard [EF] it seemed logical, but the moment I got to the institution and had to translate it to actual practice I could not find any resemblance (IF type B, site 3, NL).
Consequently, facilitators felt unequipped to act out their facilitation role. This finding is also linked to the personal characteristics of the IF, described later, which mediated their ability to engage with the requirements of the role and programme.
Further, in relation to alignment of need and expectation, there had been a mismatch in some IF's and home manager’s perceptions about the facilitation programme. Misalignment related to the programmes’ intentions around development of people to be facilitators, versus the knowledge and tools required for putting best practice in place for continence care:
… you know we already use…the assessments...and the products…if it was going to be a case that you will be introducing new ways of doing things…but that’s not what it was about, so, no, I wouldn’t do that again (IF, type B, 12 month, Eng). In this example, the IF only attended one teleconference and then did not participate further in the programme.
As a result of all these factors, although the ‘dose’ of the intervention provided by the EFs within each programme was delivered, the resulting response and actions of the IFs were mixed; and thus, the potential of what they did to impact on practice was also variable.
The success of intervention implementation was largely dependent on whether sites prioritised their involvement in both the study and the facilitation programme. In contexts where interventions were timely coinciding with a regulatory requirement, and/or a need to improve continence care, and where there were fewer disruptions such as changes in staff and management (context), this prompted the prioritisation of the project (mechanism). This resulted in a release of resources (time, staff and material resources), and a more sustained commitment to the study and facilitation intervention (outcomes).
The initial realist theories prompted us to consider the implementation context and conditions that might enable or inhibit facilitator activities and role enactment. We found that there was a mutually reinforcing relationship between regulatory expectations (macro context) and home (meso context) managers’ motivation to prioritise continence and therefore engagement in the study. For example, in the Republic of Ireland, regular Health Information and Quality Authority inspections were used as an incentive to sustain engagement in the project. In Sweden, the prioritisation of urinary incontinence was reinforced in national guidance and by external agent’s expectations:
…we have a guideline (on UI) in the regulations, so whenever a resident moves into site x…they should be offered a basic UI assessment…it’s not negotiable…because you have started the [FIRE] project they now sense they really have to do something about it (Community Chief Nurse, Baseline, Swe).
There was also a reinforcing relationship between home managers and IFs’ ability to participate fully in the facilitation programme and in enacting their role. The dynamics between managers and facilitators were continually negotiated over the intervention implementation period. Where facilitators were given the authority through protected time to carry out activities, including attending monthly teleconference support meetings, this was a function of managers’ prioritisation of the project. As managers varied in their commitment to being involved, often because the day to day demands of running a home took over their attention, subsequent support was patchy or absent:
I did ask for protected time for a couple of the teleconferences but no cover was forthcoming (IF, T2, RoI, type A).
Conversely, there were examples in the data where managers had been able to consistently prioritise the project, which resulted in resources for the IF, particularly in terms of time to work as a project team:
…it’s been really good that we had had the time…we have had the time and energy to discuss things (nurse, T2, Swe, type B).
Money to enable backfill for IFs was available; however, difficulties in finding suitable replacements meant it was not always taken up.
Change in management and/or ownership of a home was generally disruptive to prioritising project related requirements, such transitions were a frequent feature of the implementation context in all of the countries. Losing the original sponsor of the study frequently delayed, and sometimes, completely curtailed activity. Additionally, frequently changing staff or team leaders made it difficult for IFs to sustain the project as a priority at a unit level:
I have openly declared to facilitators they cannot expect anything from the project at the moment. After summer I hope everything will settle again (Manager, T2, type B, NL).
This issue was particularly challenging in homes that were smaller (particularly the case in England), where there was a more limited flexibility in workforce deployment.
Engagement in attitude and action
In contexts where the study was prioritised (including release of resources time, people, tools and infrastructure) and where managers and staff were supportive (context), this prompted collective engagement (as an attitude and action) with the facilitation interventions (mechanism) by managers, IFs and other staff. This resulted in IFs undertaking activities, which resulted in some practice changes (e.g. continence assessment) and impacts on attitudes and beliefs (outcomes).
As described earlier, the consequence of prioritisation was a commitment [or less so] to the project. This outcome forms the condition or context for greater engagement in both attitude and action with the facilitation interventions. The level of engagement reported and observed varied from withdrawal from programme activities (but not from the study), to patchy participation in the monthly support teleconferences, to some facilitators completing the programme. In this sense, engagement referred to both facilitators’ attitudes—‘I can’t do that’ (IF, T2, NL, Type B), as well as their actions—‘I dropped out of it [support programme]’ (IF, T2, Eng, type A).
Where there was not a supportive context (e.g. little support from colleagues, managers, and not enough resources such as backfill time), IFs struggled with the perceived costs of overcoming the challenges, and some gave up. Their ability to overcome these challenges was also inhibited by remote teleconference-based EF support. However, in contrast, in these situations some IFs had been encouraged to engage their local ‘buddy’ as a source of support:
…I do not think she’s fully comfortable, so *** has buddied up with her and is sort of the driving force behind it…they are spending time together and doing things (Manager, T1, Eng, Type A).
A more engaged IF tended to lead to more engaged home staff. In all facilitation intervention sites, there was some staff resistance to the practice changes needed to align with the guideline recommendations, such as continence assessment. However, active facilitators who engaged staff through meetings, team-related activities, workshops and role encouragement resulted in some success, including for example, the implementation of a new continence care screening and assessment form. This ability to engage home staff was facilitated by setting up a local project team in some sites, which became part of the support structure for the IF.
Whilst there was no impact on the primary outcome, interview and observation data showed that some facilitators had made changes to continence practices, such as introducing improved fluid monitoring, and in changing staff perceptions and approaches. Examples of making a difference to residents were also evident, for example:
…the nurse has investigated when I have to pee to see if we could do something about my incontinence. We did this together (Resident, T2, NL, type B).
There was also evidence that specific facilitation intervention activities had led to perceived changes in thinking, for example:
…the culture workshop had an even bigger impact than we expected, it was not about the collection of data alone, but an action in itself. It resulted in consciousness among staff about the impacts of incontinence for the client’ (IF, T3, months, RoI, Type B).
In contrast, data from follow-up interviews revealed that standard dissemination sites did not use the urinary continence recommendations or the implementation guide that they had received.
Learning over time
IFs’ personal characteristics and abilities, including personal and formal authority, in combination with a supportive environment prompted by managers (context) triggered the potential for making sense and learning through the support programme over time (mechanism), which could result in a sense of confidence and personal growth, and enactment of the facilitation role (outcomes).
Whilst the starting point for most of the IFs was enthusiasm and an eagerness to succeed, their ability to carry out their role, including suggested facilitator activities, appeared to be linked to their level of authority to act, which was associated with credibility, confidence and perseverance when facing challenges. Despite a set of criteria for the selection of IFs, the practicalities of identifying someone who fitted all of them was a challenge with only 6 of the 16 sites recruited an IF who met the essential facilitation criteria and stayed in post for the duration of the study. This resulted in mixed cohorts of facilitators in each arm of the intervention, with authority to act being a significant factor in successfully enacting the role. Two forms of authority were evident: formal authority from their role within the home and/or delegated by the home manager to be an IF, and personal authority, which the IF engendered amongst those with whom they worked. The levels of authority varied amongst the IFs. When there were challenges, it was the resilience and persistence of the IFs, which in some cases was reinforced by encouragement and active participation from managers, which kept some momentum going.
As each facilitator progressed on their facilitation journey, we observed some critical junctures in their learning. There was a critical point immediately after the residential development programme at the beginning of the intervention period when the issue of alignment of IFs and home expectations, and not knowing ‘where to start’ was most evident. Facilitators’ ability to connect meaningfully in monthly calls were additional critical junctures, with some reporting challenges with understanding the language of facilitation and implementation (in addition to conversing in a foreign language) as described earlier.
However, over time, and with the teleconference support from EFs, and for some, the input of buddies, we observed a growing ability and confidence in some facilitators to act in accordance with the particular facilitation approaches. Additionally, whilst there were no significant differences in effectiveness between the interventions, there was increasing compliance with recommendations over time, suggesting improvements . Key characteristics identified from field notes, interviews with managers and external and IFs that made some facilitators (irrespective of their allocation to type of facilitation or country setting) more successful than others are included in Table 5.
Data from facilitator activity logs, interviews and observations shows that learning and developing over time resulted in some facilitators enacting their roles through activities that made the particular facilitation approach they were aligned to more visible (see Table 6). Additionally, their learning pervaded other aspects of work life:
I suppose the big thing for me has been the personal journey…It goes into everything now not just continence, not just person centred care…It’s getting them to think for themselves…(IF, T3, RoI, type B).
In summary, findings show there were a number of mechanism activation continua . Figure 3 shows that the combination of greater activation of prioritisation and engagement, together with greater activation of fit and alignment of the intervention to the needs and expectations of IFs and homes, is linked to activation of learning over time. The impact of learning over time was in the activity undertaken relevant to the type of facilitation and, in some cases, to implementing practice changes.
In this realist-informed process evaluation, we have elucidated responses to facilitation intervention implementation within the context of an RCT where neither facilitation approach was effective in significantly affecting the primary outcome . Process evaluation findings showed that there were some impacts to practice, but these were not distinguishable between the two facilitation types. This was unexpected, as type B facilitation was planned to be a more intensive and holistic approach over a longer intervention period and with more support from EFs than type A. We had theorised that this additionality might result in greater impact [5, 21]. However, in reality, both facilitation types experienced similar challenges in delivery, which meant that the fidelity and dose of intervention as standardised for the trial was diluted. As such, the intervention as theorised was not delivered as intended.
In realist terms, the CMOs explained how the resources and opportunities created by both facilitation interventions were taken up (or not) in different contexts. The interconnections between these CMOs are represented in Fig. 4.
The resultant framework illustrates that these elements worked in combination as a mid-range theory . However, the impact of these combinations will be different depending on their arrangement within a particular circumstance, reflecting the significance of context to intervention implementation. Where there were impacts from being involved in the facilitation intervention, this was due to individuals enacting the facilitator role, which they achieved through learning over time. Learning over time and enacting the role was a function of a combination of elements. For example, prioritisation was important in that a reciprocal, supportive relationship between home managers and IFs combined with the stability of the home context, for example, staff complement and turnover. In turn, this combined with whether the project fitted with the priorities of the wider environment that home was operating within, for example, whether continence care was a particular focus for attention impacted on engagement. Prioritisation interacted with engagement, which was dependent upon the availability of the appropriate resources at the right time to enable facilitators to carry out their role, and whether they drew on the additional sources of support such as buddies when needed. Prioritisation, engagement, and fit and alignment together influenced how the facilitation interventions lined up with the expectations of homes and IFs, and the potential to tailor the approach including support structures to the on-going needs of both.
Whilst the elements in the framework have different combinations in different circumstances, we observed patterns. For example, one combination resulting in a positive response to the intervention related to a supportive reciprocal relationship between the IF and manager . This reciprocity led to a release of resources in the form of time to engage with the programme, which was particularly evident in sites where the intent of the programme aligned well with both home and facilitators’ expectations. A different but consistent combination included a challenge to the response to the facilitation programme where the context of the home was disruptive. This was usually because of changes in managers, which resulted in a lack of stability, lack of buying in to the facilitation programmes and an inability to mobilise resources to engage fully, which left facilitators isolated.
The role of leaders and managers alongside facilitators, and as facilitators of implementation efforts themselves, is highlighted as a key ingredient for success by others [26, 27]. Specifically, the active and visible participation of managers in implementation interventions and processes is important for the allocation of resources and provision of support. For this study, early managerial buy in and engagement with the study itself was an obvious antecedent to supporting what was required to implement the facilitation interventions over a sustained period.
Other studies of facilitation have shown that it can take some time to affect outcomes . In this study, we found that irrespective of facilitation type, for some IFs, there had been learning over time. This occurred where there was greater fit and alignment of the interventions to expectations, prioritisation and engagement, which had begun to result in some positive changes in practice. Learning over time was a feature within a rehabilitation research context in which an occupational therapist adjusted the way they worked with care homes and residents as they trialled a complex intervention and became more confident and proficient over time . The idea of learning over time also fits with a realist logic of programme implementation, where we would expect to observe a dynamic interplay between the intervention, actors, contexts and mechanisms as the resources and opportunities created by the intervention are taken up, or not.
The realist process evaluation also highlights a challenge related to the delivery of an intervention like facilitation within the context of a randomised controlled trial. Any implementation effort requires work , including tailoring to local need [4, 30], which raises a question about fidelity versus adaptation of an implementation intervention such as facilitation. The manualised facilitation interventions in this study left little scope to particularise support to the individual needs and circumstances of facilitators as they changed over time, therefore, for example, where there were critical junctures or moments of crisis  for individual facilitators which could not be responded to and opportunities to support them lost. Inevitably, this affected facilitators’ confidence and expertise to enact the role. Reframing the idea of fidelity away from adherence to delivery of specific intervention components towards alignment with intervention function and process ; as a ‘thread that pulls together implementation processes within a trial along with the theories embedded in a complex intervention’ (, p446) may be more helpful. Arguably, this view provides a more flexible framework for assessing fidelity, including being able to contextualise interventions to the needs of specific circumstances whilst still being faithful to their underpinning theory/ies.
Realist-informed process evaluation—strengths and limitations
Very few published examples of completed realist-informed process evaluations exist, and none at the scale of this study set in multiple country contexts. Indeed, much of the debate about combining realist inquiry with trials put these approaches in opposition [13, 14]. This presented a challenge because there was no example to follow, but an opportunity to fill a gap and contribute to the evidence base about realist inquiry alongside randomised controlled trials. Arguably, the strength of a realist-informed process evaluation is in the potential to provide greater explanatory power than potentially reductionist approaches centred on logic models. Whilst logic models are helpful for specifying intervention components as inputs and outputs, they can be less useful for developing contingent explanations between them. In this study, we have been able to provide an explanatory account of the antecedents and contingencies that account for the response to the resources and opportunities (i.e. realist mechanisms) offered by the facilitation intervention, moving beyond a list of facilitators and barriers and a conceptualisation of context as something that is static. As well as providing a richer explanation, the results should also be of more use to others embarking on research about facilitation because they provide an initial conceptual platform for further investigation . Additionally, we offer a framework that identifies some co-ordinates and questions for realist process evaluations within randomised controlled pragmatic trials, which may be a useful starting point for others in future research (Fig. 5). The framework is based on our experience in this study and previous realist evaluation research projects conducted by some of the authors [33, 34], and some of the principles of conducting process evaluations described by Moore et al. [6, 7].
The strength of this study is that we drew on multiple methods to test and refine the programme theories through the project and included observations and activity logs. Data collected from these approaches complemented data from interviews, enabling a more trustworthy picture to emerge. It was also a strength to engage with stakeholders to develop our initial programme theory ideas and share findings as we progressed. Furthermore, a judgement about the credibility of the findings of this realist inquiry study can be verified if read alongside the other publications arising from this study [18, 35].
A large amount of data were collected, which through the data management and analysis process may have lost some of its site and country nuance, particularly as the last part of the analysis process was managed with data translated into English. However, our analysis process involving investigators at different stages also presented multiple opportunities to enhance the reliability of the resultant findings.
This was a pioneering and complex study due to its scale and four-country context, which provided a novel circumstance in which to conduct one of the few realist-informed process evaluation as part of a randomised controlled implementation research trial. The CMO configurations were translated into a mid-range theory framework, which provides an explanation about the response to the facilitation interventions we observed in this realist inquiry. This shows that elements of fit and alignment, prioritisation and engagement can work together to determine a facilitator’s opportunity to learn over time, enact their role and have an impact, which could provide a useful conceptual platform for future facilitation research. In addition to providing a worked example, we have also outlined a realist-informed process evaluation framework that might be useful for future research of this nature as this approach continues to be trialled and developed.
Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012;10:63–74.
Dogherty EJ, Harrison M, Graham I, Keeping-Burke L. Examining the use of facilitation within guideline dissemination and implementation studies in nursing. Int J Evid-Based Healthcare. 2014;12(2):105–27.
Persson LA, Nga NT, Malqvist M, Thi Phuong Hoa D, Eriksson L, Wallin L, et al. Effect of facilitation of local maternal and newborn stakeholder groups on neonatal mortality: cluster randomised controlled trial. PLoS Med. 2013;10:5.
Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why (we think) facilitation works: insights from organisational learning theory. BMC Imp Sci. 2015;10:141.
Seers K, Cox K, Crichton N, Edwards R, Eldh A, Estabrooks C, et al. FIRE (facilitating implementation of research evidence): a study protocol. BMC Imp Sci. 2012;7:25.
Moore GF, Audrey S, Barker M, Bond L, Bonnell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.
Moore GF, Audrey S, Barker M, Bond L, Bonnell C, Hardeman W. et al. Process evaluation of complex interventions, Medical Research Council Guidance. https://www.mrc.ac.uk/documents/pdf/mrc-phsrn-process-evaluation-guidance-final/.
Pinnock H, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, Rycroft-Malone J, et al. Standards for reporting implementation studies (StaRI). BMJ. 2017;356:i6795.
Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.
Pawson P. The science of evaluation: a realist manifesto. London: Sage; 2013.
Bonell C, Fletcher A, Morton A, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75:2299–306.
Marchal B, Westrop G, Wong G, Van Belle S, Greenhalgh T, Kegels G, et al. Realist RCTs of complex interventions—an oxymoron. Soc Sci Med. 2013;94:124–8.
Van Belle S, Wong G, Westrop G, Pearson M, Emmel N, Manzano A, et al. Can “realist” randomised controlled trials be genuinely realist? BMC Trials. 2016;17:313.
Bonell C, Warren E, Fletcher A, Viner R. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Bell et al. BMC Trials. 2016;17:478.
Byng R, Norman I, Redfern S, Jones R. Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation. BMC Health Serv Res. 2008;8:274.
Masterson-Algar P, Burton CR, Rycroft-Malone J, Sackley C, Walker MF. Towards a programme theory for fidelity in the evaluation of complex evaluations. J Eval Clin Pract. 2014;20(4):445–52.
Randall R, Honey S, Hindmarsh J, Alvarado N, Greenhalgh J, Pearman L, et al. A realist process evaluation of robot-assisted surgery: integration into routine practice and impacts on communication, collaboration and decision-making. Health Serv Del Res. 2017;5:20. ISSN 2050
Seers K, Rycroft-Malone J, Cox K, Crichton N, Edwards RT, Eldh AC, et al. Facilitating Implementation of Research Evidence (FIRE): a cluster randomised controlled trial to evaluate two models of facilitation informed by the Promoting Action in Research Implementation in Health Services (PARIHS) framework. In review BMC Imp Sci. https://doi.org/10.1186/s13012-018-0831-9.
Dalkin SM, Greenhalgh J, Jones D, Cunningham B, Lhussier M. What’s in a mechanism? Development of a key concept in realist evaluation. BMC Imp Sci. 2015;10:49.
Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B, Titchen A. An exploration of the factors that influence the implementation of evidence into practice. J Clin Nurs. 2004;13:913–24.
Harvey G, Loftus-Hills A, Rycroft-Malone J, Titchen A, Kitson A, McCormack B, et al. Getting evidence into practice: the role and function of facilitation. J Adv Nurs. 2002;37:577–88.
Pearson M, Brand SL, Quinn C, Shaw J, Maguire M, Michie S, et al. Using realist review to inform intervention development: methodological illustration and conceptual platform for collaborative care in offender mental health. BMC Imp Sci. 2015;10:134.
Rycroft-Malone J, Burton CR, Williams L, Edwards S, Fisher D, Hall B, et al. Improving skills and care standards in the support workforce for older people: a realist synthesis of workforce development interventions. Health Serv Deliv Res. 2016;4:12.
Spradley JP. Participant observation. Illinois: Waveland Press; 1980.
van der Zijpp TJ, Niessen T, Eldh AC, Hawkes C, McMullan C, Mockford C, Wallin L, et al. A bridge over turbulent waters: illustrating the interaction between managerial leaders and facilitators when implementing research evidence. Worldview Evid-Based Nurs. 2016;13:25–31.
Tistad M, Palmcrantz S, Wallin L, Ehrenberg A, Olsson CB, Tomson G, et al. Developing leadership in managers to facilitate implementation of national guideline recommendations: a process evaluation of feasibility and usefulness. Int J Health Pol Man. 2016;5(8):477–86.
Mekki TE, Øye C, Kristensen B, Dahl H, Haaland A, Nordin KA, et al. The inter-play between facilitation and context in the promoting action on research implementation in health services framework: a qualitative exploratory implementation study embedded in a cluster randomized controlled trial to reduce restraint in nursing homes. J Adv Nurs. 2017;73(11):2622–32.
Eriksson L, Huy TQ, Duc DM, Ekholm Selling K, Hoa DP, Thuy NT, et al. Process evaluation of a knowledge translation intervention using facilitation of local stakeholder groups to improve neonatal survival in the Quang Ninh province. Vietnam BMC Trials. 2016;17:23.
May C, Finch T. Implementing, embedding and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–54.
McCormack B, Rycroft-Malone J, DeCorby K, Hutchinson AM, Bucknall T, Kent B, et al. A realist review of intervention and strategies to promote evidence-informed healthcare: a focus on change agency. BMC Imp Sci. 2013;8:107.
Fay B. Critical social science—liberation and its limits. NY: Cornell University Press; 1987.
Hawe P, Sheill A, Riley T. Complex interventions: how ‘out of control’ can a randomised controlled trial be? BMJ. 2004;328(7455):1561–3.
Rycroft-Malone J, Fontenla M, Bick D, Seers K. A realistic evaluation: the case of protocol-based care. BMC Imp Sci. 2010;5:38.
Rycroft-Malone J, Burton CR, Wilkinson J, Harvey G, McCormack B, Baker R, et al. Collective action for implementation: a realist evaluation of organisational collaboration in healthcare. BMC Imp Sci. 2016;11:17.
Harvey G, McCormack B, Kitson A, Lynch E, Titchen A. Designing and implementing two facilitation interventions within the ‘Facilitating Implementation of Research Evidence (FIRE)’ study: a qualitative analysis from an external facilitators. In review BMC Imp Sci. https://doi.org/10.1186/s13012-018-0812-z.
Thanks to those who were involved in this study as participants and who provided advice.
The research leading to these results has received funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 223646. The funder had no role in designing, conducting or interpreting study findings.
Availability of data and materials
The datasets generated or analysed during the current study are not publically available because consent to make data publically available was not part of the consent by participants.
Ethics approval and consent to participate
Ethical Committee approval was obtained in England (10/WSE04/20), Sweden (2009/1806-31/2) and Republic of Ireland (ECM4(u)02/03/10). In The Netherlands, researchers followed advice to get permission from either an ethical committee at site level, or where this did not exist, from a scientific or residents committee at the site (HAZ-11087777-JGS). Research Governance approval was also obtained in England and permission to collect data at the sites obtained in Sweden and Republic of Ireland.
Consent for publication
Consent form allowed the use of anonymised quotations in publications.
JRM is an Editorial Board member of Implementation Science. AK, GH, BMcC, JRM, KS and AT have all been involved in the development of the PARIHS framework. All other authors declared that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Rycroft-Malone, J., Seers, K., Eldh, A.C. et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci 13, 138 (2018). https://doi.org/10.1186/s13012-018-0811-0