In this article, we examined which components of the HH improvement strategies were particularly associated with increased nurses’ HH compliance, as well as other possible factors that may have influenced nurses’ HH compliance. We therefore linked process and effectiveness evaluations in the analysis of findings from the HELPING HANDS study .
Effect evaluation: intention-to-treat versus as-received analysis
In this article, we have tried to explain the effects of two different HH improvement strategies on changes in nurses’ HH. It is important to recognize that this research goal requires a different view on the treatment effects compared to an evaluation of effectiveness. The outcome suggests that the overall conclusions about the effectiveness of the team and leaders-directed strategy arising from the original intention-to-treat analysis may have underestimated the impact and strength of this strategy. The as-received analysis showed higher effect sizes for the team and leaders-directed group than the intention-to-treat analysis on both measurements points. In the long run, we now observed a statistically significant (p = 0.002) increase in nurses’ HH compliance due to the team and leaders-based strategy. This suggests that the team and leaders-directed strategy might have had a more permanent impact on HH outcomes than shown by the intention-to-treat analysis. This corresponds with the findings of Strange, et al. . Their as-received analysis showed higher odds ratios in decreasing risky sexual behaviour than the original intention-to-treat analysis, thereby suggesting that their peer-led sex education program, if consistently implemented, probably had a greater impact on study outcomes.
Effects of strategy adherence on nurses’ HH compliance
The evaluation of strategy adherence did not provide any explanatory variables associated with changes in nurses’ HH compliance. Thus, variation in the HH outcomes across the wards could not be explained by a so-called ‘failure of implementation’ . Nevertheless, it is noteworthy that more nurses from the team and leaders-directed group completed the knowledge quiz compared to nurses from the state-of-the-art group (37% and 13%, respectively; p = 0.001). A possible explanation is that the team and leaders-directed strategy positively influenced the adherence to specific components of the state-of-the-art strategy.
Effects of contextual factors on nurses’ HH compliance
The as-received analysis showed a hospital effect that was mainly due to one hospital. Especially in the long run, HH compliance started to decrease in this particular hospital, while HH compliance in the other two hospitals remained stable or increased further. Little is known about how hospital cultural factors are associated with the implementation of HH improvement strategies. The WHO , Larson, et al.  and Pittet  emphasize the commitment of high-level administrators to create and support a culture of safety and accountability. Culture manifests itself through the values, beliefs and assumptions embedded in organizations and is reflected in ‘the way things are done around here’ . The two hospitals that showed sustainability in HH compliance designated HH as a hospital-wide priority. The third hospital was less explicit and distinct in addressing the goal of HH as an organizational priority. This raises the question of whether the observed changes in HH compliance were affected by hospital culture.
Standard care activities
Although the average HH baseline scores of the wards were comparable between wards from both groups, our analysis showed that a high baseline HH compliance was associated with a smaller effect of both HH improvement strategies. High HH compliance at baseline was particularly seen in the paediatric wards. Wagner and Kanouse  have pointed out that standard care activities may affect adherence behaviours and thus intervention outcomes. It is possible that certain components of our improvement strategies are already part of daily practice in some wards and therefore leave less room for improvement. Despite the influence of baseline scores and hospital effect, the team and leaders-directed strategy significantly contributed to an additional increase in nurses’ HH compliance, both short and long term.
Effects of experiences with the improvement strategies on nurses’ HH compliance
The exploration of the relation between determinants of success and HH compliance provided empirical evidence for performance feedback, social influence and leadership as important vehicles for changing HH behaviour. It seems likely that the mixture of these strategy components affect the teams’ abilities to focus on achieving their HH improvement goals. Our results have strengthened the theoretical underpinning of the composition of our team and leaders-directed strategy by using a team approach for changing individual behaviour. By setting clear norms and targets within the team, individual team members are invited to support each other in achieving this goal.
The findings of our study also show that it is important to promote a team culture that empowers team members to speak up when non-adherence is observed. In this finding, we recognize key elements from the social influence theory  (e.g., team members address each other in case of undesirable behaviour), and the theory on team effectiveness [28, 29] (e.g., participation safety and task orientation) (Table 2). This is of particular interest because ‘speak up’ is positively correlated with improved HH behaviour. During the team sessions, we taught the nurses to provide feedback on the HH behaviour of their colleagues in a correct way. At the same time, we guided the nurses to receive this feedback positively.
Active commitment and initiative from ward management
The results of our study show that specific components of leadership are positively correlated with an improvement in nurses’ HH compliance. Thus, ward managers should address barriers to enable HH as recommended, designate HH as a ward priority, motivate and encourage team members to perform HH, and hold team members accountable for their HH behaviour. This finding corresponds with the key elements from theory of leadership  as displayed in Table 2.
Credits of our findings are not entirely due to the delivery of the team and leaders-directed strategy. Nurses from the state-of-the-art group were not exposed to social influence and leadership as a result of improvement activities from our study. A possible explanation is that these wards, independent of our study activities, have given priority to HH and were motivated and encouraged by their managers. This explanation is supported by the results of a further analysis within the group of the state-of-the-art strategy. We found a significant relation between changes in HH compliance and differences in nurses’ experiences with social influence and leadership. Compared to the state-of-the-art group, the analysis within the group of the team and leaders-directed strategy showed less variation in changes of nurses’ HH compliance. Therefore, an association between changes in HH compliance and differences in nurses’ perceptions of strategy components within the team and leaders-directed group could not be demonstrated. We hypothesize that the lack of variation in this group is due to the consistent implementation of the team and leaders-directed strategy. As already shown by our evaluation of strategy adherence, all nurses within the group of the team and leaders-directed strategy were equally exposed to the main components of this strategy.
Strengths and limitations
The principal strength of our study was the comprehensive process evaluation within the context of a pragmatic randomised controlled trial. Questions about variations in the adherence to both HH strategies, and about factors contributing to the relationship between the HH improvement strategies and nurses’ HH outcomes, would not have been apparent as a result of only analysing the HH outcome data. Process evaluations are, in this sense, part of a more theory-based approach to evaluation, responding to the need to understand which theoretical constructs of an improvement strategy make a difference . By linking data of effectiveness to process data, a theoretical explanatory model can be derived from the process evaluation itself .
Some researchers encourage the simultaneous application of a process evaluation in control groups [5, 38]. By doing so, we discovered the impact of specific aspects of social influence and leadership in the state-of-the-art group that served as a control group. This finding has strengthened the theoretical underpinning of the composition of our team and leaders-directed strategy.
In combining process with outcome evaluations, we collected data using a wide range of methods as recommended by several authors [5, 15]. We developed a questionnaire, derived from the components of the improvement strategies. We undertook extensive pilot work to ensure that all important components of the strategies were adequately captured in questionnaire measures. We then pre-tested the questionnaire among 90 nursing students.
An important issue concerns the use of ‘as-received’ analysis as distinct from the conventional ‘intention-to-treat’ analysis used in the analysis of RCTs. These analyses differ not only in terms of the estimation procedure, but also in terms of the underlying research goal for a specific study. This study is an example of explanatory research, and the as-received analysis was therefore appropriate. Our as-received analysis was illuminating but also lost the benefits of the original random assignment, and therefore the potential for bias exists. This should be considered when interpreting our results .
A limitation of our study concerns the low questionnaire response rate of 48%. This may be a potential source of bias. We didn’t test the psychometric properties of the questionnaire. For these reasons, our findings from the nurses’ experiences analysis need to be interpreted with caution.
This is the first prospective study that has assessed the working mechanisms of two HH improvement strategies, demonstrating the added value of specific aspects of social influence and leadership. This is an important finding for hospital administrators and ward managers who want to improve nurses’ HH behaviour. Currently, most strategies focus on the individual and the organization. Including activities aimed at social influence and leadership could be a promising development. Our results point to: addressing each other in case of undesirable behaviour, support from colleagues, accountability, goal setting, and active commitment of the ward manager. The methodology of our team and leaders-directed strategy can probably be used to improve team performance on other patient safety issues as well.
Our study points to ways in which the design of process evaluations within randomised controlled trials may be conducted. Our initial results require affirmation by further process evaluations of HH improvement strategies. Further research is also needed to examine the different aspects and impact of social influence and leadership. Finally, future research should explore the influence of hospital culture.