Table 4 Empirical articles

Strengths:

• As an organizing device for highlighting differences between intervention and implementation studies.

Issues:

• Variable interpretation of elements/sub-elements relative to the model, which implies its lack of definitional clarity and/or need for more direction in its application.

Strengths:

• Novel in using the framework to highlight differences between original and translational trials.

Issues:

• Just a description of a protocol; no data.

Strengths:

• Finds evidence of test-retest reliability for scale measuring PARIHS elements suggesting stability of constructs.

Issues:

• Item wording taken directly from Swedish translation of PARIHS, with some respondents confused about the meaning of related survey items.

• Variable interpretation of PARIHS elements; e.g., 'task-oriented' role was placed on the negative/low end of their rating scale.

Strengths:

• One of only two articles included in the synthesis that attempts to develop an instrument based on PARIHS.

Issues:

• Only test-retest, and follow-up conducted after >4 weeks, too long for test-retest; categorical ratings were dichotomized to assess reliability with Kappa, instead of using a measure appropriate to categorical ratings.

Strengths:

• Conclude that 3 PARIHS components apply very well to translation of pain management evidence into practice.

Issues:

• There appear to be 2 types of roles not differentiated in the model highlighted by this review: 1) Those in pre-existing roles, like clinical nurse specialists or nurse managers, which are a built-in facilitator as implementation/change may be an inherent part of what they do; 2) Someone on a project that is appointed to that interim role.

Strengths:

• Systematic review.

Issues:

• Qualitative/observational review only, with no inclusion of interventional studies.

• No data tables and lack of information re: methods for analysis and interpretation.

• Focused on pain management literature which is very sparse.

Strengths:

• Demonstrated flexible use of model whereby user chooses only those elements that applied to the target at hand, i.e., levels of Networks.

• Authors viewed findings as consistent with PARIHS, which emphasizes need for context-sensitive facilitation activities.

• Results suggest that PARIHS has potential as a guide for evaluating other knowledge networks.

Issues:

• Highlighted the need to add focus on impacts or results to the framework.

• Authors focused on understanding the knowledge exchange dimensions at the element level without exploring them at their sub-element level.

Strengths:

• Explicitly defined outcomes as they relate to PARIHS.

• The network level allowed context which can be seen as the resources or opportunities for effective communication and infrastructure opportunities like web cast.

Issues:

• Limited project with little data or clear logic for how results or conclusions were derived, and how the PARIHS elements were associated with the outcomes.

Strengths:

• Indirect support for facilitation being correlated with context. Higher RU and lower rate of adverse events associated with positive context (culture, leadership, evaluation).

Issues:

• 'Two unanticipated findings were that the concepts of innovation and facilitation had no significant influence on nurses' research utilization' (p S35).

Strengths:

• One of only 2 studies that use quantitative models to test influence of specific context and facilitation measures on research utilization.

Issues:

• Variables loosely mapped to PARIHS along with other non-PARIHS variables.

• Complex constructs measured using single-items that were selected post-hoc. RU (dependent variable) also calculated based, in part, on contextual variables (e.g., autonomy, organizational slack). Authors note that perhaps facilitation was '... not operationalized ideally.'(p. S35)

Strengths:

• 'The model is helpful in identifying the important elements within the practice setting that need to be in place in order to foster the uptake of evidence into practice. It shows that evaluation is an important component of the context for change and indicates that multiple methods and sources of feedback should be incorporated into an organization's evaluation framework.' (p. 4)

• Authors operationalized all three main components of PARIHS - apparently choosing only sub-elements that seemed to apply to their objective.

Issues:

• '...previous descriptions of the model do not specifically address what indicators are appropriate for evaluating nursing systems and services or how to use performance measurement and feedback to design and evaluate practice change.'

Strengths:

• Provides another example of the flexible and selective use of PARIHS and additional thoughts on the evaluation component.

Issues:

• Model yet to be applied/tested.

Strengths:

• Rationale for use based on: 'Embraced by academics, clinicians, and managers because it resonates with their own experience' (p. 85).

• Supported PARIHS components; authors thought overall outcomes probably due to leadership, evidence, and facilitation and felt one of six hospitals did not implement due to 'clear' lack of leadership.

Issues:

• Noted by authors as not including underlying motivations (e.g., relative advantage or dissatisfaction as tension for change) related to protocol/intervention.

• Variable definitions of elements.

Strengths:

• At least to some extent, assessed the nature of the framework and needs for refinement.

Issues:

• Low-level qualitative case study; some details of methods unclear (e.g., what proportion of participating hospitals' nurses attended); convenience sample.

• 'Many of workshop participants did not work in practice location...where the protocol was to be implemented... ' (p. 91).

Strengths:

• Facilitation, context (leadership, evaluation, and culture) were significant at the specialty level in addition to other contextual measures; e.g., nurse-to-nurse collaboration (p. S7).

Issues:

• 'Variation in research utilization was explained mainly by differences in individual characteristics, with specialty- and organizational-level factors contributing relatively little by comparison...' (p. S7).

• Results imply that PARIHS should be extended to include other contextual variables not explicitly included in the current version (e.g., nurse-to-nurse collaboration).

Strengths:

• One of only two studies that use quantitative models to test influence of specific context and facilitation measures on RU.

• First demonstration of multi-level modeling approaches.

Issues:

• Variables loosely mapped to PARIHS along with other non-PARIHS variables.

• Complex constructs were measured using single-items that were selected post-hoc.

• RU (the dependent variable) is calculated based, in part, on contextual variables (e.g., autonomy, organizational slack).

Strengths:

• Most comprehensive attempt to operationalize context CAI appeared to be successful for practitioners to generically reflect on their practice.

• Provided useful information for potentially refining the framework in terms of enhancing the meaning of context.

Issues:

• Findings were said to suggest that some contextual characteristics are 'less theoretically robust than thought.'

• Findings included 'factors' not consistent with the current structure of the four sub-elements under Context; variable placement of sub-sub-elements.

• Tool seems to be especially useful for a holistic practice focus rather than for task-specific implementation.

Strengths:

• Rigorous empirical development.

Issues:

• Need for further research regarding validity, reliability, and usability in other settings and with different clinical topics.

Strengths:

• In the literature, 'Six contextual factors were identified as having a statistically significant relationship with research utilization, namely the role of the nurse, multi-faceted access to resources, organizational climate, multifaceted support, time for research activities and provision of education' (p. 622).

• 'The contextual factors could successfully be mapped to the dimensions of context in PARIHS (context, culture, leadership), with the exception of evaluation' (p. 622).

• Authors 'believe that PARIHS is a fruitful starting point for better understanding of the impact of context on research utilization and more studies should explore this area of inquiry' (p. 632).

Issues:

• 'No single included study was assessed to be of high methodological quality' (p 626).

Strengths:

• A comprehensive review of the literature.

Issues:

• The basis for mapping of contextual variables found in the literature onto the PARIHS framework was unclear.

Strengths:

• Authors report general match of empirical findings to PARIHS.

Issues:

• Empirical findings didn't map to many sub-elements.

Strengths:

• Systematic review with very thorough search strategy and clear inclusion/exclusion criteria.

Issues:

• Lack of clarity about how independent variables were measured (i.e., how factors were to be mapped to PARIHS elements).

• Focus seemed primarily on user's characteristics in general, not on role as an explicit facilitator, and not explicitly on successful implementation.

Strengths:

• Used 1998 version of PARIHS but content highlighted in case study confirmed later PARIHS modifications: i.e., use of evidence not just from RCTs (e.g., from program eval), use of local data, and patient 'experiences.'

• Brainstorming around E, C, and F seemed to illustrate dynamic interactions among these elements, as aspects of one were reflected in another.

• Were able to use the framework to analyze their current situation.

• Noted the importance of patient engagement.

• Used along with other models of practice and evaluation.

Issues:

• Needs more emphasis in the model on 'motivating multi-disciplinary groups of staff to change and accept new ideas' (p 230).

• Importance of patient engagement was highlighted but unclear if is part of both evidence and/or culture.

Strengths:

• With open, albeit limited case study format, able to identify important 'additional' components beyond the cited 1998 model.

Issues:

• Lacks sufficient details about methods to evaluate changes, e.g., re: services; source of recommendations; interviewees, data analysis or results.

Strengths:

• They added 'fit' under context; i.e., 'Initiative fits with strategic goals and is a key practice/patient issue' (p. 922).

• Added 'Receptive' to sub-element of context; within that sub-element, added 'Resources - human, financial, equipment - allocated' as well as - 'Professional/social networks '(p. 922).

• Adequately connected the three key variables of the PARIHS framework to the barriers and influences of getting evidence into practice.

Issues:

• Despite Strengths, 'the findings also suggest that further consideration is required to ensure that the PARIHS framework is appropriate, comprehensive, and accurate' (p. 921).

• Criteria for inclusion and related meanings not always clear.

Strengths:

• Presentation of findings was well organized and categorized by themes that emerged in the data.

Issues:

• Conclusions that findings confirm PARIHS did not seem adequately grounded.

• No definitive a-priori measure of success and projects studied were complete yet.

• Authors acknowledge study limitations as: 1) small sample sizes, 2) data credibility limited due to self-report, 3) potential bias as participants may have been 'evidence-based practice enthusiasts' (p. 920) and 4) successful implementation was 'defined largely by its absence than its presence' (p. 920) in the study.

Strengths:

• 'Desired outcomes can be achieved when the context is less than ideal but outcomes are generally poor when attention to both context and facilitation are lacking' (p. 137).

• Authors learned the utility of PARIHS whereby new strategies can be developed.

• Used as a diagnostic tool for retrospective study where interventions didn't work very well.

Issues:

• Variable definitions of elements; and variable placement of sub-elements.

Strengths:

• Reinforced the importance of needs assessment of evidence, context and facilitation factors prior to the initiation of intervention implementation.

• PARIHS model utilized to organize data and link empirical data to the model to demonstrate how it can inform real life situations.

Issues:

• Authors linked factors to outcomes globally, but not within sites, which would have helped understanding of the data, given the variable findings noted (there seemed to be an overlap of some barriers and facilitators).

Strengths:

• The study affirmed the importance of facilitation as a distinct role with a number of potentially crucial behaviors and activities.

• Highlighted the importance of the task-oriented purpose.

• Role of individual facilitator characteristics found to be important.

Issues:

• Categories under skills/attributes in PARIHS don't provide some of the details found in the study, nor does the framework adequately highlight the mixed facilitation approach found in primarily such task-oriented projects.

Strengths:

• Use of a stimulated recall method gave interviewees several opportunities to continue recalling and adding to the richness of the qualitative data while further commenting, affirming or challenging the analysis

Issues:

• Authors noted the evaluation was 'both small scale and reliant on self-report data, thus potentially limiting its generalizability. Additionally, its purposively sampled participants represented a specific perspective and are likely EBP enthusiasts, particularly in terms of facilitation' (p. 12).

• Only external facilitators were interviewed.

Strengths:

• Results support the role of the three main components (evidence, context, facilitation) in uptake of quality improvement initiatives.

• Reasonable to use PARIHS to help frame discussion of findings.

• Highlighted strong role of internal leadership.

Issues:

• Difficult to tease out sub/elements of PARIHS because of dynamic interrelationships between elements.

Strengths:

• The only study that used PARIHS to frame results from a process evaluation within a randomized control trial.

Issues:

• PARIHS was used loosely as an organizing framework to present results and authors did not reflect back on utility of PARIHS.

Strengths:

• Results '...demonstrated empirical support for the validity of the context dimension of the PARIHS framework.' (p. 156) Showed a positive relationship between RU and context (culture, leadership, and evaluation) and further demonstrated a positive incremental relationship between RU and rank ordering of context from low to high.

Issues:

• Unclear implications for PARIHS definition of context, given how narrowly measured/defined out of unrelated dataset.

• Unclear implications for definition of facilitation as it relates to inherent leader roles, such as a nurse manger.

Strengths:

• Clear presentation of hypotheses and results.

Issues:

• RU was derived, in part, from contextual variables including autonomy and organizational slack, with rationale for doing so unclear.

• Authors interpret results as validation for PARIHS but also recognize that 'only one of the PARIHS components - context - was used, and [they chose] only one variable to characterize each contextual dimension' (p. 158).

• RU and context variables were selected, post hoc, from a dataset developed for another study.