The database search produced 1816 results, after duplicates were excluded. After screening by title and abstract using the inclusion and exclusion criteria, 184 articles were identified for full-text screening, 23 of which were selected for data extraction (Fig. 1). A manual search for relevant studies conducted after the database search, based on searching key journals (Social Science & Medicine, BMJ Quality & Safety, Implementation Science, Sociology of Health & Illness) and suggestions by PAG members, including book chapters, identified eight additional studies for inclusion, meaning 31 studies were reviewed.
The quality of studies was assessed using the Critical Appraisal Skills Programme (CASP) Qualitative Checklist , which includes nine questions for assessing the validity of study findings numerically and a tenth, non-quantifiable question for judging the overall relevance or value of the research (recognising that the checklist represents a series of inter-related questions for assessing study quality). After reviewing all of the studies using the CASP checklist, we agreed that seven studies should be considered lower quality studies. This assessment took into account how each study performed against the numerical questions and making a value judgement about the quality of each study as a whole (question ten). Those seven studies were excluded from the thematic analysis due to low confidence in the validity of results (studies shown in ‘greyed out’ rows in the data extraction tables).
A summary of the studies’ characteristics is provided in Additional file 3. The majority of the studies was conducted in the UK (14), followed by Canada (5), Australia (5), the USA (3), Sweden (1) and Italy (2). An interview-based study , of lower methodological quality, included participants from Australia, Denmark, Ireland, the Netherlands, Slovenia, Switzerland, Spain and Sweden. The types of innovation examined were technological innovation (6), staff and patient involvement in quality improvement (4), responses to clinical guidelines or tools (7), organisational innovation including quality improvement programmes (6) and technology assessment and priority setting (8). The studies covered acute care (16), primary care (11), commissioning (8), health and social care (2) and mental health (1). Nearly all (28) of the 31 studies employed qualitative interviews. In combination with interviews, these studies used observations (9), documentary analysis (9), focus groups (4) and surveys (5). Of the remaining three studies, two relied on observations and one did not specify data collection methods within a case study approach. Research evidence was the most cited form of evidence in decision-making on innovation (19 studies); other forms of evidence were professional experience (15), local data (12), national guidance (7), translational information, e.g. education/ summaries (4), patient involvement (3) and expert opinion (3).
There were 24 studies of sufficient quality to be included in the thematic analysis. Thematic analysis examined how processes at different levels (professional, organisational, local system) influenced the use of evidence in decision-making on innovation (Additional file 4).
Professional level processes influencing evidence use
Preferences for evidence
Preferences for evidence varied by professional group and across health care sectors. Service payers (commissioners) drew on a range of evidence, including alternative evidence such as patient stories, and prioritised local need for innovations over research evidence [14, 48]. In the acute sector, nurses tended to combine practical (‘how to’) and scientific (‘principles’) knowledge, while medical professionals placed greater weight on the latter . In primary care, general practitioners (GPs) did not necessarily privilege scientific evidence; research-based studies were contested by GPs as results were weighed up against their knowledge of patient need . Evidence can be given different meanings by different stakeholders resulting in uncertainty about whether evidence was lacking, was not of good quality, or was limited .
Some studies highlighted that decisions to develop and adopt innovations reflected professional interests. A study of surgical innovation found that surgeons ‘spoke for’ patients by introducing new techniques that would ‘make sense’ for them, even though supporting data were lacking . A study of remote care (telecare) found that evidence was actively constructed and adapted to fit managers’ agendas . There was recognition that evidence could be ‘gamed’ whereby evidence was found to support decisions that had already been taken . Professional interests could influence how different stakeholders responded to evidence. A primary care study of the failure to implement externally mandated rules, National Service Frameworks, was linked by GPs to concerns about the accessibility of evidence (e.g. document length, complexity, local applicability), but the authors suggested these were mere ‘constructions’ because the frameworks did not fit in with GPs’ professional identities .
Power dynamics between different professional groups influenced evidence use. A study of interventions to improve prescribing practice in primary care found that managers leading the programme privileged scientific evidence, while attempting to marginalise GPs’ clinical and experiential knowledge . Similarly, managers used evidence to decline clinicians’ ‘unreasonable’ requests for innovation in the area of robotic surgery . Conversely, a study of committees considering technology coverage found that clinicians, especially those with powerful personalities, were able to influence the committees . Even where decision-makers agreed on the evidence base for an intervention, there could be disagreement based on practitioner and patient judgements about how such evidence should be used to make decisions and/or change services .
The stakeholder feedback indicated that professional processes influenced decision-making. They confirmed that professional credibility of those presenting evidence, as well as clinical leadership and ‘soft’ persuasion skills and relationship-building (including ‘endless discussions’), encouraged evidence to be taken seriously and acted upon. There was recognition that preferences for evidence varied by stakeholder and therefore the same evidence often need to be framed differently to influence different stakeholders, particularly the needs of commissioners or funders of potential innovations, ‘as everybody has different buttons’. The ongoing process of building relationships during decision-making was more apparent in the stories of innovation shared in the focus groups than in the literature review (due perhaps to a lack of processual studies in extant literature) .
Organisational level processes influencing evidence use
Organisations contributed to assessing non-clinical aspects of innovation. Along with evidence of clinical need or effectiveness, budgetary and financial issues were important in decision-making [17, 49]. Organisations enabled stakeholder involvement in decision-making, including staff , which aided subsequent implementation . Stakeholder involvement in quality improvement projects, particularly patients and the public, was supported by effective communication channels and a ‘non-hierarchical’ environment for decision-making . Centralised approaches to decision-making, coupled with a lack of communication, inhibited evidence use by planners within regional health authorities in Canada . Organisations limited innovations proposed by clinicians and other stakeholders where evidence was lacking: funding for surgical innovation was cut by a hospital due to a lack of evidence on cost, safety and effectiveness, despite local surgeons’ perceptions that it improved patient outcomes and safety .
A number of organisational facilitators to evidence use in decisions about innovation were identified. In a study of technology adoption within hospitals, access to and use of research evidence in decision-making was enabled by organisational processes, including infrastructure redevelopment projects and an emphasis on collaboration . In a study of priority setting within a provincial health services authority, evidence use was enabled by strong leadership, a culture of openness and learning, and commitment to being ‘data-driven’ . The importance of research culture was borne out by a study of a multi-systemic therapy, where entrepreneurial leaders of adopter sites suggested that they could make decisions to adopt innovations more readily than non-adopters because they were more aware of the evidence base . Innovation was supported by creating leadership for change, which included marketing evidence of benefit and building a supportive community of practice . Another study highlighted the importance of involving both managers and clinicians in decision-making . The chairs of decision-making committees moderated the use of evidence type. A study of networks responsible for enhancing multidisciplinary cancer care found that some chairs steered the conversation more to scientific and technical themes at the expense of narrative perspectives .
Underlying organisational issues could act as barriers to introducing innovations . A lack of time, resources and pressures inhibited evidence use . A lack of authority to make changes to processes also influenced decision-making . In some contexts, organisations were not receptive to change. A study of telehealth services found that its spread was limited in two out of five cases by a lack of alignment between the adopting organisations’ values and managers’ agendas . One study suggested that those proposing innovations should ensure these were aligned with other activities already familiar to decision-making stakeholders . Another study found that involvement processes for enabling patient organisations to participate in funding decisions were inadequate for including patients’ experiences .
Organisational politics influenced the type of evidence accessed and how it was interpreted. The use of economic evaluation by committees making technology coverage decisions was limited by unclear relationships with resource allocators, an explicitly political decision-making process, and poorly specified decision-making criteria . A study of commissioners’ information use  found that organisational processes changed the original information gathered during decision-making (evidence was re-framed over time to suit competing agendas).
The stakeholder feedback confirmed that an innovation was more likely to be adopted when it was aligned with organisational needs, e.g. when it is a priority (including meeting external targets or initiatives) and it addressed a clear, practical problem. The focus groups elaborated on the influence of the decision-making approach taken in relation to innovations of different scales. There was recognition that large-scale change was difficult because a wide range of stakeholders were often involved and that evidence often showed both pros and cons. The stakeholders discussed different approaches to organisational decision-making; ‘autocratic’ as opposed to ‘democratic’ organisations were better placed to introduce change, but once a decision had been made, there was the challenge of getting a change accepted and having a culture that valued evidence was deemed important for this.
Local system level processes influencing evidence use
External pressures, including system restructuring , meeting policy targets  and budgetary constraint [7, 17, 59], influenced how evidence was used in decisions about innovation. The political context influenced decision-making , e.g. decisions needed to stand up to external scrutiny . Such pressures could lead to an emphasis on ‘what works’ in making adoption decisions over use of rigorous evidence . One study reported staff being overwhelmed when using evidence to make decisions about changing services due to competing priorities and variable managerial support during major external change . A context of austerity could encourage evidence to be viewed differently. To receive funding, home telehealth services needed to demonstrate savings or efficiencies as well as evidence of benefit . Due to the need to consider rationing of the health care system, another study argued that decision-makers viewed economic evaluation narrowly–based on budgetary impact and costs rather than cost effectiveness . Another study found that financial and resource issues facing commissioners made them more conservative when changing services in response to new national guidelines .
Pan-regional organisations influenced how evidence was used in decisions about innovation. On the one hand, such organisations had a downward influence on evidence use in local decision-making. A study of a collective primary care organisation showed how it influenced GPs’ prescribing practice by emphasising evidence that appealed to this professional group (i.e. improving quality through prescribing targets), while deemphasizing the contribution of the interventions to cost containment which appealed less to GPs . A national improvement programme was the source of evidence for improving ward productivity, which had a national organisational profile and established links with providers, aiding adoption . However, a regional health technology advisory group in Sweden had less influence on decision-making because it was not embedded sufficiently in local decision-making . On the other hand, an upward relationship from the organisational to local system level existed whereby pan-regional organisations helped legitimise local innovations or encourage disinvestment. Hospitals’ participation in a national improvement campaign afforded external validation of decision-makers’ opinions at a local level, aiding programme commitment . One Canadian study found that a regional body was used by a hospital to justify withdrawing funding for an innovation, based on a perceived lack of evidence .
Widening stakeholder involvement
Participation in external systems or networks enabled a wider range of potential stakeholders to inform decision-making on innovation. However, taking into account a range of external stakeholders’ views could hinder implementing innovations based on formal evidence alone; the politics of decision-making could be more important than evidence, including the assessment of likely public perceptions of decisions taken . Decision-making could be enhanced through the use of deliberative involvement processes enabling multiple stakeholders to participate .
The stakeholder feedback indicated that organisations at the local system level played an important role in enabling innovation. The backing of research organisations and other knowledge intermediaries, e.g. Academic Health Science Networks and CLAHRCs, provided a facilitating role–one participant referred to them as ‘ambassadors’ for innovation–that could help to bring together relevant stakeholders. The role of intermediaries in mobilising evidence for innovations by brokering social relationships came through more clearly in the focus groups than in the literature review, possibly because studies of knowledge mobilisation tend to consider implementation processes (which were excluded from the review) rather than adoption decisions . The focus groups confirmed the importance of the political context, especially perceived pressure to reduce or control costs, and the need for evidence for innovations to align with those setting the political direction.