The use of evidence in English local public health decision-making: a systematic scoping review

How has the structure of local public health decision-making changed?

Since 2013, local public health leadership has mainly transferred to local authorities (LAs), whose public health remit now includes commissioning services across most aspects of public health (see [12] for a full outline). LAs are also responsible for improving health determinants and reducing health inequalities across nine key areas impacting on population health including early years, education, planning, housing, leisure and communities [13]. This was said to be emblematic of a “social model of health” [14], with LAs having an interest, and potentially an ability to influence, most local public health activities and many local determinants of health. Newly created health and wellbeing boards provide some strategic leadership, support, and coordination on the response of LAs to local health challenges. Most other local health services are commissioned through local clinical commissioning groups (CCGs; which replaced primary care trusts (PCTs); although some aspects of public health are retained here).

The transition to new health decision-making structures described above is marked by heterogeneity (e.g. [16]), which is in part reflective of existing differences in local authority needs and practices [17]. The impact of this heterogeneity on public health decision-making practices and actions is difficult to quantify; although, there have been some tangible actions common to all areas. First, all local authorities were expected to appoint a Director of Public Health (DPH) responsible for the public health budget and creating a staff of public health consultants. Second, local authorities were jointly tasked with producing/ updating Joint Strategic Needs Assessments (JSNAs), whose purpose is to describe the current and future health needs of the population. A third tangible action was the creation of Health and Wellbeing Strategies (HWSs), produced by HWBs, which present a response outlining strategies and services that would be commissioned to address the needs identified within JSNAs [18].

Implications for evidence use

The transition to a new landscape of decision-making is perceived to have progressed reasonably smoothly in some quarters [19], even if future challenges have been identified including anticipated further squeezing of public health budgets. However, this standpoint is not universal across professionals and academics working in public health, and more recent evidence highlights substantial levels of concern about ongoing risks and severe challenges to the local delivery of public health [20].

Structural changes have implications for the way in which research evidence is used in public health decision-making. For example, the appointment of DPH exposed differing levels of commitment between those LAs who followed DH guidance and created new senior directorship posts for DPH and those with different interpretations [21]. Furthermore, reflective of the nature of LAs, final decision-making responsibility ultimately now falls with the elected members, not with officers [22, 23]. Consequently, public health is now exposed to a culture that is more likely to be shaped by political and legal constraints [24] and from an evidence-use perspective, an increased political element to local decision-making may have implications in terms of the type and format of evidence needed. A political dimension can also make public health priorities vulnerable to rapid changes due to transitions in power, and producers of evidence will need to respond quickly to support new directions or provide evidence to justify existing activities. Reductions in public health grants supporting the extensive public health remit of LAs [14] and the different interpretations around how ring-fenced public health monies should be spent [25] are also likely to influence the evidence required.

Within the extensive literature on evidence-informed public health policy-making more broadly, a pattern of underutilisation of research evidence persists despite its abundance [26, 27] and despite the recurring cycles of prioritisation by policy-makers [28]. Common barriers to evidence use include perceived limitations in the relevance, importance and credibility of evidence [24, 29], including concerns around the application of evidence generated in other settings [24, 30]. A previous review concluded that few studies actually reveal the process through which research evidence is used in public health decision-making [31], and how this sits within broader knowledge-utilisation frameworks. In addition, the use of the word “evidence” is not necessarily synonymous with drawing upon a robust body of research evidence [32]. Where researchers have started to examine these issues in the post-2013 English landscape, they have indicated continued difficulties in using research evidence. In Beenstock and colleagues’ [18] analysis of 47 HWSs, they found that very few (5) referred to research evidence published in academic journals, just three referenced NICE guidance, and none directly referenced a Cochrane systematic review. Furthermore, research evidence was used more commonly to demonstrate evidence of need than to demonstrate evidence of effectiveness.

The aim of the review presented here is to offer further insight on these issues through mapping the use of research evidence in public health decision-making at a sub-national level, and where possible to compare patterns of evidence utilisation before the reconfiguration of public health services (2010–2012/13) and afterwards (2013/14–2016). Unlike previous reviews, this review adopts a broad set of inclusion criteria around study design, with both observational and experimental studies included, but a narrow scope in terms of geography and date range. Nevertheless, the patterns of evidence use and the obstacles that decision-makers in local areas face when sourcing and using evidence are expected to have a much broader resonance and be of interest to readers outside England. This study was conducted by a team of systematic reviewers who are interested in the contribution of systematic reviews to public health decision-making in England, and how this evidence contributes alongside other sources of research evidence and public health intelligence more broadly.

Search strategy

Four databases were searched for studies published between 2010 and 2016 (June) (PubMed, HMIC, EconLit and Scopus) and specific search strategies were designed for each (see Additional file 1). In addition, a manual search of databases was carried out in order to find potentially relevant studies. A bibliographic database was created in EPPI-Reviewer 4 to store and manage the references [33], and data were extracted into Microsoft Excel.

Assessment of eligibility

Titles and abstracts of the documents retrieved in the searches were independently screened by two reviewers to determine eligibility (full inclusion criteria provided in the Additional file 2). Included studies were England-based studies published from 2010 onwards that were focused on public health decision-making in local (regional or sub-regional) areas. The search strategy was limited to studies that included “public health” (or “health promotion”) in their title, abstract or keywords. Public health decision-making was defined broadly as that which aims to promote and protect the health and wellbeing of groups, communities and populations, mirroring broad definitions employed in previous reviews [31]. We did not extend this definition to explore decision-making around the social determinants of health (e.g. education or employment inequalities) except when this was described as taking place specifically within the context of public health (see [34] for an example where this took place). Included studies focussed on decision-making involving (i) local public health services or (ii) local public health prioritisation (using LA’s commissioning responsibilities [12] as a guide where there was any ambiguity), or (iii) explored local decision-making among professionals working in public health. Studies should have directly included decision-makers and a focus on the process of evidence use in decision-making. Studies published during 2010 and onwards were selected for inclusion. This date range was selected to provide evidence of research use practice immediately before and after the enactment of reform to decision-making structures in England (and where possible to contrast these); 2010 also coincides with the publication of the first systematic review by the Cochrane Public Health group, a recognition of the need for greater applicable evidence on specialist complex community-based interventions to contribute to the evidence-based policy movement. Where other systematic reviews were encountered, these were examined primarily as a further source of studies. We also excluded studies that did not directly involve observations of decision-makers (including [18]). Studies fulfilling the inclusion criteria were selected for full-text assessment, after which a new independent assessment was performed. Disagreements were resolved through discussion between the reviewers.

Data extraction and synthesis of the results

After piloting, we extracted information from all included studies. We did not undertake formal quality assessment of the studies since the aim of the review is to map the literature in this area and to lay the groundwork for primary research and more detailed synthesis (if supported by the data). This means that we did not formally employ existing taxonomies of implementation to structure or guide our synthesis; although, our aims align with an ambition of understanding the processes of translating research into practice (developing process models) and understanding influencers on evidence use (developing determinant frameworks) [35]. In this systematic scoping review, our synthesis methods were confined to a narrative, configurative approach [36] and we intended to provide a descriptive account of the main recurring themes. We followed five stages: (a) initial coding the text by producing preliminary textual descriptions of studies and their findings in a tabular format (see Additional file 3); this also involved grouping the studies according to their characteristics (e.g. setting and stakeholders) in order to understand the characteristics of the body of literature and to observe emerging patterns in the data; (b) further inductive coding of the textual summaries and identifying key preliminary themes and their recurrence across studies; (c) developing a framework for arranging groupings and clusters of studies according to the themes and exploration of these within and between the studies; (d) further generation of analytical themes through attempting to develop a common rubric to describe these findings; (e) finally, although we did not formally quality assess the robustness of individual studies, we did consider the completeness and applicability of evidence, the robustness of the synthesis methods and the quality of evidence in terms of its relevance to the ambitions of the review, and this is presented in our discussion [37–39]. This process was carried out by two of the authors (DK and ARG), and any disagreement was also resolved by discussion.

Locality of evidence

Many studies suggest that the research evidence base does not match the evidence needs of decision-makers with respect to the locality of evidence [34, 40–44]. This was identified as an underlying reason why decision-makers consult with sources other than research evidence [34, 40, 41]. In one of the included studies, having local evidence was said to “trump” other forms of evidence, even if this is of lower methodological quality [41]. For example, Wye and colleagues [41] contrast the low impact that a briefing based on academic research evidence had among a committee considering commissioning telehealth, when compared to experiential local evidence based on eight service users. Despite the methodological robustness of the former, the latter evidence refuted the findings of the former and was instrumental in increased positivity towards telehealth among the committee [41]. A similar theme was shared among other studies where contextually relevant evidence was prioritised in decision-making [34, 45], including local public health intelligence [34, 42, 46]. Problematically, decision-makers may emphasise the uniqueness of their local areas and their public health challenges, as opposed to highlighting commonalities with other areas [40].

In some studies, “local” denotes geographically bounded evidence that would allow for service planning (evidence of need) or benchmarking or drawing comparisons of effectiveness with neighbouring areas (based on evidence of effectiveness) [34]; in other studies, “local” appeared not to be strictly geographically bounded but involved a broader consideration of evidence that was contextually salient [42].

Decision-making and local contexts and local conceptualisations of public health

Evidence that was aligned to the local political ideology [40, 47] and could support broader organisational aims (where benefits could be visible beyond public health alone [43, 48]) was most useful to decision-makers. Philips and Green’s study, for example, which aims to understand the processes and practices of local public health decision-making, highlights how decision-makers construct unique identities for their organisations that translated into specific processes for action, which in turn led to perceived distinctiveness in terms of information needs [40]. One study found that services and strategies did not always need to be supported by evidence if these initiatives were congruent with the current direction of the organisation [49]. This political ideology was shaped by the strong affiliation with accountability to local populations, and sometimes, this could lead to conflicting perspectives as to which public health issues to prioritise. Marks and colleagues [14] provide a compelling example from a HWB board member highlighting the tension between evidence and local accountability: “…it was pointed out that lots more people die of smoking related conditions than they do of drug related conditions, alcohol and drug related conditions, but nobody complains to me about the next door neighbour smoking. But they will complain about the drug dealers on the corner and the alcohol, noise and abuse and all that stuff, which has a big effect on peoples’ lives. It ripples out on the community. But they’ve got a point, but we’ve got a point as well. (p1200)” Political considerations and public perceptions directly shaped elected members’ decision-making, even where there existed robust evidence to suggest an alternative course of action, as articulated in one study where a DPH found challenges in “advocating something [in this case minimum alcohol pricing] at a population level that is just not palatable from a political” ([44], p32).

One study, which took place after 2013, also found that the local political ideology and the type of case that was required to made around a given public health issue changed over a short duration [49], which may impact on the type of evidence required to support, or in some cases challenge this.

The value of expert opinion

The need for evidence that was transferable to the local political context may also be an underlying explanation as to why expert opinion, advice, and experience were found to be highly utilised and valued in several studies [42, 46, 50, 51]. Expert opinion and advice was sometimes utilised more than other robust forms of evidence, including NICE and other national guidelines [41, 46] and systematic reviews and meta-analyses [42]. Experts often included professional colleagues, suggesting that expertise may be again related to experiential knowledge [46].

There were some indications of conflicting evidence-use patterns, with usage patterns not necessarily reflective of importance. One study found that NICE guidelines were one of the most frequently used sources of evidence, but practice guidelines (including NICE guidelines) were not universally regarded as the most useful [42]. The sample composition and the range of roles and specialisms may have partially accounted for some of this discrepancy between perceptions of usefulness and frequency of usage; although, the study did not publish disaggregated results that allow this to be interrogated further. This study also suggested that expert opinion was one of the most frequently utilised sources of evidence, but one that was not always highly valued [42].

Social network analyses suggest that the most powerful influencers on public health decision-making were able to form a bridge between local authority and NHS organisations [51]. In contrast, academic bodies and individual academic experts were rarely considered to be influential in their own right [51], perhaps reinforcing the findings of other studies which emphasise the disconnect between academics and policy-makers with regards to what constitutes useful and robust knowledge [52] as well as the broader disconnect between cultures of public health evidence generation and evidence use [53, 54].

Evaluation evidence, experts and localism

A number of studies suggested that some of the most influential evidence on local public health decision-making was generated through local evaluation activities [40, 41, 55]. Such data meet decision-makers’ requirements around the locality of evidence and are generated within the local political context therefore meeting requirements around the transferability of findings [40, 47]. One study reported that evaluation evidence, even if based largely on an anecdote or a very small number of cases, could sway senior decision-makers’ views [41]. Despite the local evaluation evidence being generally regarded as useful for decision-makers, some studies also identified that this form of evidence had limitations particularly around the timing and intended usage [45, 56].

The degree to which the demand for and usage of evaluation evidence, evidence from experts, and locally embedded evidence are linked trends is not clearly expressed in the literature. Frequent deployment of local experts in public health decision-making may be as a result of their ability to blend inter/national sources research evidence with knowledge gained from local evaluation/experience. Oliver and DeVocht [42] identified a potentially broad knowledge translation role for reliable experts in public health decision-making in explaining the importance of findings, translating evidence into contextually “comprehensible statements” and “providing clear direction for decision-makers”. This suggests that the remit of experts in this case could extend beyond the usual boundaries of knowledge translation [57] to a much more directive role. The data they collected in the study did not illuminate the characteristics or motivations of existing experts or precisely the type of evidence where a knowledge translation role was most desired. However, they did note that decision-makers expressed a desire to utilise existing data and evidence through improved interpretation, suggesting that further refinement of knowledge translation practice was desirable [42].

Demand for economic evidence falls short of the mark

Several studies suggested that decision-makers would find evidence around the economic impacts of interventions useful, but that this was not always available [14, 45, 47, 49, 53, 55]. In some cases, an increased demand for evidence around cost and benefit implications was directly related to austerity and the prioritisation that local decision-makers would now have to undertake as a result of reduced, and in some cases, unprotected budget [14]. However, not all methods of prioritisation based on economic methods are viewed as being sensitive to local contexts and some are viewed as excessively technocratic [14].

Two studies provided further disaggregation around the type of economic evidence that decision-makers valued the most [48, 58], which included evidence of impact across the remits of local authorities beyond traditional departmental siloes [48] as well as evidence of the way in which existing local services and structures were likely to influence cost effectiveness [58]. Marks and colleagues [14] provide an outline of different potential approaches and tools that can support decision-making and prioritisation and map these onto different stages of decision-making. For example, the first decision-making stage involves reaching an agreement on public health objectives; here, relevant information includes JSNAs and other public health intelligence, while other decision-support methods that may be employed include broad stakeholder consultation and involvement. Further stages of decision-making outlined include identifying options and resources for reaching the objectives, identifying measurable criteria, deciding on preferences and making choice. This study provides one of the few examples where authors have tried to understand decision-making as a series of sequential processes. However, they do not present explicit information on levels of awareness or usage of the different decision-support tools, or how much resonance the stages of decision-making hold across the three localities included in the study.

Solutions, facilitators and barriers identified in the literature

One study identified evidence briefings based on systematic reviews as a promising but currently underutilised approach [40] while a further study suggested methodologically robust case studies of local innovation as being a way of enhancing the usefulness of academic research evidence for policy-makers [34]. Other studies identified strengthening networks and communications between evidence producers and evidence users, including recognising the role of interpersonal relationships in determining evidence use and influence, as being important in working towards meeting the evidence needs of public health decision-makers [42, 48, 51, 52]. None of the studies specifically mentioned “co-production” as a term; although, some recommended greater collaboration between generators of evidence and evidence users [52, 56]. For example, one study suggested that different “cultures of evidence” were not incommensurable in collaborative work between evidence producers and users when the gains to be made from collaborative working were made clear to all parties involved [56].

Named barriers to evidence use included access [41], capacity to analyse and interpret evidence [42], availability and relevance [58] and knowledge of different sources and types [42]. In terms of economic evaluations, another study also highlighted the difficulties to analyse the return on investment due to the organisational culture, capacity, the status of services, or administrative or political inertia [48]. A final study provided indicative evidence that the use of evidence was perceived as being tied to other decision-making processes (such as strategic partnership working) that were perceived as “bureaucratic” [59].

Summary

This systematic scoping review identified 21 studies (23 papers) that included a specific focus on the process of English public health decision-making from 2010 onwards. These studies allowed us to identify some recurring themes around the role research evidence plays in decision-making, as well as some of the major influencers on the use of research evidence use. As we discuss below, few studies provided descriptive accounts of the way in which research evidence is used in practice, which precluded the development of process models representing the way in which research evidence is translated into local public health practice [35]. Similarly, no studies directly contrasted the use of research evidence before and after the 2013 reorganisation of public health structures, although some did describe the changes in some of the influencers of evidence use, including the changing way in which priorities were set and the influence of local accountability and politicisation [14, 41]. Therefore, with regards to meeting our aims of exploring evidence-use processes and understanding how these processes may have changed as a result of the reforms described earlier, the body of available evidence did not allow us full insight.

Consistent across the studies was a tendency to describe research evidence as being underutilised in decision-making but generally to be absent of sufficient detail around the type, process and context of the decision being made that could illuminate a way forward. Many of the suggestions being put forward in studies tend to be from the perspective of researchers, and arguably, this has led to a pre-occupation with identifying different forms of evidence, as opposed to different form and stages of decision-making where evidence is needed but is currently not, or under-, utilised.

Complexity and heterogeneity in decision-making and public health

Heterogeneity in process and structure is ostensibly a defining characteristic of public health decision-making post-2013. Local decision-makers are keen to emphasise the uniqueness of their areas [40], as opposed to identifying commonalities. It is important that we acknowledge this heterogeneity in processes and structures as part of the ecosystem in which public health services and interventions operate, and in which evidence is used. However, it is also important that we attempt to understand and respond to this heterogeneity. Research has not advanced to understanding the landscape in terms of typologies and groupings of public health decision-making processes and structures. Developing such typologies could facilitate understanding broad patterns of evidence use (and need) within these groupings and ultimately bridge a gap between evidence use and generation. Similarly, the demand for local evidence uncovered [34, 40–42, 62] should be interrogated further.

A changing balance of factors influencing decision-making

It is difficult to fully establish how the transition into local authorities has influenced the politics of decision-making. Certainly, some studies suggested that political influence had disrupted previously established flows of evidence into different stages of decision-making [24, 40, 48, 49], which may impede on evidence-based policy-making. Some studies suggested that a more politicised environment influenced priority setting [14, 47] and that the local political ideology and the case required for a public health issue changed over a short duration [49]. This may support the idea that more politicised environments could foster cultures of “short-termism” in a priority setting [63] and indicate the way in which a change in administration after an election may lead to rapid changes in priorities; both of which present challenges to evidence production.

The demand for economic evidence, a theme discussed in this review, is likely to strengthen against the background of funding cuts to public health budgets outlined in the introduction. Meanwhile, the role of local experts in a knowledge translation role is also likely to be sustained if research evidence continues to hold either low resonance or transferability or usability among end users. However, there is much left to learn about the role that experts do indeed fulfil, given that one study suggested that experts take a directive role. Regarding our own concern around the role of systematic reviews in public health decision-making, it is notable that almost half of policy actors in one study viewed meta-analyses as useful; although, none reported using this evidence regularly [42].