Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Exchanging and using research evidence in health policy networks: a statistical network analysis

Implementation Science20149:126

https://doi.org/10.1186/s13012-014-0126-8

Received: 2 April 2014

Accepted: 4 September 2014

Published: 30 October 2014

Abstract

Background

Evidence-informed health policymaking is a goal of equitable and effective health systems but occurs infrequently in reality. Past research points to the facilitating role of interpersonal relationships between policy-makers and researchers, imploring the adoption of a social network lens. This study aims to identify network-level factors associated with the exchange and use of research evidence in policymaking.

Methods

Data on social networks and research use were collected from seventy policy actors across three health policy cases in Burkina Faso (child health, malaria, and HIV). Networks were graphed for actors’ interactions, their provision of, and request for research evidence. Exponential random graph models estimated the probability of evidence provision and request between actors, controlling for network- and individual-level covariates. Logistic regression models estimated actors’ use of research evidence to inform policy.

Results

Network structure explained more than half of the evidence exchanges (ties) observed in these networks. Across all cases, a pair of actors was more likely to form a provision tie if they already had a request tie between them and visa versa (θ = 6.16, p < 0.05; θ = 2.87, p < 0.05; θ = 2.31, p < 0.05). The child health network displayed clustering tendencies, meaning that actors were more likely to form ties if they shared an acquaintance (θ = 2.36, p < 0.05). Actors’ use of research evidence was positively associated with their centrality (i.e., connectedness).

Conclusions

The exchange and use of research evidence in policymaking can be partly explained by the structure of actors’ networks of relationships. Efforts to support knowledge translation and evidence-informed policymaking should consider network factors.

Keywords

Evidence-informed health policy Knowledge translation Social network analysis Exponential random graph model Policy network

Background

The use of research evidence to inform health policy decision-making has been identified as a means to improve the effectiveness of health policy decisions [1]-[3], to strengthen health systems [4], and to achieve universal health coverage [5]. Yet, evidence-informed policymaking occurs relatively infrequently for reasons related to the evidence itself [6], the policy issue being discussed [7], and because evidence `competes’ with many other inputs in a complex policy-making environment, including institutions, interests, ideas and external events [1],[3]. There is a growing understanding among stakeholders of the complexity of evidence-informed policymaking and thus the need for approaches that understand and address such complexity [8].

The literature on evidence exchange and use in health policymaking [3],[8]-[11] consistently identifies interpersonal relationships between researchers and policy-makers as having a positive effect [8],[12]-[16]. Developing relationships between producers and users has become a core focus of knowledge translation interventions [16],[17], operationalized into approaches such as `knowledge brokers’ and `deliberative dialogues’ [18]. Justifications for such strategies include arguments related to shifting the incentives for research producers [17], reducing the costs to research users [8], reducing conflict around research evidence [17], and facilitating trust [8],[17]-all of which are thought to increase the likelihood of exchange and use. Yet, few of these approaches explain why some relationships form and others do not, beyond the simplistic `research producer’ versus `research user’ dichotomy. Even deliberative dialogues, which succeed in bringing together a range of actors, and thus affect network structure, operate with limited understanding of the social structure they are intervening on, or the causal mechanisms underlying their intervention. Thus, despite calls for more attention to social structure and social networks [8],[19], little has been done to elucidate the causal mechanisms underlying the current field of socially-targeted interventions for evidence-informed health policy.

This study applies an explicit social network approach in order to answer two research questions related to evidence exchange and use in health policymaking: first, what factors are associated with the formation of research exchange relationships between policy actors; and second, to what extent are these exchange relationships associated with the use of research evidence by policy actors in the policymaking process? These results will have important implications for the design of interventions for knowledge translation and evidence-informed health policymaking.

Social network analysis as a framework to understand evidence-informed health policy

Social network analysis (SNA) is a theory as well as a set of tools for exploring socially-influenced behaviours. Social network theory offers a persuasive explanation of why evidence exchange and use should be understood from an interpersonal, as well as structural, perspective. As seen through a social network lens, behaviours’in this case, evidence exchange and use-are predicted not only by an individual’s attributes, but by his or her position in her social environment, and the larger structure of that environment. Social network analysis has recently been applied to evidence exchange in a municipal public health department [20] and to information, broadly speaking, within organizations [21]-[25] and political networks, [13] but never to evidence exchange and use in a national policy setting. We thus extend the existing research agenda to answer questions about the exchange of research evidence (the social process) within policy networks (the social setting). Policy networks are simply social networks specific to policy actors-defined here as sets of individuals who interact on a given policy issue, and may include a range of actors from various sectors and levels of governance. In contrast to formal organograms or stakeholder maps that document who are expected to (or should) participate according to organizational boundaries [26], policy networks are empirically-driven measurements of who actually participates, thus embracing a policy arena’s fluidity and diversity. In many settings, including low- and middle-income countries, policy decisions are made by a range of diverse actors with varying levels of formal and informal power [27]-[29].

While social network analysis can be applied to research questions at actor, tie, or network levels, this study specifically exploits recent methodological innovations in statistical network analysis enabling the prediction of ties between pairs of actors as some function of their individual attributes and network structure. These models are referred to as the exponential random graph class of models (ERGM) [30],[31]. The present paper uses ERGMs to model the existence of research evidence exchange ties, specifically evidence provision and request, in a policy network.

Ties can form, in theory, between any two individuals. But in practice, network scientists observe more frequent tie formation in the presence of certain network structures. The next section presents common network hypotheses adjusted to the context of Burkina Faso (see Figure 1 for a synthesis), where, as in many low-income countries, the formal culture of research production, access and use is relatively weak due to language barriers, poor Internet access, and general resource limitations [32],[33]. A better understanding of interpersonal modes of exchange will be highly relevant for Burkina Faso and other low-income countries.
Figure 1

Network covariates and their hypotheses.

First, individuals are more likely than chance alone to form ties with other individuals if they have a friend in common [34]. This phenomenon is referred to as triangle closure and looks like a triangle between three actors on a network graph (see Figure 1). In the policy science and governance literature, triangle closure is associated with stronger group cohesion, cooperation, and shared norms [35]-[37], and many redundant ties is associated with the fast diffusion of ideas amongst dyads and sub-groups. At the same time, triangle closure limits `structural holes’ in a network; that is, the areas of a network where actors are relatively unconnected [34]. It is the bridging of these structural holes that is associated with the efficient exposure to and diffusion of innovations and new ideas at the macro-level [35]. We expect to observe a positive association between triangle closure and tie formation in evidence exchange networks with cohesive cognitive paradigms. A network with a tendency for closure will likely exchange ideas that reinforce the status quo at the dyadic and sub-group level, but will limit the introduction and exchange of new ideas at the macro level.

`Reciprocity’ is another social process that is frequently observed more often than chance alone [36],[38]. For example, a request for evidence from Person A to Person B is reciprocated when Person B requests evidence from Person A (see Figure 1). Networks with high reciprocity are associated with `flat,’ or non-hierarchical, governance [39]. An evidence exchange network with high reciprocity would indicate that its actors have relatively similar levels of expertise, access to and demand for evidence, and the political capital to exchange it with each other, as was recently observed in a Canadian municipal public health department [20]. In contrast, policy actors in Burkina Faso have varying levels of capacity for finding and using research evidence and the policymaking culture is hierarchical, with power and expertise centralized among certain few actors. For these reasons, we do not expect to observe a positive association between reciprocity and tie formation in these networks.

`Layering’ is the other side of the coin (also known as multiplexity or entrainment). Instead of measuring the existence of two ties for the same relation between a pair of actors, layering measures the co-existence of two different relations between the same pair of actors [36]. For example, imagine that Person A requests evidence from Person B. Layering occurs when Person B provides evidence to Person A (see Figure 1). Research exchange networks with high layering are achieving their instrumental, or practical, purpose-research is provided when requested-and we suggest that layering is an empirical signature of true `exchange’ models of knowledge translation [17]. A lack of layering could indicate many things. A tendency for provision ties to exist without request ties might indicate that evidence is being disseminated for advocacy purposes, and at the most extreme might signal the symbolic or political use of evidence [40]. On the other hand, a tendency for request ties to exist without provision ties might suggest that no evidence is available to fulfill the request, or that actors are refusing or ignoring requests. Both outcomes suggest a poor climate for research evidence. We anticipate that research is provided by a small group of development partners and researchers in this context, often in the absence of requests, and thus we do not expect to observe significant layering.

In addition to these structural effects, we assume that the individual attributes of actors will influence their propensity to provide or request research evidence. There have been few studies exploring which individual characteristics influence evidence exchange, other than to say that evidence is more likely to be exchanged by a researcher or someone with research experience [12]. We hypothesize that in this context, where the culture of research and evidence use is nascent, the provision and request of research evidence will occur more often by actors with research experience, actors with higher employment rank, and development partners. Each of those actor types will have greater resources and technical skills related to finding, exchanging and using research evidence. Higher employment rank and development partners may also signal exposure to larger external networks.

Finally, the phenomenon of homophily specifies that actors are more likely to form ties with `like’ individuals [41]. Homophily reduces transaction costs associated with exchanging evidence but limits the wide and efficient dissemination of evidence that would occur if ties connected researchers to non-researchers, crossed organizational boundaries, and traversed job levels. We expect to observe homophily in these networks.

Methods

Data collection

Social network and demographic data were collected between October 2011 and March 2012 from policy actors active in one or more of three health policy issues in Burkina Faso: community integrated management of childhood illness; home management of malaria; and removal of user fees for antiretroviral treatment for human immunodeficiency virus (HIV); henceforth referred to by their substantive categories: child health; malaria; and HIV. Policy cases were sampled according to pragmatic reasons, including large enough networks to enable statistical analysis, as well as for their diversity on network structural variables of interest. High quality and locally relevant research evidence was available to inform each of these policy cases and each decision involved some amount of uncertainty that could have been addressed using research evidence, typical of many health policy decisions made by ministries of health in Sub-Saharan Africa. That said, Burkina Faso faces many of the same challenges in using evidence to inform policy as other countries and some challenges that are context-specific; namely, slow internet connection limiting searches for evidence, and a working language (French) that offers fewer research publications. Commonalities include the fact that evidence must compete with many other inputs in the policymaking process-including institution-, interest-, and idea-related factors, and that the overall number of researchers, and thus research capacity, tends to be fewer in low-income countries. A recent rise in donor applications requiring evidence of the problem and options has perhaps introduced incentives to exchange and use research evidence.

Burkina Faso is like other Sub-Saharan African countries in terms of level of development, political regime type, and dependence on foreign aid, as well as a health policy sector characterized by participation of diverse actors and frequent changes in institutions. These characteristics will improve the generalizability of results, contributing to knowledge translation efforts, and evidence-informed policymaking, in other Sub-Saharan African countries.

We defined policy actors as any individual who participated in policy formulation for each policy issue. Actors were identified according to established procedures for respondent-driven censuses of social networks [42]. Beginning with a review of policy documents and informational interviews with key informants, the lead researcher (JS) approached two actors from each policy issue considered to be central in their issue networks for an interview. The researcher asked: `With whom did you interact during policy formulation?’ generating `interaction’ ties. Respondents were encouraged to provide as many names as possible; respondents provided an average of five names. The researcher approached each actor named and the same process was carried out (see Additional file 1 for a consort diagram detailing this process). Following other studies of policy networks [43], we chose to cease sampling when a round elicited fewer new names compared to the previous round. This resulted in four rounds of nominations: most new nominations came during the second and third rounds; the fourth round consisted of 23 of a total of 116 nominations (19.8%) and actors interviewed during the fourth round nominated only six new actors. Of 101 unique actors identified, 69 were reached for an interview. Actors who participated in more than one issue were interviewed successively for each issue. Missing ties were dropped in the analysis.

Upon listing their interaction relationships, respondents were then asked to specify whether they had provided research evidence to any of the names they listed (provision ties), or requested research evidence from any of the names they listed (request ties). Ties were coded as 1 if at least one of the actor-pair reported that a tie existed. Provision and request ties were coded as directed from one actor to another based on each respondents’ reports of their exchanges and could occur in both directions. Provision and request ties could occur simultaneously (i.e., layering), but were considered separate constructs, and as such could also exist in the absence of the other.

Data were collected on actors’ relevant individual attributes (see Table 1), including: `research experience,’ defined as any formal experience participating in or leading a research project; `organization type,’ differentiating between actors working for government, development partner organizations, civil society organizations (CSO) and/or non-government organizations (NGO), and others (research organizations or independent consultants); and employment rank, dichotomized into manager/director-level and higher, or not. In-depth interviews elicited respondents’ perceptions and understandings of the policy issues as well as their awareness and use of research evidence during the policy-making process. Social desirability bias was avoided by indirect lines of questioning that did not suggest that respondents’ use of research evidence was being measured and response validity was achieved through probing strategies and in asking for specific examples or citations if a respondent claimed to have been aware of research on the topic, or to have included evidence in their reports. We hypothesize that an actor’s score on the use scale will be positively associated with their connectedness in the network, or `degree,’ where out-degree counts the number of ties an actor sends and in-degree counts ties received. We expect to see the highest rates of use amongst actors who provide and request evidence more frequently (i.e., high out-degree in either network) as the active sending of ties suggests positive attitudes related to evidence.
Table 1

Descriptive statistics

Variables

Child health

Malaria

HIV

Actor variables

Mean (sd) or n (%)

Mean (sd) or n (%)

Mean (sd) or n (%)

Total actors nominated in 4 rounds

39

49

40

Actors surveyed

21

30

19

Male

13 (68.4)

24 (80.0)

16 (76.2)

Graduate-level degree

17 (89.5)

27 (90.0)

18 (90.0)

Years in position

3.11 (2.85)

4.48 (4.06)

7 (4.87)

Experience as researcher

8 (44.4)

14 (46.7)

12 (57.1)

Government org

14 (73.7)

17 (56.7)

8 (38.1)

Civil society organization

0 (0)

9 (30.0)

9 (42.9)

Development partner organization

4 (21.1)

2 (6.67)

2 (9.52)

Other organization

1 (5.26)

2 (6.67)

2 (9.52)

Manager or higher employment level

9 (47.4)

8 (26.7)

13 (61.9)

Research use outcomes

   

Research use (continuous)

3.50 (1.03)

2.69 (1.44)

2.89 (1.52)

Any research use (binary)

7 (43.8)

10 (34.6)

6 (31.6)

Provision network

   

Edges

37

36

28

Mean degree

1.95

1.20

1.33

Density

0.11

0.04

0.07

Triangles

80

10

6

Reciprocated edges

6

1

2

Layered edges

16

20

8

Request network

   

Edges

17

25

10

Mean degree

0.895

0.833

0.476

Density

0.05

0.03

0.02

Triangles

10

8

0

Reciprocated edges

1

1

1

Layered edges

16

20

8

Qualitative interviews were tape-recorded and transcribed and data were managed and coded using NVivo software. Ethical approval was received from McMaster University’s Faculty of Health Sciences Ethical Review Board and the Burkina Faso Ministry of Health Ethics Committee in Health Research. Signed consent was received from all study participants.

Analysis

The probability of a tie existing between any two given actors is modeled using ERGMs, which can be expressed as a conditional log-odds of individual ties in a network:
logit P Y ij = 1 | rest of network = θ Δ g y ij
(1)

where Yij is an actor pair in network Y, θ is the vector of coefficients, Δ(g(y))ij is the change in the vector of network statistics g(y) when the value of Yij changes from 0 to 1. As in other regression models, we can include covariates to control for other processes that might affect the existence of a tie. ERGMs are different from normal logistic models in that they can control for network structure, or structural effects (SE), as well as actor characteristics, or attribute effects (AE). Structural effects models (SE models) included parameters for triangle closure, reciprocity and layering. Triangle closure was modeled using the geometrically weighted edgewise shared partner distribution (GWESP) statistic, which has been shown to overcome model degeneracy by specifying decreasing marginal impact of the formation of triangles on tie formation. [26],[35] AE include researcher experience, organization type, employment rank and homophily. SE and AE models were run separately and then aggregated into full models.

Provision and request ties, the dependent variables, were modeled separately for each of the three policy networks, conditional on the existence of an interaction tie. Covariates were each modeled separately to determine which were most significant according to p-values and significant covariates were then entered into the full model step-wise based on their p-value in the null model (most to least significant). Covariates were retained in the final model if they improved model fit as tested by Akaike Information Criterion (AIC) and likelihood ratio tests. ERGM coefficients represent the conditional log-odds of a tie. Data were managed using Microsoft Excel and analyzed in R using the Statnet suite of packages [44], including `ergm’ [45]. The ERGM model likelihood function was approximated using Markov chain Monte Carlo simulation methods [44].

Goodness of fit was tested by comparing simulated networks to the observed networks to determine how well the model could reproduce global network properties (see Additional file 2 for goodness-of-fit test results).

The determinants of an actor’s use of evidence were explored in logistic regression models where the dependent variable was evidence use and was derived from in-depth interviews. A validated scale of evidence use by policy-makers [46],[47] was applied to interview data, assigning each actor a value from 0 (no evidence use) to 5 (`I made efforts to use this research evidence in decisions related to this policy issue’) based on the qualitative analysis of respondents’ discussion of how they used evidence during the policy process. While we had hoped to explain the ordinal outcome of `evidence use’ in regression models, those models did not converge and each actor’s score was collapsed into a binary dependent variable where the fourth category (`I cited the research evidence in my own professional reports, documents or conversations’) and fifth (`I made efforts to use the research evidence in decisions related to this policy issue’) were coded as `use’ and the third category (`I participated in meetings for discussion and dissemination of the research evidence’) and those below it were coded as `non-use.’ Univariate logistic regression models tested whether use was associated with in-degree and out-degree in the provision and request networks, and with actor attributes.

Results

Table 1 describes the networks and their actors. Network composition varied slightly across issues, particularly in terms of organizational affiliation and employment level. Across all issues, provision networks were denser than request networks, meaning that evidence was provided more than requested. The child health networks were the densest and had the highest average degree, indicating that evidence exchange occurred more often for this issue than for the others. The child health networks had more triangles than other networks (80 triangles in the provision network). Approximately 13% of actors participated in more than one policy network. Figure 2 illustrates provision and request networks for each case.
Figure 2

Graphs of policy networks. (a) Child health evidence provision (left) and evidence request (right) (b) Malaria evidence provision (left) and evidence request (right) (c) HIV evidence provision (left) and evidence request (right). Arrows indicate direction of relationship.

Why do policy actors exchange research evidence?

The provision and request of research evidence were associated with factors related to both structural and attribute effects. All model converged [44], although some covariates did not improve model fit as judged by AIC criteria and were thus excluded.

Provision models

Models for child health and malaria networks fit best when they combined structural and attribute effects (see Table 2 for full results). HIV models were slightly better fit when only modeling actor attributes. Actors were more likely to form provision ties if they also had a request tie (i.e., layering), an effect that was statistically significant across issues. Only the child health network demonstrated a tendency for triangle closure (θ = 2.36; OR = 10.6, p < 0.05) where the odds of a provision tie were 10 times greater if that tie closed a triangle between three actors, conditional on the rest of the model. This suggests that evidence provision may have been used strategically to reinforce a cohesive paradigm. In the malaria network, the odds of a provision tie were 2.77 times more likely (θ = 1.02; OR = 2.77, p < 0.05) if actors had research experience and were half as likely (θ = -0.671, OR = 0.51, p < 0.05) if they worked for a CSO/NGO compared to working for the government. As hypothesized, there was no evidence of reciprocity in any of the issues, suggesting that evidence was provided and requested in a hierarchical manner.
Table 2

Parameter estimates (standard errors) of provision networks

 

Child health

Malaria

HIV

 

Structural effects model

Attribute effects model

Full model

Structural effects model

Attribute effects model

Full model

Structural effects model

Attribute effects model

Full model

Structural effects

   

Edge intercept

-4.52 (0.96)*

-1.18 (0.70)*

-4.65 (1.83)*

-2.13 (0.32)*

-2.20 (0.44)*

-2.56 (0.52)*

-1.03 (0.37)*

-1.99 (1.03)*

-1.79 (1.14)

Triangle closure

2.81 (0.63)*

__

2.36 (0.78)*

0.74 (0.47)

__

0.78 (0.85)

0.382 (0.79)

__

0.405 (1.45)

Reciprocity

-5.10 (1.92)*

__

-3.52 (2.20)

-1.20 (1.15)

__

-1.71 (1.29)

0.007 (0.93)

__

0.318 (1.40)

Layering

5.15 (1.44)*

__

6.16 (1.73)*

3.23 (0.59)*

__

2.87 (0.63)*

2.29 (0.83)*

__

2.31 (0.98)*

Attribute effects

   

Researcher

__

-0.23 (0.42)

-0.93 (0.98)

__

1.43 (0.37)*

1.021 (0.44)*

__

0.505 (0.56)

0.154 (0.60)

Civil society org.

__

No obs.

No obs.

__

-0.46 (0.32)

-0.671 (0.40)*

__

0.150 (0.46)

-0.120 (0.49)

Development partner org.

__

1.02 (0.45)*

0.31 (1.14)

__

-1.10 (0.67)

-0.813 (0.79)

__

3.37 (1.15)*

3.50 (1.16)*

Other

__

2.43 (1.19)*

5.04 (5.29)

__

0.81 (0.74)

0.217 (0.94)

__

2.10 (0.89)*

1.61 (1.04)

Manager/director

__

0.33 (0.35)

0.16 (0.80)

__

nf

nf

__

nf

nf

Homophily

   

Researcher

__

nf

nf

__

nf

nf

__

nf

nf

Organization

 

nf

nf

__

nf

nf

__

nf

nf

AIC

233.7

179.5

176.4

345.3

369.1

342.8

210.1

191.6

193.1

*p < 0.05; nf = did not improve model fit.

Request models

Request ties were best fit in models combining structural and attribute effects (see Table 3 for full results). Layering, or the simultaneous existence of a request and a provision tie, remained the strongest predictor of tie formation in these models. Child health again had a significant coefficient for triangle closure (θ = 1.53; OR = 4.62, p < 0.05), meaning that the odds of tie formation were 4.62 times greater if the tie closed a triangle between three actors, conditional on the remainder of the model. Although this coefficient is not as large as in the provision network, request ties still seemed to occur to either strengthen or reinforce cohesion and shared norms in this case. As in the provision models, malaria actors were more likely to have request ties if they had research experience (θ = 1.73; OR = 5.64, p < 0.05), suggesting that this policy issue, in particular, had narrowly defined roles for who exchanged evidence. The malaria network demonstrated a significant negative effect of homophily within organizations (θ = -2.24; OR = 0.11, p < 0.05), meaning that actors were more likely to request research evidence outside of their organizations as compared to within, which runs counter to our hypotheses but presents a picture of interorganizational exchange of evidence for this case. The odds of forming a request tie were 3.36 times higher (θ = 1.85, OR = 3.36, p < 0.05) if an actor belonged to the `other’ organization category in the HIV network, a category which includes consultants and researchers. This is to be expected in a network that had high representation of CSO/NGO actors, who typically perceived a smaller role for research evidence as compared with lived experiences.
Table 3

Parameter estimates (standard errors) of request networks

 

Child health

Malaria

HIV

 

Structural effects model

Attribute effects model

Full model

Structural effects model

Attribute effects model

Full model

Structural effects model

Attribute effects model

Full model

Structural effects

   

Edge intercept

-4.00 (0.997)*

-1.47 (0.406)*

-4.67 (1.15)*

-3.66 (0.545)*

-2.64 (0.787)*

-4.19 (1.19)*

-3.37 (0.743)*

-5.83 (1.92)*

-6.74 (2.21)*

Triangle closure

1.49 (0.812)*

__

1.53 (0.848)*

1.26 (0.601)*

__

1.65 (2.88)

nf

__

nf

Reciprocity

-1.93 (1.49)*

__

-1.90 (1.62)

0.029 (1.28)

__

-1.43 (1.49)

1.98 (1.34)

__

-0.842 (2.14)

Layering

2.72 (1.12)*

__

2.96 (1.14)*

3.28 (0.604)*

__

3.18 (0.733)*

2.27 (0.832)*

__

2.31 (0.992)*

Attribute effects

   

Researcher

__

0.088 (0.401)

0.652 (0.486)

__

1.90 (0.473)*

1.73 (0.688)*

__

1.70 (0.901)*

1.64 (1.01)

Civil society org.

__

nf

nf

__

-0.490 (0.498)

-0.313 (0.593)

__

1.30 (0.731)*

1.31 (0.788)

Development partner org.

__

nf

nf

__

-2.29 (0.979)*

-3.34 (1.41)*

__

nf

nf

Other

__

nf

nf

__

-0.407 (0.944)

-1.27 (1.21)

__

2.32 (0.953)*

1.85 (1.10)*

Manager/director

__

nf

nf

__

0.619 (0.379)

0.613 (0.478)

__

nf

nf

Homophily

   

Researcher

__

nf

nf

__

nf

nf

__

nf

nf

Organization

__

nf

nf

__

-1.60 (0.663)*

-2.24 (0.933)*

__

nf

nf

AIC

183.4

208.7

183.5

310.6

334.7

306.9

162.7

171.6

DNC

*p < 0.05; nf = term did not improve model fit.

Are exchange relationships associated with the use of research evidence?

Based on analysis and coding of interview data, 43.0%, 34.6%, and 31.6% of policy actors in child health, malaria, and HIV domains, respectively, used research evidence during the policy processes to inform their professional decisions. Actively providing evidence was positively associated with the use of evidence across all issues (see Table 4). Receiving a request for research evidence (in-degree) was associated with use in the malaria domain, and sending a request (out-degree) associated with use in the user-fees domain. The statistical significance of the continuous degree variable suggests a dose-response effect; in other words, an actor is more likely to use evidence for each additional exchange he/she has. Multivariable models combining actors’ degree and their individual attributes demonstrated that degree was more predictive of research use than their attributes.
Table 4

Univariate logistic regressions of actor degree on evidence use

 

Child health

Malaria

HIV

Independent variable

Log odds (SE)

Log odds (SE)

Log odds (SE)

Indegree (provision)

-0.336 (1.16)

0.896 (0.941)

-1.01 (1.16)

Outdegree (provision)

3.87 (1.51)*

Predicted perfectly

3.31 (1.34)*

Indegree (request)

0.223 (1.10)

1.86 (0.949)*

0.118 (1.05)

Outdegree (request)

1.79 (1.31)

1.40 (0.881)

3.31 (1.34)*

N

16

26

19

*p < 0.05.

Discussion

This study illuminated the conditions under which evidence was provided and requested in three policy cases, and confirmed that evidence exchange is closely correlated to its use in health policymaking. Evidence provision and request ties were best predicted by structural factors, particularly layering (i.e., actors are more likely to send ties when they complement existing ties), consistent with social network theories of the significant role of networks and structure in predicting individual-level behaviours. Some individual attributes mattered, particularly the role of research experience in the malaria domain, but should not be solely relied upon to design or target knowledge translation interventions. In terms of evidence use, network position-as judged by connectedness to others-predicted use better than any individual characteristic-not job level, not organizational affiliation, not experience as a researcher. This finding is consistent with other findings that show interpersonal relationships with researchers to be the best predictor of evidence use by policy-makers [12]. Taken together, these findings support taking a network lens when designing knowledge translation or evidence-informed policymaking interventions [48] and focusing on strategies that increase an actor’s capacity to provide and request evidence.

Findings in relation to our hypotheses

Our findings were generally consistent with our hypotheses and clarify network theories that had not yet been tested in a low-income country policy setting. Only the child health network demonstrated a tendency for triangle closure. This network had the highest density of research provision and request ties and also had the highest proportion of actors who used research evidence to inform their decision-making, consistent with other studies that find the transfer of complex or tacit knowledge is aided by closed, cohesive networks that enable repeated exposure necessary for the synthesis and interpretation of complex ideas [22]. However, findings from linked studies suggest that while evidence was exchanged by dyads and used by actors in this network, the macro-level network used evidence `symbolically’ to justify pre-determined policy positions of certain actors [49]. The child health policy process advanced as part of a funding proposal process where the funders required that development partners had an equal seat at the table; respondents reported that most research evidence was disseminated strategically by development partners, typically to persuade hesitant government policy-makers. Thus, it is possible that active advocacy and persuasion during this policy process led to the observed triangle closure as opposed to pre-existing cohesiveness amongst actors in this domain. This finding is consistent with studies of information diffusion that contrast structural effects at dyadic and macro levels [34]. There is no question that actors can influence the shape of networks and their outcomes [50].

Ties were not reciprocated. Patterns of provision and request were hierarchical and unidirectional, as was expected for this context. Layering of the two relations was observed to a large degree, suggesting that these networks function instrumentally in that research is generally provided only when it is requested, and that most requests are realized. In considering these findings together, one can imagine a hierarchy of evidence flows, where requests flow one way and provisions flow the opposite direction, most often between the same individuals, but rarely will a pair reverse their roles as requesters and providers. Overall, evidence was provided more often than it was requested. This is consistent with the qualitative interviews where respondents suggested that certain actors, particularly development partners, provided unsolicited evidence.

We observed an absence of homophily, counter to our hypotheses. Homophily covariates did not improve model fit and were thus excluded, except in the case of the malaria request network where actors were less likely to exchange evidence if they belonged to the same organization. This finding reflects the malaria domain in Burkina Faso, where formal and informal institutional arrangements encourage exchanges between government policy-makers and research organizations. In the child health domain, the strategic dissemination of evidence by development partners overcame tendencies towards homophily. The finding that no single individual attribute seemed to drive tie formation was counter to our hypothesis, and to the implicit design of many knowledge translation interventions, but was also recently observed in a municipal public health department in Ontario [20].

Strengths and limitations

This study is the first to empirically measure and model research evidence exchange in policy networks and provides important insights for evidence use in low-income countries. The finding linking frequency of exchanges with likelihood of use is likely generalizable across policy issues and jurisdictions. However, understanding how evidence is being used (and therefore why it is being exchanged) will require knowledge of specific issues and context.

Linked studies exploring the contextual factors surrounding evidence use in these cases add clarity to the motivations for evidence exchange and use; for example that dense exchange networks for child health were associated with symbolic use, that there was little motivation for evidence use in the malaria case, and that evidence was used instrumentally to change policy in the HIV case. Future examples of policy networks will enable the refinement of theories related to network-based markers of evidence use and exchange. Finally, this study is limited, as are most social network analyses, by challenges in collecting complete data on whole networks. Missing data may affect our results.

Implications for policy and practice

In considering how to design network-based interventions for various settings, we suggest the aphorism: `know your network.’ Any knowledge translation intervention should begin with a baseline mapping of relevant policy network(s). While well-designed sociometric surveys remain the gold standard for network mapping, we acknowledge that sampling or ego-based approaches may still add-value above and beyond the status quo. Many local stakeholders have excellent intuition about their networks that can be harnessed to this end.

Deliberative dialogues could use this information to ensure that dialogue attendees are in strategic network positions-not just in strategic organizational positions. Dissemination of research evidence should include a message to share the evidence with a colleague or friend, and an encouragement to also request evidence from those same people. Ultimately, cultural shifts are needed to increase the perception that anyone can provide (or request) evidence; these shifts have occurred in other settings by building capacity and shifting incentives [17].

Conclusion

This study explores the exchange and use of research evidence among policy actors in Burkina Faso. It is among the first of its kind of describe structural and attribute-related factors associated with exchange relationships among policy actors. Study findings suggest that while research exchange networks and their outcomes are highly issue-dependent, networks have a significant influence on knowledge exchange and use. Network variables-including the propensity to send complementary ties, to join sets of three actors, and overall connectedness-were more important than individual characteristics in predicting whether research evidence was provided or requested between actors, and were certainly more important in predicting an actors’ use of evidence. These findings can be leveraged to design knowledge transfer interventions which focus on facilitating or reinforcing exchange relationships.

Authors’ contributions

JCS conceived, designed, carried out the study and drafted the manuscript; MD provided input on study design, data collection, and statistical analysis; JNL provided input on study design, data collection, and measurement. All authors read and approved the final manuscript.

Additional files

Declarations

Acknowledgements

We would like to thank Julia Abelson and Gill Walt for feedback on earlier drafts of this paper, as well as members of the University of Washington netmodeling seminar for their comments on the ERGM analysis. Funding for this work was provided to JCS through the Canadian Institutes of Health Research (CIHR) Canada Graduate Scholarship-Doctoral Award, and to JCS and JNL through Canada IDRC’s Research Chair in Evidence Informed Policies and Health Systems.

Authors’ Affiliations

(1)
Department of Clinical Epidemiology and Biostatistics, Centre for Health Economics and Policy Analysis, McMaster University
(2)
Department of Political Science, McMaster University

References

  1. World report on knowledge for better health. 2004Google Scholar
  2. The mexico statement on health research. 2004, World Health Organization, GenevaGoogle Scholar
  3. Lavis JN, Lomas J, Hamid M, Sewankambo NK: Assessing country-level efforts to link research to action. Bull World Health Organ. 2006, 84 (8): 620-628. 10.2471/BLT.06.030312.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Frenk J: The global health system: Strengthening national health systems as the next step for global progress. PLoS Med. 2010, 7 (1): e1000089-10.1371/journal.pmed.1000089.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Chalkidou K, Marten R, Cutler D, Culyer T, Smith R, Teerawattananon Y, Cluzeau F, Li R, Sullivan R, Huang Y, Fan V, Glassman A, Dezhi Y, Gyansa-Lutterodt M, McPherson S, Gadelha C, Sundararaman T, Squires N, Daulaire N, Sadanandan R, Shiro K, Lemgruber A: Health technology assessment in universal health coverage. Lancet. 2013, 382 (9910): e48-e49. 10.1016/S0140-6736(13)62559-3. doi:10.1016/S0140-6736(13)62559-3; doi:10.1016/S0140-6736(13)62559-3. 3/26/2014 6:40:10 PMView ArticlePubMedGoogle Scholar
  6. Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers’ perceptions of their use of evidence: A systematic review. J Health Serv Res Policy. 2002, 7: 239-244. 10.1258/135581902320432778.View ArticlePubMedGoogle Scholar
  7. Moat KA, Lavis JN, Abelson J: How contexts and issues influence the use of policy-relevant research syntheses: A critical interpretive synthesis. Milbank Q. 2013, 91 (3): 604-648. 10.1111/1468-0009.12026. doi:10.1111/1468-0009.12026; doi:10.1111/1468-0009.12026View ArticlePubMedPubMed CentralGoogle Scholar
  8. Contandriopoulos D, Lemire M, Denis JL, Tremblay E: Knowledge exchange processes in organizations and policy arenas: A narrative systematic review of the literature. Milbank Q. 2010, 88 (4): 444-483. 10.1111/j.1468-0009.2010.00608.x. doi:10.1111/j.1468-0009.2010.00608.x; doi:10.1111/j.1468-0009.2010.00608.xView ArticlePubMedPubMed CentralGoogle Scholar
  9. Mitton C, Adair CE, McKenzie E, Patten SB, Waye PB: Knowledge transfer and exchange: Review and synthesis of the literature. Milbank Q. 2007, 85 (4): 729-768. 10.1111/j.1468-0009.2007.00506.x. doi:10.1111/j.1468-0009.2007.00506.x. 12/18/2012 2:33:52 PMView ArticlePubMedPubMed CentralGoogle Scholar
  10. Lavis JN: How can we support the use of systematic reviews in policymaking?. PLoS Med. 2009, 6 (11): e1000141-10.1371/journal.pmed.1000141. doi:10.1371/journal.pmed.1000141; doi:10.1371/journal.pmed.1000141. 12/18/2012 2:19:47 PMView ArticlePubMedPubMed CentralGoogle Scholar
  11. Oxman AD, Vandvik PO, Lavis JN, Fretheim A, Lewin S: SUPPORT tools for evidence-informed health policymaking (STP) 2: Improving how your organisation supports the use of research evidence to inform policymaking. Health Res Policy Syst. 2009, 7 (Suppl 1): S2-4505-7-S1-S2-doi:10.1186/1478-4505-7-S1-S2; doi:10.1186/1478-4505-7-S1-S2. 12/18/2012 2:19:47 PMGoogle Scholar
  12. Lavis JN, Ross SE, Hurley JE, Hohenadel JM, Stoddart GL, Woodward CA, Abelson J: Examining the role of health services research in public policymaking. Milbank Q. 2002, 80 (1): 125-154. 10.1111/1468-0009.00005.View ArticlePubMedPubMed CentralGoogle Scholar
  13. Carpenter DP, Esterling KM, Lazer DM: Friends, brokers and transitivity: Who informs whom in washington politics?. J Polit. 2004, 66 (1): 224-246. 10.1046/j.1468-2508.2004.00149.x.View ArticleGoogle Scholar
  14. Lavis JN, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E: Towards systematic reviews that inform healthcare management and policymaking. J Health Serv Res Policy. 2005, 10: 35-48. 10.1258/1355819054308549.View ArticlePubMedGoogle Scholar
  15. Lomas J: The in-between world of knowledge brokering. BMJ. 2007, 334 (7585): 129-132. 10.1136/bmj.39038.593380.AE. doi:10.1136/bmj.39038.593380.AEView ArticlePubMedPubMed CentralGoogle Scholar
  16. Lavis JN, Oxman AD, Moynihan R, Paulsen EJ: Evidence-informed health policy 1 - synthesis of findings from a multi-method study of organizations that support the use of research evidence. Implement Sci. 2008, 3: 53-5908-3-53-doi:10.1186/1748-5908-3-53; doi:10.1186/1748-5908-3-53. 3/27/2014 11:51:32 AMGoogle Scholar
  17. Lomas J: Using `linkage and exchange’ to move research into policy at A canadian foundation. Health Aff (Millwood). 2000, 19 (3): 236-240. 10.1377/hlthaff.19.3.236. 12/18/2012 2:26:43 PM; 12/18/2012 2:26:43 PMView ArticleGoogle Scholar
  18. Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N: Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Soc Sci Med. 2012, 75 (11): 1938-1945. 10.1016/j.socscimed.2012.06.016. doi:10.1016/j.socscimed.2012.06.016; doi:10.1016/j.socscimed.2012.06.016View ArticlePubMedGoogle Scholar
  19. Greenhalgh T: Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q. 2004, 82 (4): 581-10.1111/j.0887-378X.2004.00325.x.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Yousefi-Nooraie R, Dobbins M, Marin A: Social and organizational factors affecting implementation of evidence-informed practice in a public health department in ontario: A network modelling approach. Implement Sci. 2014, 9 (1): 29-5908-9-29-10.1186/1748-5908-9-29. doi:10.1186/1748-5908-9-29; doi:10.1186/1748-5908-9-29View ArticleGoogle Scholar
  21. Hansen MT: Knowledge networks: Explaining effective knowledge sharing in multiunit companies. Organ Sci. 2002, 13 (3): 232-10.1287/orsc.13.3.232.2771.View ArticleGoogle Scholar
  22. Reagans R, McEvily B: Network structures and knowledge transfer: The effects of cohesion and range. Adm Sci Q. 2003, 48 (2): 240-267. 10.2307/3556658.View ArticleGoogle Scholar
  23. Borgatti SP: A relational view of information seeking and learning in social networks. Manage Sci. 2003, 49 (4): 432-10.1287/mnsc.49.4.432.14428.View ArticleGoogle Scholar
  24. Levin DZ, Cross R: The strength of weak ties you can trust: The mediating role of trust in effective knowledge transfer. Manage Sci. 2004, 50 (11): 1477-1490. 10.1287/mnsc.1030.0136.View ArticleGoogle Scholar
  25. Inkpen AC, Tsang EWK: Social capital, networks, and knowledge transfer. Acad Manage Rev. 2005, 30 (1): 146-165. 10.5465/AMR.2005.15281445. 12/18/2012 5:02:58 PMView ArticleGoogle Scholar
  26. Brugha R, Varvasovszky Z: Stakeholder analysis: A review. Health Policy Plan. 2000, 15 (3): 239-246. 10.1093/heapol/15.3.239.View ArticlePubMedGoogle Scholar
  27. Walt G: Health policy : An introduction to process and power. 1994, Witwatersrand University Press and Zed Books, Atlantic Highlands, N.J and LondonGoogle Scholar
  28. Mathews J: Power shift. Foreign Affairs. 2007,: 50-66.Google Scholar
  29. Woelk G, Daniels K, Cliff J, Lewin S, Sevene E, Fernandes B, Mariano A, Matinhure S, Oxman AD, Lavis JN, Lundborg CS: Translating research into policy: Lessons learned from eclampsia treatment and malaria control in three southern African countries. Health Res Policy Syst. 2009, 7: 31-10.1186/1478-4505-7-31.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Wasserman S, Pattison P: Logit models and logistic regressions for social networks: I. an introduction to markov graphs and p*. Psychometrika. 1996, 61 (3): 401-425. 10.1007/BF02294547. 12/18/2012 5:09:44 PMView ArticleGoogle Scholar
  31. Robins G: An introduction to exponential random graph (p*) models for social networks. Soc Netw. 2007, 29 (2): 173-10.1016/j.socnet.2006.08.002.View ArticleGoogle Scholar
  32. Kouyate B, Sie A, Ye M, De Allegri M, Muller O: The great failure of malaria control in africa: A district perspective from burkina faso. PLoS Med. 2007, 4 (6): e127-10.1371/journal.pmed.0040127. doi:10.1371/journal.pmed.0040127View ArticlePubMedPubMed CentralGoogle Scholar
  33. Dagenais C, Queuille L, Ridde V: Evaluation of a knowledge transfer strategy from a user fee exemption program for vulnerable populations in burkina faso. Glob Health Promot. 2013, 20 (1 Suppl): 70-79. 10.1177/1757975912462416. doi:10.1177/1757975912462416View ArticlePubMedGoogle Scholar
  34. Granovetter MS: The strength of weak ties. AJS. 1973, 78 (6): 1360-Google Scholar
  35. Burt R: Structural holes and good ideas. AJS. 2004, 110 (2): 349-399. Available:http://dx.doi.org.libaccess.lib.mcmaster.ca/10.1086/421787,Google Scholar
  36. Robins G, Lewis JM, Wang P: Statistical network analysis for analyzing policy networks. Policy Stud J. 2012, 40 (3): 375-401. 10.1111/j.1541-0072.2012.00458.x.View ArticleGoogle Scholar
  37. Lubell M, Scholz J, Berardo R, Robins G: Testing policy theory with statistical models of networks. Policy Stud J. 2012, 40 (3): 351-374. 10.1111/j.1541-0072.2012.00457.x. doi:10.1111/j.1541-0072.2012.00457.xView ArticleGoogle Scholar
  38. Goodreau SM, Kitts JA, Morris M: Birds of a feather, or friend of a friend? using exponential random graph models to investigate adolescent social networks*. Demography. 2009, 46 (1): 103-125. 10.1353/dem.0.0045. 12/18/2012 5:26:46 PMView ArticlePubMedPubMed CentralGoogle Scholar
  39. Jones C, Hesterly WS, Borgatti SP: A general theory of network governance: Exchange conditions and social mechanisms. Acad Manage Rev. 1997, 22 (4): 911-945.Google Scholar
  40. Weiss CH: The many meanings of research utilization. Public Adm Rev. 1979, 39 (5): 426-10.2307/3109916.View ArticleGoogle Scholar
  41. McPherson M, Smith-Lovin L, Cook JM: Birds of a feather: Homophily in social networks. Annu Rev Sociol. 2001, 27: 415-444. 10.1146/annurev.soc.27.1.415. 12/18/2012 5:43:59 PMView ArticleGoogle Scholar
  42. Knoke D, Yang S: Social network analysis. 2008, SAGE Publications, Thousand Oaks, CAView ArticleGoogle Scholar
  43. Lewis JM: Being around and knowing the players: Networks of influence in health policy. Soc Sci Med. 2006, 62 (9): 2125-2136. 10.1016/j.socscimed.2005.10.004. doi:10.1016/j.socscimed.2005.10.004View ArticlePubMedGoogle Scholar
  44. Handcock MS, Hunter DR, Butts CT, Goodreau SM, Morris M: Statnet: Software tools for the representation, visualization, analysis and simulation of network data. J Stat Softw. 2008, 24 (1): 1548-7660.View ArticleGoogle Scholar
  45. Hunter DR, Handcock MS, Butts CT, Goodreau SM, Morris M: Ergm: A package to fit, simulate and diagnose exponential family models for networks. J Stat Softw. 2008, 24 (3)View ArticleGoogle Scholar
  46. Knott J: If dissemination is the solution, what is the problem?. Sci Commun. 1980, 1 (4): 537-578. 10.1177/107554708000100404.View ArticleGoogle Scholar
  47. Landry R, Amara N, Lamari M: Utilization of social science research knowledge in canada. Res Policy. 2001, 30 (2): 333-349. 10.1016/S0048-7333(00)00081-0. doi:10.1016/S0048-7333(00)00081-0View ArticleGoogle Scholar
  48. Valente TW: Network interventions. Science. 2012, 337 (6090): 49-53. 10.1126/science.1217330. 10.1126/science.1217330; 10.1126/science.1217330View ArticlePubMedGoogle Scholar
  49. Shearer JC: "Social networks, research evidence, and innovation in health policy making in Burkina Faso.". PhD diss., McMaster University, 2014.Google Scholar
  50. Agranoff R, McGuire M: Big questions in public network management research. J Public Admin Res Theory. 2001, 11 (3): 295-326. 10.1093/oxfordjournals.jpart.a003504.View ArticleGoogle Scholar

Copyright

© Shearer et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.