Teasing apart “the tangled web” of influence of policy dialogues: lessons from a case study of dialogues about healthcare reform options for Canada

Abstract

Background

The knowledge exchange literature suggests that policy dialogues are intended to enhance short-, medium- and long-term capacities of individuals, organizations and health systems to use evidence to inform policy-making. Key features of effective dialogues have been suggested, but the linkages between these features and the realization of improved capacities for evidence-informed policy-making among dialogue attendees and the subsequent influence on policy-making activities are not well understood.

Methods

We conducted a qualitative case study of a series of four policy dialogues that were convened in Canada among national, provincial and regional stakeholders on topics pertaining to healthcare financing and funding in 2011. Data sources included videos of participant perspectives captured during or immediately following each event and follow-up key informant interviews among dialogue participants held 4 years later in 2015. Three conceptual frameworks pertaining to (i) policy dialogues and capacities for evidence use, (ii) factors shaping policy-making across the policy cycle and (iii) factors shaping implementation of evidence guided the thematic analysis. We then synthesized the findings across the three frameworks.

Results

The results suggest the potential benefits of policy dialogues described in the literature were developed among the participants at these dialogues. Informants elaborated on how dialogue features influenced their capacities to use evidence, the ideas, interests and institutions during the agenda-setting and policy formulation stages of policy-making and how implementation was affected by characteristics of policy options, individuals, organizations, the external environment and processes.

Conclusions

We present a conceptual framework that furthers our understanding of the potential influence of policy dialogues on the content and mechanisms of policy development and illustrate pathways of influence on various stages of the policy cycle from agenda setting through formulation and implementation. The framework highlights important factors for consideration in designing and evaluating policy dialogues and in supporting post-dialogue knowledge exchange efforts.

Background

Policy dialogues are a knowledge transfer and exchange approach [1,2,3] that has been used at the national level in Canada to shine a spotlight on areas for potential health policy reform [4,5,6,7,8] in a context where provincial and territorial governments are responsible for policies pertaining to healthcare delivery. Policy dialogues are group processes that provide opportunities for interactive knowledge-sharing that bring research evidence together with the views, experiences and tacit knowledge of those involved in or affected by future decisions about high-priority issues [9]. By promoting interactions between researchers and policy-makers (elected or appointed official who set the agenda and propose, develop and evaluate policies), policy dialogues can help in identifying and interpreting available evidence and finding areas of accord between research evidence and policy-makers’ beliefs, values, political goals and strategies [4, 10].

A key feature of effective policy dialogues is their deliberative nature. Deliberative approaches foster structured conversations that value listening as much as speaking, informed and reasoned argument to develop collaborative understanding of the values that may underlie opposing views, and weigh reasons for and against different policy options with the aim of building toward common action [11]. These approaches enable participants to explore diverse views, build trust, connect research to personal experience, and develop agency to take action within their own spheres [11,12,13]. With the right mix of participants, appropriate use of evidence, and a conducive dialogue setting, Boyko and colleagues suggest dialogue participants may enhance their capacities (knowledge, abilities, skills and actions to meet goals) [8] to use evidence in policy-making in the short term. These enhanced individual capacities contribute to enhanced organizational and system capacities over the medium and longer terms respectively [8].

Better understanding is needed of how the capacities gained through attending policy dialogues can contribute to evidence use in policy-making processes. These processes refer to how government and organizational policy decisions are made, where policy refers to statements of collective decisions, actions, processes or goals. The policy literature describes a policy cycle [14] that includes defining problems [15, 16] and agenda setting [17], policy formulation, decision-making, implementation and evaluation [18] which may or may not occur sequentially. Each stage is subject to complex interacting elements that can influence policy progression and can similarly influence the effect of any given policy dialogue [8]. For example, the 3-I framework (ideas, interests and institutions) recognizes that research evidence is only one consideration when it comes to ideas in policy-making; values, goals and perceptions of stakeholders (individuals who may be representative of governments, interest groups, non-governmental organizations) [16] their relative strength, relationships and positions (interests) on different policy issues, and institutions such as laws, regulations and tradition are also influential [19]. For example, Canada’s complex institutional landscape means the federal role has traditionally been set to guide principles and contribute funding toward provincially run publicly financed health insurance programs for hospital and physician services and deliver care to specific populations (e.g. First Nations, armed forces, federal penitentiary) [20, 21]. Any policy dialogue intended to influence policy-making must reflect this context.

The implementation science literature similarly describes the implementation of research into practice as a “complex and messy task” [22] that depends on many factors including evidence, context and factors that facilitate the change process [23]. While a recent systematic review identified 61 theories and frameworks (models) to enhance the spread of evidence-based interventions, only two models focused on implementation and policy level considerations [24]. The proponents of one of these models [10] point to the importance of better understanding how ideas are diffused across policy-making. The other model, the Consolidated Framework for Implementation Research (CFIR) [25], offers a consolidation of key constructs from published implementation theories as a “pragmatic structure for approaching complex, interacting, multi-level and transient states of constructs” [26]. It can be combined with the 3-I and policy cycle frameworks to better understand how policy dialogues can influence evidence use in policy-making.

This study used data from follow-up key informant interviews with delegates who attended four policy dialogue events hosted in 2011 by the Canadian Foundation for Healthcare Improvement (CFHI), as well as video commentary captured at the events. The goal was to better understand the extent to which the intended effects of policy dialogues on capacity development described in the literature were borne out in practice [27] and to learn from participants whether, and if so how, enhanced capacities influenced their subsequent activities in the policy realm across the various stages of the policy cycle. The findings contribute to the literature about the capacities that can be gained by participating in a policy dialogue and elaborate the mechanisms by which those capacities may influence policy processes and implementation of reform options. Better understanding of these mechanisms offers the opportunity to enhance the design and use of policy dialogues as knowledge exchange vehicles to foster use of evidence in policy-making.

Methods

We conducted a case study [28] to explore the role of policy dialogues in knowledge uptake and use by policy decision-makers and stakeholders based on the perspectives of participants at such events. The case study approach is particularly well suited to investigate “a contemporary phenomenon within it’s real-life context” [28] and can help to draw lessons to improve practices in other cases [29]. It typically involves the use of a priori theory or a conceptual framework to improve rigour in data-gathering and analysis [30, 31] and allows theory to evolve based on emerging findings [31].

The case is defined to include the influence (if any) on subsequent policy processes of four policy dialogue events within the Healthcare Financing and Transformation (HFiT) dialogue series held by CFHI in 2011: three half-day stakeholder dialogues on healthcare financing and payment with approximately 20 to 25 participants each and the International Health Economics Association (iHEA) pre-conference symposium with 92 participants and an international expert panel [32]. These policy dialogues were selected because (i) they followed similar formats that met the effectiveness criteria spelled out in the literature [8]; (ii) they included national, provincial and regional policy-makers and stakeholders in Canada (see Table 1); (iii) sufficient time had passed that research informants could comment on policy influence over the short- and medium-terms, while still being able to recall the events; and (iv) they focused on a high-priority issue (healthcare financial sustainability) that was expected to influence policy-making because of anticipated federal-provincial Health Accord negotiations.

Table 1 HFiT dialogue participants classified by policy role

Three conceptual frameworks were used as guides to the research. As discussed above, the Boyko model [8] relates features of policy dialogues (setting, mix of participants, use of evidence) to capacities of individuals, organizations and systems to engage in evidence-informed policy-making at each level over the short, medium and long terms respectively. The 3-I framework describes factors that influence policy development across the various stages of the policy cycle. The CFIR model describes characteristics of the intervention and implementation processes and contextual factors at multiple levels (individual, organizational, external policy and systems) that influence implementation of evidence-based options.

Data sources in this study included: (i) eight video recordings of commentary from 27 dialogue participants who volunteered to share their perspectives immediately after the initial events, (ii) 10 follow-up key informant telephone interviews with policy-makers and other stakeholders and (iii) a follow-up telephone focus group interview involving five researchers who had presented policy options at the dialogues. Most research attendees were presenting policy options for other participants to respond to. To capture diverse government, stakeholder and researcher perspectives, the lists of attendees of the dialogues were purposively sampled. As can be seen from Fig. 1, the numbers of participants by type varied by dialogue topic and event format. In the first three dialogues, participation was by invitation. The first dialogue focused on pharmaceutical pricing and health technology assessment at the national level and so had more national stakeholders compared to other dialogues. The second dialogue focused on hospital funding and physician payment and so included relatively greater provincial participation. The third dialogue on healthcare financing had relatively high national representation because of the joint federal-provincial role in healthcare financing in Canada. The iHEA symposium was a larger international event open to attendees of a research conference and so drew a larger proportion of researchers and national stakeholders.

Fig. 1

HFiT policy dialogue participants by dialogue and participant type

Interviews were held between May and June 2015. Semi-structured and focus group interview guides asked informants to reflect on their experiences at the dialogues, the helpfulness of attending the dialogue, and any influence of the dialogues on subsequent policy processes. The interviews averaged 45 min in length, and the focus group lasted 60 min. Data were professionally transcribed.

The thematic coding of the interview, focus group and video data proceeded through two stages. Research team members initially reviewed all the data to obtain an overall impression of the findings. A subset of transcripts was then thoroughly reviewed by two researchers to reveal common themes. There was a very high level of agreement across coders (85%) based on an initial sample of 10% of coded content. The resulting codebook was used to analyse the remaining transcripts. We focus in this paper on comments about the influence of the dialogue on capacities, actions taken and influence on subsequent policy-making rather than on participants’ views of the policy options themselves.¹ We included any subthemes within the Boyko model (e.g. enhanced mutual understanding as an individual capacity) that were mentioned by a minimum of two informants. During this stage of coding, we identified emergent themes that pertained to how those capacities were shaped by the dialogue and any subsequent influence on policy-making (e.g. seeding new ideas) within the 3-I framework during the agenda setting and formulation stages of the policy cycle as discussed in the Boyko model. In the second stage, team members recoded the data using the categories of the CFIR model to reflect informant statements about contextual factors that could shape implementation of the options discussed at the dialogues [26].

In the final stage of analysis, we synthesized the various themes to develop a conceptual framework and set of illustrative figures that propose pathways by which policy dialogues can influence capacity for evidence-informed policy-making at different stages of the policy cycle. Informants had an opportunity to review relevant portions of the thematic analysis and quotes attributed to them to ensure their accuracy. Ethics approval was granted from the McMaster University Research Ethics Board.

Results

We first discuss the thematic analysis across using each of the conceptual frameworks. Tables 2, 3, 4 and 5 provide illustrative quotes for each theme and subtheme, which are italicized in the text that follows. We begin with dialogue features (Table 2) and influence on capacity development (Table 3) consistent with the themes and subthemes of the Boyko framework [8]. We then discuss emergent subthemes about how the capacities that were developed influenced subsequent activities in the policy domain using the 3-I [19] (Table 4). We next discuss contextual factors that influence implementation using the CFIR framework [26] (Table 5). We then present a thematic synthesis that integrates themes across the three models, a conceptual framework, and propose potential pathways of influence of policy dialogues on different stages of policy-making. We indicate the informant type abbreviation (see details in the “Abbreviation” section) and number in square brackets following each quote. For example, R02 refers to the second researcher informant.

Table 2 Statements about dialogue features

Table 3 Statements about strengthened capacities for evidence-informed policy-making from attending dialogues

Table 4 Influence on factors that affect policy-making

Table 5 Influence of policy dialogues on policy implementation

Thematic analysis

Dialogue features

Informants felt the dialogues were well structured [see Table 2]. One informant felt that the environment provided an opportunity for “free-forum free-flow discussion … that, when you put people in rooms where they're wearing their hats and their titles you don't always have that kind of possibility” [NSO2]. Several informants described the environment as conducive to discussion and exploration of potential policy solutions [NSO2, PPM1, R03] on timely issues because a 10-year Health Accord between the provinces and federal government [33] was nearing completion and the possibility of a new accord was being discussed [FPM1]. Informants thought the dialogue participants represented “a good cross-section of key players” [FPM1] including individuals from “government, universities …various types of health organizations and non-governmental organizations” [R26]. The expert knowledge syntheses that were circulated as background to the dialogues and the presentations by their authors to set the stage for discussion [FPM1] were described as helpful because they provided in-depth analysis of each option in theory and practice.

Influence on capacity development

The opportunity to discuss “a whole range of issues” [NSO2] [see Table 3] enhanced mutual understanding. New policy solutions and different ways of thinking about the problem emerged from hearing the full range of perspectives and sharing participants’ experiences [NSO2]. Informants also described how they made important connections and developed relationships with others who they normally would not have had the chance to connect with. One informant commented that these types of events are designed to “make you feel good about yourself…that’s important because people … get a little bit more confident” [PPM3] to undertake action.

Over the medium term, informants described using what was learned at the dialogue within their organizations, for example, during “more sporadic briefing at the minister’s office” as a way to move the topic forward in agenda setting by saying “we’d had a chance to speak to a bunch of people across Canada…” [PPM2]. The discussions also helped in policy formulation because as one informant explained, they were “…in the back of my mind as we went through some … discussions at that time about how to do this [reform]” [PPM2]. The face-to-face contact made it easier to follow-up with people from other jurisdictions for clarification during policy development after the event [PPM3]. Several informants discussed how their organization was better prepared when policy windows opened and how “even if it's unpredictable - you have to have people who are ready and have the information, and have some commitment and understand how to do things.” [PPM3].

Few interview informants discussed longer-term capacity development at the systems level, which is not surprising since only 4 years had elapsed since the dialogues. However, researchers did comment on how the dialogues helped them better understand the priorities of policy-makers and to identify areas where new knowledge and new types of knowledge exchange products could significantly impact health systems over the longer term.

Influence on policy-making

Informants also described how these capacities influenced ideas, interests and institutions in policy-making [see Table 4].

Informants described how the dialogue discussions provided a chance to view the broad policy landscape by offering a “bird’s eye view” of different views around issues, expanded the range of policy ideas by “seeding new ideas” [FPM1] and helped “socialize ideas” by contextualizing research evidence to determine feasibility of options based on participant feedback [PPM1]. These all contributed to policy ideas that could shape how an issue was framed to advance it on the policy agenda.

Over the medium term, one federal policy-maker described how the dialogues shaped options for discussion, as (s)he was “factoring in” what (s)he had heard at the policy dialogue in terms of the “advice that I put up the line” [FPM1] during policy formulation. Informants also described how dialogue discussions helped prioritize options and actions, which helped policy-makers decide where to put their energies, based on “the relative importance of different mechanisms, or the relative likelihood of payoff” [PPM1]. One key informant stated “You could see when it was a policy option that was … embraced, that there [were] incentives to put them in place as opposed to other options where…they could see there were more limitations” [PPM4]. Informants described how a “tightened consensus” was developed among participants about what needs to happen [FPM1]. One informant described how the dialogue brought to light a “collective consciousness” [R7] among the policy-making and stakeholder communities that may have already been forming on these issues.

Similarly, the enhanced mutual understanding and relationships that were developed influenced interests. Informants described how the dialogue created “a more cohesive policy community” [R7] that was“…less confrontational, because more leaders from different sides of the equation knew each other” [NSO2] and that “communities [were] strengthened, just by opening more effective lines of communication” [NSO2] through face-to-face connection. Informants explained that non-attribution of comments made the dialogue an environment that was “non-confrontational; there are no bags of money on the table” [NSO2], where typically opposing groups could identify a common ground. A regional health authority decision-maker explained how attending the dialogue “… helped me build a little bit better bridges at the time, with the people … at our own ministry of health” [PPM3]. However, lack of will for change on the part of governments and stakeholder organizations was described as an impediment to advancing the policy issues being discussed at the dialogues.

Many informants felt that Canada’s institutional structure challenged major reform because “the more you have decentralization and devolution, the more, actually, the need for that central information requirement” [R11]. Several informants suggested that the federal government could play a greater role in gathering and sharing evidence and in establishing benchmarks and common performance measures at the national level to support system management and in evaluating any reforms undertaken [PPM1]. This could enable information about what was working in each jurisdiction to be shared and set the stage for pan-Canadian approaches, such as joint purchasing of pharmaceuticals.

Informants also discussed factors that influence policy implementation [Table 5]. One informant discussed how the strength and quality of evidence presented at a dialogue could either make a compelling case to take it forward or not [R08FG]. Another described how the dialogues helped informants understand the relative advantage of different financing options in terms of which has “the biggest potential payoff” [R01SI], the adaptability of options such as hospital funding approaches in rural settings, and the need for flexibility when it comes to physician payment depending on the context. Informants also discussed how the dialogues packaged evidence together as multiple options in a way that was more accessible to policy-makers. Finally, discussions of cost were seen to be important, particularly the opportunity cost of selecting one reform over another. In this way, the dialogues expanded participants’ knowledge and beliefs and self-efficacy to advocate for options within their organizations.

A number of themes related to the external context of the organizations represented at the dialogues were also discussed. For instance, several informants appreciated having a journalist present at one dialogue to represent public perceptions to convey patient needs and resources. The cosmopolitanism of participants’ organizations were enhanced through newly established relationships, which made it easier to follow up and ask questions about how implementation unfolded elsewhere. This also contributed peer pressure for reform in areas such as hospital funding. Many informants also pointed to a changing set of external policy and incentives that impeded implementation of many reform presented at the dialogues. Informants were clear that anticipation of a new funding agreement between the federal and provincial governments at the time of the dialogues was a key motivator for the dialogue discussions; however, momentum had been lost once the decision was made not to renew a health accord several months later [34]. This effectively closed a policy window because informants knew that there would “be lack of any kind of national … impetus behind these things” [PPM2], without which there is no strong incentive “… to provinces to have dialogue sessions … that can sort of bring them together” [PPM2]. As one informant stated, the dialogues were helpful at the time they were held, but given that the external environment for policy-making can change suddenly and make them less useful [PPM3].

Informants also discussed characteristics within organizations. Some were amenable to change through policy dialogues, while others were not. For example, the rate of turnover of key staff within governments and regional health authorities was pointed to as a structural factor that severely limited organizational capacity for implementing reform. In addition, the culture within government was described as being highly risk averse, with public servants facing a “web of rules” that could impede implementation [PSO2]. In contrast, the dialogues were seen as having the potential to shift the relative priority that organizations might give to different policy options and the extent of leadership engagement. For example, one key informant described how the spotlight being placed on hospital funding through the dialogue series encouraged one provincial finance minister to build on this momentum by hosting a similar dialogue at the provincial level and to establish an external advisory committee to advance reform. More often, however, informants felt that politicians and organizational leaders were not leading discussions about potential policy options. In part this reflected a lack of tension for change within healthcare organizations as the rate of growth of spending on healthcare began to diminish. Another challenge that could arise was having available resources to support implementation. On the other hand, the knowledge and information shared at the event were helpful in planning, expanded knowledge and beliefs about the proposed policy interventions and self-efficacy to undertake reform were enhanced.

From a process perspective, key informants described the potential for policy dialogues to assist with planning for implementation and about the importance of engaging opinion and implementation leaders. Lack of engagement of the most senior levels of government officials was described as “the biggest barrier to real change” [PB02SI] arising from most policy dialogues. For example, engaging an external implementation leader from outside government who “could take the heat” if anything went wrong was an effective strategy in one province. In another, the establishment of a 3-year expert panel as an external change agent following the dialogue facilitated implementation of hospital funding reform. The dialogues also led participants to reflect on evaluation approaches to meet the stated objectives of implementing reforms. There was considerable discussion about the need to be clear about the outcomes that were to be measured, including “measur[ing] the health gain” [R02] associated with proposed funding options in addition to economic impacts.

Thematic synthesis, conceptual framework and pathways of influence

In Fig. 2, we present a conceptual framework that combines the thematic analysis across frameworks. It illustrates how dialogue features lead to capacities, which influence policy ideas, interests and institutions in agenda setting and policy formulation that can both shape and be shaped by contextual factors and processes that affect implementation.

Fig. 2

The dialogue to policy—web of influence. The dialogue to policy web of influence framework integrates elements of three frameworks from the published literature: the Boyko, 3-I framework and CFIR frameworks. The features needed for an effective policy dialogue and the capacities that can be developed through attending a policy dialogue (Boyko framework) are presented on the far left. Short-term individual and medium-term organizational capacities correspond to individual and organization inner setting factors identified in the CFIR framework shown immediately to the right. These capacities have the potential to influence the various stages of the policy cycle (agenda setting, formulation, implementation and evaluation) as they shape ideas of policy actors (policy-makers and other stakeholders) and their interests in light of institutional factors (the 3-I framework). The outcomes of policy processes pertain to the policy interventions at various stages of development across the cycle shown at the centre of the Venn diagram. The subthemes raised by study key informants are shown within each element of the conceptual framework and may link to other elements of the CFIR framework (e.g. external policy and incentives within institutions and patient needs and resources, cosmopolitanism and peer pressure within interests). The double arrow at the bottom underscores how engaging opinion and implementation leaders and external change agents during or in follow-up to a policy dialogue about the ideas discussed may support implementation processes of planning, reflecting and evaluation, consistent with CFIR. Similarly, the CFIR framework points to characteristics of policy interventions that influence their implementation and that can be part of the policy dialogue discussions

In Fig. 3a–c, we illustrate and tease apart what one informant described as a “tangled web” [PPM1] of influence of policy dialogues on policy-making at the agenda setting, formulation and implementation stages of the policy cycle respectively.

Fig. 3

a–c Present potential pathways of influence of policy dialogues on policy-making by illustrating potential links between the various elements of the conceptual framework with policy-making at the agenda setting, formulation and implementation stages of the policy cycle respectively. Note that the suggested pathways do not imply direct causation but are meant to illustrate areas where policy dialogue designers may want to focus efforts to increase likelihood of influence of a given dialogue on policy processes. a Pathways of influence of policy dialogues on agenda setting. Illustrates potential pathways by which the capacities developed at policy dialogues can influence the ideas and interests within the 3-I framework and in turn the capacity to move different policy options onto public policy agendas at the agenda-setting stage of the policy cycle, as well as challenges to the influence of any given dialogue. The features of an effective policy dialogue are shaded ovals at the top of the diagram and the capacities developed from the Boyko framework are shown as boxes marked with an x. Potential linkages between capacities, the 3-I framework elements or the capacity for agenda setting are shown in boxes with upper and lowercase text. The elements of the 3-I framework (uppercase font) are illustrated as a triad of influence on the capacity for agenda setting. For example, with the right mix of features (evidence, participants and environment), policy dialogues can enhance new thinking by seeing and socializing ideas and providing a view of the policy landscape that influence ideas in circulation among policy-makers and stakeholders. Enhanced mutual understanding and relationships can foster a more cohesive policy community and face-to-face communication and strengthen the will to undertake reform among interests. These influences on ideas and interests and realities of the institutional setting, competing ideas and interests will together influence both capacity for agenda setting and which policy options make it onto policy agendas. b Pathways of influence of policy dialogues on policy formulation. Illustrates potential pathways by which the capacities developed at policy dialogues can influence ideas and interests within the 3-I framework and in turn the capacity to formulate policy solutions during the policy formulation stage of the policy cycle. The features of an effective policy dialogue are shaded ovals at the top of the diagram from the Boyko framework. The capacities described in the Boyko framework that are developed through policy dialogues are marked with an x. In addition to the capacities developed in Fig. 3a, a policy dialogue can enhance mutual understanding among dialogue participants that can help in shaping options, reaching a tighter consensus and assisting in setting priorities among options. These in turn can influence ideas and interests, which along with institutions can influence organizational capacity for policy formulation, and the policy options that are ultimately developed, as represented by the oval at the bottom of the figure. c Pathways of influence of policy dialogues on implementation. Illustrates potential pathways by which the capacities developed at policy dialogues can influence the implementation stage of the policy cycle. Constructs from the CFIR framework are represented by unshaded rectangles with flattened corners. For example, characteristics of interventions that influence implementation such as evidence strength, cost, adaptability, relative advantage and design and packaging of options can be shared as part of the evidence presented at the dialogue. Additionally, opinion leaders, implementation leaders and change agents can be engaged at or in follow-up to the dialogue to enhance likelihood of implementation. Building on the pathways of influence shown in Fig. 3a-b, new thinking enhances individual knowledge and beliefs about options that can contribute to organizational knowledge and priority setting, while relationships may lead to peer pressure and/or greater cosmopolitanism that along with enhanced self-efficacy can affect leadership engagement and tension for change within an organization. These factors affect organizational capacity for implementation including planning, reflecting and evaluating processes. External policy and incentives (including ideas, interests and institutions) and organizational capacity together will shape the likelihood and nature of policy implementation

Figure 3a illustrates pathways by which policy dialogues influence agenda setting. Through effective use of evidence, stakeholder representation and meeting environment, policy dialogues can encourage short-term individual capacities that include new ways of thinking, mutual understanding, relationships and empowerment. Potential pathways of influence on agenda setting are as follows:

• Pathway A1: Policy dialogues build participants’ individual capacities for new thinking by seeding and socializing new ideas and providing a view of the landscape that contributes to policy ideas that shape problem definition in policy-making and organizational capacity for agenda setting.

• Pathway A2: Policy dialogues build participants’ individual capacities for mutual understanding and relationships that can foster a more cohesive policy community and face-to-face connection and may create greater will for change among interests that contribute to organizational capacity for agenda setting.

• Pathway A3: Enhanced empowerment interacts with these influences on ideas and interests over time to promote greater organizational level capacity to advance policy problems onto the agenda.

Whether policy problems that make it onto the agenda move forward in the policy cycle will depend on other ideas, interests and institutional factors.

Figure 3b illustrates pathways through which policy dialogues can influence policy formulation. In addition to the influence of participants’ enhanced mutual understanding on policy ideas and interests discussed above with respect to agenda setting,

• Pathway B: Enhanced mutual understanding developed at policy dialogues can assist in shaping options for discussion, prioritizing options and creating a tighter consensus around ideas that can enhance organizational capacities over the medium-term not only for problem definition but also for formulation of policy solutions.

Figure 3c illustrates pathways through which policy dialogues can influence policy implementation. In addition to the influences of dialogues on agenda setting and policy formulation discussed above, there are a number of pathways by which policy dialogues may influence implementation by expanding (i) knowledge of key characteristics of options and shaping (ii) individual characteristics, (iii) organizational characteristics, (iv) the outer setting of organizations and (iv) implementation processes.

• Pathway C1: The evidence presented at policy dialogues can include information about the relative advantage, cost, adaptability and evidence strength of different policy options and package them in a way that may be more attractive for implementation.

• Pathway C2: Greater individual empowerment can lead to greater self-efficacy and enhanced organizational capacity for implementation.

• Pathway C3: Individual capacities for new thinking can enhance individual knowledge and beliefs about the options that enhance mutual understanding, contribute to a tighter consensus about policy ideas and shape organizational priorities, as well as organizational access to knowledge and information to support organizational capacity for implementation.

• Pathway C4: The relationships developed at the dialogue that create a more cohesive community can also foster greater cosmopolitanism and peer pressure that may contribute to tension for change, enhancing organizational interest in policy implementation.

• Pathway C5: The external policy context of organizations as shaped by the interacting forces of ideas, interests and institutions may influence the extent to which organizations are successful in advancing knowledge shared at policy dialogues at all stages of the policy cycle, including implementation.

• Pathway C6: Policy dialogues have the potential to influence implementation processes by engaging opinion and implementation leaders and external change agents and providing input to organizational processes of planning, reflecting and evaluating. Leadership engagement can also help to establish greater tension for change within organizations and engage organizational leaders as interests in policy implementation.

Informants also pointed to several organizational contextual factors that are less amenable to influence through policy dialogues such as staff turnover, poor communication and a risk adverse organizational climate.

One element in an incremental process

While informants described policy dialogues as “an important part of the process of policy change”, along with “generating useful research … hearing views and doing appropriate policy analysis” [FPM1], they also indicated that it is difficult to attribute any direct causal relationship, because policy moves often moves slowly over time.

It's hard to really point to something really specific that this particular series of conversations gave rise to. These areas of policy, they change so slowly over time, and it's really kind of a building thing, an accumulation of pieces of work, of conversations, of positioning by stakeholders and governments, and you see movement over time. [FB11SI]

Dialogues were described as contributing to a continuing conversation that may last many years within the public debate before any action is taken. One informant described the work as being “extremely influential” and “picked up by a lot of other people,” but that “…it's just overall, a whole lot of different pieces of work that come together to give you a picture” [R4].

This involved a whole bunch of motivated people. We're taking some good pieces of research. We're talking about how those things influence decision-making, but at the end of the day, no one in government is going to stand up and say, It is because of - only because of - the discussion that was at this conference. [PPM1]

This temporal aspect is difficult to capture in the figures because the journey through the policy cycle is non-linear and may be disrupted as priorities shift, for example, due to external events, shifting political considerations, realignment among interests [17] or poor evaluation results that send a policy back to an early stage of the policy cycle.

Discussion

Our findings support the role that policy dialogues can play in developing individual, organizational and system level capacities for policy reform, as has been discussed in the literature [8, 9, 35, 36], and illustrate the multiple factors that can influence whether they impact policy decision-making, typically through an incremental influence on policy processes over time. The analysis suggests that the intersection of interests, institutions and ideas, as well as characteristics of individuals, organizations, policy options and the external environment and implementation processes can influence the impact of a policy dialogue on capacity development and policy-making. It is also important to recognize that many of the factors described in Fig. 3a–c that can be influenced by policy dialogues can also act as barriers to influencing policy decision-making at different stages of the policy cycle. For example, some options discussed at dialogues do not move forward because of weak evidence, lack of will for reform, insufficient leadership engagement, or lack of self-efficacy or peer pressure.

By teasing apart these mechanisms, the findings point to considerations that designers of policy dialogues may want to target in order to meet specific objectives related to the stage of the policy cycle. It may be beneficial for organizers to reach out to participants prior to the dialogue to assess the contextual factors they face in advancing options at the individual, organizational and external policy levels and tailor discussions at the events accordingly. This could include asking participants to reflect on their own readiness for implementation, the implementation climate of their organizations and external environment and potential implementation processes. Such advance preparation may enable the discussions at the dialogue to focus on what it takes to move evidence into practice to inform various stages of policy development across the policy cycle. If the objective is to help to move an issue onto the policy agenda or shape ideas for policy formulation, the range of evidence and expert input should ensure that participants have the opportunity to view the full landscape of stakeholder perspectives, to hear new ideas, and that the structure of the discussions create processes by which evidence can be “socialized” and prioritized into feasible policy options [17]. If the objective is to influence implementation, the selection of dialogue participants should include senior decision-makers, opinion leaders and external change agents, or they should be targeted in follow-up activities to contribute to the tension for change that can shift organizational priorities [4]. The findings also suggest that there needs to be a realistic consideration of the contextual influences before embarking on policy dialogues intended to promote reform, to avoid wasting resources if there is an absence of organizational capacity and system level will for change or if institutional factors such as external policies and incentives or performance measures are not in place. The framework may be especially helpful in future work to more clearly evaluate the success or failure of dialogues against stated goals in light of the policy and organizational context.

The model makes an important contribution by helping to address gaps in the implementation literature identified by Bowden and colleagues [10] with respect to policy considerations, such as the influence of evidence on policy networks, the need to consider capacity for implementation at multiple levels, while recognizing that effective knowledge transfer is not a “one off” event, but instead a powerful and continuous process in which knowledge accumulates and influences thinking over time.

At the same time, this research is subject to several limitations. In any study of this kind, there are concerns about how well interview informants will recall the content discussed at dialogues held several years earlier and how this affects their interpretation of subsequent policy developments. In addition, there may be other contextual influences beyond those mentioned here.² There is also the risk that study participants feel more positive about the influence of policy dialogues than those who declined to participate. Finally, the findings rest on a single case involving four national level dialogues in Canada related to healthcare financial sustainability. The framework may serve as a basis for future research to test the proposed pathways of influence developed here while studying the influence of policy dialogues in other contexts.

Conclusions

The findings provide a conceptual framework and illustrate pathways by which policy dialogues can advance the use of evidence to inform policy content and processes. Particular attention should be placed on the intended objective of the dialogue in influencing different stages of the policy cycle, while creating a dialogue environment that can help participants to expand capacities. Strategic thought must be given to the characteristics of the organizational and political context to assess whether there is potential for knowledge gained at the dialogue to be put into practice.