- Study protocol
- Open Access
- Open Peer Review
Protocol for the process evaluation of interventions combining performance-based financing with health equity in Burkina Faso
Implementation Science volume 9, Article number: 149 (2014)
The low quality of healthcare and the presence of user fees in Burkina Faso contribute to low utilization of healthcare and elevated levels of mortality. To improve access to high-quality healthcare and equity, national authorities are testing different intervention arms that combine performance-based financing with community-based health insurance and pro-poor targeting. There is a need to evaluate the implementation of these unique approaches. We developed a research protocol to analyze the conditions that led to the emergence of these intervention arms, the fidelity between the activities initially planned and those conducted, the implementation and adaptation processes, the sustainability of the interventions, the possibilities for scaling them up, and their ethical implications.
The study adopts a longitudinal multiple case study design with several embedded levels of analyses. To represent the diversity of contexts where the intervention arms are carried out, we will select three districts. Within districts, we will select both primary healthcare centers (n =18) representing different intervention arms and the district or regional hospital (n =3). We will select contrasted cases in relation to their initial performance (good, fair, poor). Over a period of 18 months, we will use quantitative and qualitative data collection and analytical tools to study these cases including in-depth interviews, participatory observation, research diaries, and questionnaires. We will give more weight to qualitative methods compared to quantitative methods.
Performance-based financing is expanding rapidly across low- and middle-income countries. The results of this study will enable researchers and decision makers to gain a better understanding of the factors that can influence the implementation and the sustainability of complex interventions aiming to increase healthcare quality as well as equity.
Universal health coverage entails that everyone should have access to good quality healthcare services without experiencing financial hardship resulting from healthcare payment . Despite major healthcare reforms in many low- and middle-income countries (LMICs), access to quality healthcare remains inadequate. On average, in LMICs, the maternal mortality ratio is 230 for 100,000 live births and the mortality ratio for children under the age of 5 is 53 per 1,000 ,. Meager improvements have led to heated international debates on the best policies to improve access to quality healthcare services and reduce health inequity.
Recently, numerous LMICs have introduced performance-based financing (PBF) to improve the delivery of healthcare services. The World Bank, for instance, supported the design and implementation of PBF in over 30 LMICs ,. Different PBF labels exist and are associated with different types of incentives and payment arrangements ,. PBF generally entails that performance agreements between Ministries of Health and healthcare facilities are established to define indicators and targets to be reached in delivering these services. According to the model proposed by the World Bank, bonus payments to health providers are made after verification by reviewers of the quantity and quality of healthcare services. Across settings, facilities retain different degrees of autonomy in deciding how to use PBF payments. The underlying assumption of PBF is that financial incentives, combined with increased supervision and enhanced autonomy, will motivate health providers to improve the quantity and quality of services delivered .
The body of evidence on the implementation and effectiveness of PBF is not well established. In Rwanda, for instance, PBF increased the number of assisted deliveries, children's consultations, and perceived quality of care ,. Meanwhile, in Democratic Republic of Congo, PBF resulted in lower out-of-pocket payments for patients while receiving equal or better quality care compared to those in the control group . On the other hand, a Cochrane systematic review  and a literature review found that several dimensions of PBF have not yet been adequately studied (e.g., impact on equity, organizational change, stakeholder satisfaction) ,. The difficulties associated with the PBF program in Uganda demonstrated the importance of considering the context and the implementation process . These difficulties included the lack of consideration of other health programs implemented simultaneously, insufficient communication between stakeholders, overly rapid selection of performance indicators, inability to modify inappropriate indicators over time, implementation delays causing institutional memory loss, limited ability of auditors to identify and tally contract-relevant services, overwhelming workload of extracting performance data, insufficient feedback meetings, and financial shortfalls .
Researchers have suggested that several dimensions must be taken into account when evaluating any PBF intervention, including the characteristics of the healthcare system. Most studies on PBF focus on primary healthcare facilities. Local stakeholders have highlighted the need to examine the implementation process of PBF in higher level health facilities. Other elements to be considered include the governance structure, organizational cultures, training activities, nature of contracts, redistribution of incentives, indicators of performance, mechanisms to monitor performance, means of dialogue between stakeholders, power relations, community involvement, and perceptions of stakeholders ,-.
In addition, one must consider that PBF can also trigger unintended processes and effects. Financial incentives might erode intrinsic motivation  or lead to the neglect of non-incentivized services, as observed in Latin America and Rwanda ,. Other risks include the migration of health personnel to better performing health centers, provision of healthcare to easily accessible populations, a decrease in the quality of care, a reduction in transfers of patients to other health facilities, and manipulating data regarding performance (gaming) .
Combining supply- and demand-side interventions
Achieving universal health coverage will likely require a comprehensive approach that simultaneously tackles supply-side and demand-side barriers to effective healthcare coverage. Offering financial incentives to motivate healthcare providers may not be sufficient to tackle all the barriers that influence access to healthcare (e.g., user fees). Without making specific provisions for the poor, PBF is unlikely to produce the same benefits for all. To promote equitable access to healthcare, PBF could be combined with pro-poor targeting strategies and/or community-based health insurance (CBHI). Thus far, empirical evidence on the feasibility of combining PBF with other supply- or demand-side interventions that address social protection is scarce.
In LMICs, pro-poor targeting has been implemented both on the supply side and the demand side. On the supply side, one strategy is to pay healthcare providers more for services offered to the poor compared to the rest of the population . On the demand side, one pro-poor targeting strategy is to exempt the worst-off from paying user fees. One challenge is the identification of the poorest. Recent studies have shown that community members can use selection procedures to identify the poor -. Such participatory processes to identify the poor has been found to be socially valued and effective in rural areas, but less so in urban areas. The challenges of pro-poor targeting strategies include conflicts of interests of management committees, lack of knowledge about policies, inability to travel in order to benefit from the services, apprehensions of healthcare providers, and diminished quality of care ,. Rigorous studies on pro-poor targeting strategies are very rare in Africa, resulting in a knowledge gap on their implementation processes . Thus far, no intervention or research program combined PBF with a community-based selection of the poor.
On the demand side, CBHI could be combined with PBF to increase risk sharing among the population. Since the 1990s , several experiments in sub-Saharan Africa have shown that CBHI membership facilitates access to healthcare - and offers a degree of financial protection -. However, participation in CBHI generally remains very low (10%-15% penetration rate ). Access to CBHI is often not equitable because the poorest populations do not have the financial capacity to pay membership fees -. Even when they become members, the poor are generally unable to benefit from healthcare coverage to the same extent as others due to indirect costs or copayments ,. Moreover, healthcare providers often perceive the implementation of CBHI as a threat to their personal interests -, which has important repercussions on the quality of services provided ,.
Although CBHIs have adopted a wide range of initiatives involving various stakeholders, none of them have been able to resolve the issue of low membership on their own . To date, however, there have been no studies on the combination of PBF with CBHI.
It is essential to produce evidence on the feasibility of implementing combined interventions with synergetic potential, as a means to address equity concerns related to traditional PBF approaches. In Burkina Faso, the implementation of various intervention arms combining PBF with CBHI and pro-poor targeting offers a unique opportunity to explore existing gaps in knowledge.
In Burkina Faso, nearly half of the population lives below the poverty line and more than 100,000 children under the age of 5 die each year . The probability that a newborn baby will die before reaching the age of 5 is 102 per 1,000 . The maternal mortality ratio is 400 per 100,000 live births . In the poorest quintile, only 12% of women receive prenatal care at least four times during their pregnancy, compared to 32% of women in the richest quintile . The percentage of births attended by a skilled healthcare provider also ranges widely from 19% in the poorest quintile to 84% in the richest quintile . With such important inequalities in access to care ,, the health of the poorest population is a concern.
In 2011, the Government of Burkina Faso launched a pilot PBF intervention targeting maternal and child healthcare services in three districts. In 2012, the evaluation of the pilot project  found a substantial improvement in the quantity and quality of healthcare services provided . However, evaluators were not able to establish a causal relationship between PBF and the outcome variables due to the study design. With regard to the implementation process, results highlight the importance of ensuring that stakeholders are well informed, feel involved in the process, and accept the rules relating to incentive sharing. However, given the small scale of this evaluation, it remains unclear how the different elements of the PBF intervention, the context, and the stakeholders involved influenced each other to affect the intervention's effectiveness.
In 2013, the government decided to expand PBF to 12 districts in six regions, covering a population of approximately four million people. The operation is financed by the World Bank and carried out by the Ministry of Health's Technical Support Unit in charge of PBF (TSU- PBF). The objective remains that of improving the utilization and quality of maternal and child healthcare services. In comparison to the earlier pilot project, PBF will be combined with additional interventions directly targeting health equity: demand- and supply- side pro-poor targeting and CBHI. The objective of combining PBF with pro-poor targeting and CBHI is to ensure that healthcare improvements are accompanied by adequate measures to enhance equity in utilization rates and secure financial protection. Four intervention arms are being implemented to compare their outcomes:
Performance-based financing only (PBF 1): Health facilities are paid according to the quantity and quality of healthcare services delivered. Patients are required to pay user fees. PBF performance agreements concluded between the MOH and the health center define the package of basic services to be provided and the indicators and targets to be reached in delivering these services. The results achieved against these targets are assessed by external reviewers every 3 months. Based on these verified results, each facility under a PBF contract receives payments in partial reimbursement for the services delivered. The payments are based on unit prices, based on a number of factors designed to achieve the desired results. This includes the basic cost of the inputs required (and not financed elsewhere) for services to be rendered, adjusted for quality of the service. This intervention group does not include any systematic targeting and subsidization of the poor.
Performance-based financing + systematic targeting and subsidization for the poor (PBF 2): PBF is combined with a community-based selection of the poor (approximately 15%–20% of the population) who are entitled to benefit from user fee exemptions. A specific PBF indicator is introduced for purchasing services for the poor, using a fixed unit price for each consultation package (including services and drugs). The fixed unit price is higher than for non-targeted patients, taking into consideration the delivery of care without direct payments from patients. If issues of oversupply arise after the first few months of implementation, a cap may be established so that a maximum of about 10% of consultations can be reimbursed as consultations offered to the poor. In that case, health facilities would receive the normal rate for consultations offered to the poor exceeding the 10% cap.
Performance-based financing + systematic targeting and subsidization for the poor + provider motivation (PBF 3): PBF is combined with a community-based selection of the poor who are entitled to benefit from user fee exemptions (approximately 15%–20% of the population). Healthcare providers are paid more for services provided to the poor than in PBF 2. The multiplication factor varies depending on the services. If issues of oversupply arise, a reimbursement cap of about 10% may be established. The higher reimbursement rate acts as a financial incentive for healthcare providers to take personal initiative to increase the utilization of healthcare by the poorest households.
Performance-based financing + community-based health insurance (including targeting of the poor) (PBF 4): PBF is combined with CBHI offered to the general population. The insurance premiums of the poor (approximately 15%–20% of the population) are subsidized. Healthcare services are purchased at varying payment levels, as defined in PBF 3 (with higher payment levels for services provided to the poor). Thus, subsidization of services provided to the poor who do not pay the premiums will pass via the PBF mechanism at budget-neutral levels, while capitation payments will replace out-of-pocket payments for the general population.
Primary healthcare facilities are randomly allocated to the different groups. All district and regional hospitals included will be allocated to PBF 2.
The theory of the overall intervention combining PBF with pro-poor targeting and CBHI is presented in Figure 1. As with any real-life intervention, these four modalities (PBF 1, 2, 3, and 4) may be modified or adapted in the future. Four of the six regions implement PBF 1, PBF 2, and PBF 3; one region implements PBF 1 and PBF 4; and one region only implements PBF 1.
The World Bank also finances the impact evaluation of the four intervention arms. The impact assessment adopts a blended experimental and quasi-experimental design. For the experimental portion of the design, facilities within PBF districts are randomly assigned to PBF intervention arms to test the effects of the additional components (pro-poor targeting - PBF 2 and 3 and CBHI - PBF 4) versus PBF alone (PBF 1). For the quasi-experimental portion, facilities are matched with other facilities in neighboring control districts to compare the effects of the intervention arms with the absence of any intervention (counterfactual). Two comparison districts in each of the six regions, where the intervention districts will take place, have been selected to act as comparison health facilities. The research protocol presented in this manuscript is not for this impact evaluation. Our protocol focuses exclusively on the implementation process and, as such, adopts a design that is complementary to the one used for the impact evaluation.
General objectives and research questions
The overall objective of this study is to analyze the emergence, the fidelity, the implementation process, the sustainability, the possibilities for scaling-up the intervention arms nationwide, and the ethical implications of the intervention arms. The research questions (Q) are:
Q1. How was the decision taken to combine PBF with CBHI and pro-poor targeting?
Q2. Were the interventions' activities implemented as planned and why?
Q3. How do social actors perceive the interventions?
Q4. How were the interventions implemented?
Q5. What adaptation/invention/innovation strategies did the interventions trigger?
Q6. Did the interventions induce unintended effects? If so, which ones?
Q7. How did the different contexts shape the way the interventions were deployed?
Q8. Which specific activities or combination of activities produced changes in the performance of health centers? How were these changes produced over time?
Q9. Will the implementation process of the interventions allow their sustainability?
Q10. What is the potential for scaling-up the interventions nationwide?
Q11. Which ethical considerations are associated with the interventions?
The current study is anchored in methodological pragmatism -. When evaluating public health interventions, researchers are faced with complex and dynamic objects that are composed of various intertwined actions. These objects are often characterized by non-linear relations, organized in specific contexts, and understood as open systems that encompass the researcher ,. For this type of evaluative study , researchers must adopt a pragmatic position to answer research questions and provide recommendations to decision makers .
The methodological strategy was developed during a pre-evaluative phase , which took place between July and December 2013. Through literature reviews, workshops, and dialogue with stakeholders, the evaluability assessment phase provided a better understanding of the interventions arms, their components, and their expected effects. We developed the research questions and methodological strategy based on the knowledge gaps, the theory of intervention , the available budget, policy-maker needs, and a realistic timeline .
Types of evaluation
Several types of evaluations will be conducted as part of this study. Examples of typical questions representing each type of evaluation are presented in Table 1.
The assessment of emergence (Q1) will examine the factors that led to the decision to combine PBF with pro-poor targeting and CBHI. An analytical framework derived from the multiple streams approach will be used to understand public policy agenda setting .
Assessing the fidelity of implementation (Q2) is essential because stakeholders plan to implement a combination of interventions that they believe to be effective. But in order to be effective, the interventions must be either implemented as planned (to avoid type 3 errors ), or the changes/innovations brought about must be beneficial. The evaluation of fidelity compares the activities that were initially programmed with those that were implemented (content fidelity), as well as their level of achievement (fidelity of the temporal and geographical coverage) -.
The assessment of processes (Q3–7) will focus on the internal dynamics of the intervention arms, the roles, perceptions and coping strategies of actors, the unintended effects, the evolution of actions, and the mediating effect of the context ,. We also seek to understand local dynamics, the relationship between healthcare providers and the populations, the local history, and the socio-historical context that could affect the performance of primary healthcare centers (PHC) and district or regional hospitals, as well as other processes. We will also examine the adaptation processes, which may be essential to the interventions' effectiveness ,.
The performance assessment (Q8) will focus on activities related to the interventions in PHC and district or regional hospitals. It is related to an effectiveness analysis. Primary healthcare centers and hospitals will be considered to be effective when they are better than others (median) and they improve their score on indicators associated with the interventions' activities.
The sustainability assessment (Q9)  will focus on three types of events that help achieve different levels of sustainability ,: i) specific to sustainability (stabilization of resources and organizational risk taking); ii) jointly related to sustainability and to the implementation of the interventions (incentives for actors, adaptation of activities, alignment of objectives, transparency of communication, shared cultural artifacts, integration of rules); and iii) specific to the implementation (sufficient resources and compatibility with the host institution).
The conditions to potentially scale-up the interventions (Q10) will be studied by examining the characteristics of the innovation (e.g., combining PBF with CBHI and pro-poor targeting), the context, the host organization (Ministry of Health), the implementation strategy, and the support team (TSU-PBF, World Bank) .
The assessment of ethical considerations (Q11) will involve analysis of the moral and political issues associated with the design and implementation of the intervention arms. The goal is to go beyond ethical considerations traditionally associated with procedural ethics in order to examine a wider scope of ethical implications pertinent to the interventions.
Selection of the context for case studies
Since the social context and the district teams' leadership affect the performance of the PHC in Burkina Faso ,,, case studies will be conducted in 3 of the 12 districts. The sample size takes into account our resource constraints but is sufficient to represent a variety of contextual situations conducive to the process of analytical generalization . Six of the 12 districts had to be removed from the potential case studies because they do not represent the normal context of the healthcare system. Given the constraints and the need to select districts in different regions, the three selected districts are Diébougou (in the Southwestern region), Ouahigouya (in the Northern region), and Solenzo (in the Boucle du Mouhoun region).
Selection of contrasted cases
The cases will be PHCs representing different intervention arms and the district or regional hospital located within each of the three selected districts. Figure 2 illustrates the selection of cases within each district. The total number of cases will be 18 PHCs (out of a potential 106 PHCs) and three hospitals (n =21 cases). More specifically, in Diébougou and Ouahigouya, where three modalities are implemented (PBF 1, PBF 2, and PBF 3), we will select two PHCs for each of the three intervention arms. In Diébougou, we will also select the district hospital (DH). In Ouahigouya, we will select the regional hospital (RH) because there is no district hospital. In Solenzo, where only two intervention arms have been implemented (PBF 1 and PBF 4), we will choose six PHCs, that is three for each of the two intervention arms, and the district hospital.
The selected cases will be considered to be typical cases . For each intervention arm, we will choose the two or three most contrasted cases in terms of their initial performance (good, fair, poor) before the start of the interventions. We will define performance in relation to key activity indicators: number of births attended by a skilled healthcare provider, vaccination coverage, and number of preventive and curative consultations. We will assess the initial performance of each PHC over the 24 months preceding the intervention implementation. Based on a participatory approach with Health District Team member, we will select the PHC with the most contrasted performance scores using graphic reading and according to computed mean initial performance scores. The hypothesis of effectiveness and equity is the same for all PHCs, regardless of the initial performance. Thus, this case selection procedure follows the principal of “literal replication” whereby each case predicts similar results .
This evaluative research will adopt a mixed methods approach . We will give more weight to the qualitative data collection (QUALI) compared to the quantitative data collection (QUANT). The temporality and the type of data collected for the different types of evaluations are presented in Figure 3.
During the first month (M1) of the study, we will measure the initial performance levels of the PHCs to allow the selection of cases. Our previous work in Burkina Faso and that of other researchers have demonstrated the quality of the routine data ,,,. We will also organize a pilot case study in each district .
Three months (M3) after the beginning of the study, we will collect data to measure the fidelity of the intervention arms' implementation and to understand the process of implementation. Under the supervision of a researcher in each district, research assistants will be present in each case for a 2-week period. This relatively long immersion will not only create a sense of trust with the stakeholders, but also provide a better understanding of the contextual issues. On site, research assistants will conduct non-participant observation (e.g., at the health centers, in the villages). They will keep research diaries, take notes after informal interviews, and make ethnographic  summaries. The research assistants will begin formal data collection only during the second week in order to take into account the context and also the social dynamics by meeting important people. Research assistants will conduct in-depth interviews until we reach saturation of the data (at least 30 interviews) with key local stakeholders. This analysis (M3) will offer an opportunity to understand the performance of the PHC before the beginning of the interventions (Figure 3).
After 6 months (M6), a qualitative data collection round will be conducted on a national level to better understand the factors that contributed to the emergence of PBF with pro-poor targeting and CBHI. We will collect data from all of the available documentation and from approximately 20 in-depth interviews with key stakeholders. In-depth interviews will be conducted until we reach saturation of data.
After 12 months (M12), a second data collection phase will take place in all of the selected healthcare facilities. In terms of performance, quantitative data will be collected using the same method as in M1. However, without waiting for the results of these analyses, we will conduct in-depth interviews with healthcare providers and members of the management committee to capture their perceptions of the evolution of this performance. Following the same method used during M3, we will analyze the evolution of the fidelity of the implementation and the processes using document analysis, observations, and in-depth interviews. In addition, the sustainability and conditions to potentially scale-up the intervention arms nationwide will be studied through interviews conducted on a national level. We will conduct interviews until we reach saturation of data (approximately 30 interviews) with stakeholders.
At 18 months (M18), after analyzing the quantitative data on the evolution of the performance, research assistants will conduct a final 2-week field visit in each case. They will conduct interviews with key stakeholders to discuss and understand the quantitative performance data. Not only will we try to understand how the interventions are organized at M18 but, due to preliminary data analysis of M3 and M12, we will also conduct interviews on the evolution of the implementation and potential explanations for the challenges, modifications, and difficulties. We will also conduct the assessment of ethical considerations during this phase.
Each case has a specific historical, social, environmental, and political context as well as relations between the populations and healthcare providers. The temporal dimension of these relations is essential to understand, as the intervention arms will certainly bring about changes that will evolve over time ,. The method adopted to collect data should be able to face this challenge.
Analysis of case studies
The in-depth research strategy and use of multiple sources of evidence  reinforce the internal validity of the 21 case studies. We will organize the inter-case analyses as a stepwise process. After their field trips, each assistant will prepare a monograph of each case and a cross-sectional analysis of the intervention arm. We will conduct a quantitative analysis of the change in the performance of health centers by comparing the mean performance from the 24 months preceding the intervention with the mean performance obtained 1 year after the intervention. Then, each researcher will conduct a synthesis of the seven cases in the district. Finally, the principal investigator will analyze the 21 cases globally. For each of these steps, researchers will be able to move to a higher level of analytical generalization . This collective process, illustrated in Figure 4, will strengthen the validity of the conclusions.
A preparatory workshop between researchers (M1) will ensure a common understanding of concepts, analytical framework, and data collection strategy. An expert will give a workshop on the NVivo software to the research staff in order to establish a common platform for the data analyses throughout the study. We will record, transcribe, and code all interviews. We will conduct content analyses ,. We will organize collective analyses (workshop at M6 and M15).
There are no constraints or restrictions weighing on the autonomy and independence of the study or the publication of its results. Ethical committees in Burkina Faso and at the University of Montreal Hospital Research Center (CRCHUM) have approved the study. The study's objectives are aligned with Burkina Faso's national priorities. The Ministry of Health and donors have requested an evaluation of the four intervention arms. We will inform all relevant stakeholders and authorities about the purpose of the study and the presence of research assistants on the field. We will not disclose the performance of selected cases to the participants. We will obtain oral or written informed consent from the participants depending on their literacy level.
The study is financed through the first author's (VR) programmatic research funds allocated by the Canadian Institutes of Health Research (CIHR). The data collection will begin during summer 2014. The data cleaning or analyses will not have begun at the time of submission of this protocol.
An important strength of this protocol rests in its rigorous design. Langley et al.  have argued that process evaluations tend to adopt deceptively simple designs that rely on cross-sectional qualitative or ethnographic data. Many studies focus on only one level of analysis and fail to look at the evolution of the implementation process over time. For this study, we have adopted a longitudinal multiple case study with several nested levels of analysis. This study will collect real-time qualitative and quantitative data over a period of 18 months. We explicitly incorporated temporal progressions to help understand the intervention arms' complete life cycles, from their emergence to the conditions that may affect their sustainability. The prolonged involvement of researchers will allow them to be close to events and practices. Moreover, we will examine ongoing interactions among different groups of individuals and across multiple levels of healthcare provision. The study of PBF in higher level healthcare facilities is novel in Africa and responds to a knowledge gap. Overall, this complex study design will make a significant contribution to the field of health financing in LMICs as well as to the development of robust methodologies for process evaluations.
Moreover, the context in Burkina Faso provides a unique opportunity to examine the interactional implementation process of combined interventions. This study has been designed so that we can disentangle and compare the distinct contributions of PBF, pro-poor targeting, and CBHI. This will allow us to understand the added value of each component.
As with any population health intervention research, various methodological and operational challenges are likely to arise during the course of this study. One difficulty will be to avoid or control for the implementation of other studies or interventions overlapping with our area of interest. Health needs in Burkina Faso are urgent, so a number of prominent international health agencies are intervening with their own agendas or priorities at stake. The lack of coordination between all the actors makes it difficult to select cases with the assurance that no other confounding study or intervention will affect or contaminate our sample. Potentially confounding factors will be handled on a case-by-case basis, for example through negotiation with other stakeholders and adaptation of methodological strategies. Our close ties and constant communication with the country's Ministry of Health, funders, and local stakeholders will allow us to remain well informed throughout the study in order to take appropriate measures in a timely fashion, if needed.
Another limitation of this study will be the difficulty in establishing a causal chain between the inputs, the processes, and the results. Inferring causality for interventions implemented in a real-life setting is more complex, but on the positive side, results tend to be more pertinent for decision making. Another concern for this study is that multiple stakeholders have been directly implicated in the implementation of the interventions and may have vested interests in its promotion. Although the perceptions and strategies of the actors are part of the object of the present study, we must be wary of the possibility that participants may introduce biases in the study. Overall, we will try to strengthen the validity of the results by using multiple sources of data and varied methods of data collection and analysis.
This evaluation study will promote the utilization of the results. Throughout the study, the research team will collaborate with key decision makers and stakeholders to ensure that the research questions are pertinent and that the results will be useful for evidence-based decision making. This collaboration will flourish without influencing the scientific independence of research. We will organize workshops to plan the study and share results locally. We will disseminate the results using reports, policy briefs, and lay publications distributed in French and in English. These will also be shared on our webpage (www.equitesante.org). Moreover, we will present the results in local and international conferences. Lastly, we will publish scientific articles in open access journals to promote access to the findings.
Independent evaluations of PBF in LMIC are rare and mainly examine impacts. To date, little research has helped to understand issues related to the implementation process and the coping strategies of actors that are at the heart of complex interventions. Moreover, it is urgent to clarify how PBF can be combined with other approaches to promote equitable access to healthcare services. The implementation of interventions with high synergistic potential in Burkina Faso provides a unique opportunity to respond to the existing knowledge gap. In this article, we have presented a comprehensive protocol that aims to study the emergence, the fidelity, the processes, the sustainability, the conditions that can influence scaling-up, and the ethical considerations of four intervention arms that combine PBF with pro-poor targeting and CBHI. The complex mixed methods design will allow researchers and decision makers to understand the evolution of these approaches over time. Ultimately, the scientific evidence produced should be used to promote access to high-quality healthcare for vulnerable groups.
The World Health Report - Health Systems Financing: the Path to Universal Coverage. 2010, World Health Organization, Geneva
World Bank: Health, nutrition and population millennium development goals. 2012, , [datatopics.worldbank.org/hnp/HNPMDG]
Trends in Maternal Mortality: 1990 to 2013. Estimates by WHO, UNICEF, UNFPA, The World Bank and the United Nations Population Division. 2014, World Health Organisation, Geneva, Switzerland
Performance-Based Financing ToolKit. In Edited by Fritsche GB, Soeters R, Meessen B. Washington, DC: World Bank; 2014 , [https://openknowledge.worldbank.org/handle/10986/17194]
World Bank: Our project. 2013, , [http://www.rbfhealth.org/project/our-projects]
Eldridge C, Palmer N: Performance-based payment: some reflections on the discourse, evidence and unanswered questions. Health Policy Plan. 2009, 24 (3): 160-166. 10.1093/heapol/czp002.
Musgrove P: Financial and other rewards for good performance or results: a guided tour of concepts and terms and a short glossary: World Bank. 2011. , [https://www.rbfhealth.org/sites/rbf/files/documents/Rewards%20for%20Good%20Performance%20or%20Results%20-%20Short%20Glossary.pdf]
Witter S, Toonen J, Meessen B, Kagubare J, Fritsche G, Vaughan K: Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation. BMC Health Serv Res. 2013, 13: 367-10.1186/1472-6963-13-367.
Rusa L, Ngirabega J, Janssen W, Van Bastelaere S, Porignon D, Vandenbulcke W: Performance-based financing for better quality of services in Rwandan health centres: 3-year experience. Tropical Med Int Health. 2009, 14 (7): 830-837. 10.1111/j.1365-3156.2009.02292.x.
Basinga P, Gertler P, Binagwaho A, Soucat A, Sturdy J, Vermeersch C: Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet. 2011, 377 (9775): 1421-1428. 10.1016/S0140-6736(11)60177-3.
Soeters R, Peerenboom P, Mushagalusa P, Kimanuka C: Performance-based financing experiment improved health care in the Democratic Republic of Congo. Health Aff (Millwood). 2011, 30 (8): 1518-1527. 10.1377/hlthaff.2009.0019.
Witter S, Fretheim A, Kessy F, Lindahl A: Paying for performance to improve the delivery of health interventions in low- and middle-income countries. Cochrane Database Syst Rev. 2012, 15 (2):
Gorter A, Ir P, Meessen B: Evidence Review, Results-Based Financing of Maternal and Newborn Health Care in Low- and Lower-Middle-Income Countries, February 2013, study commissioned and funded by the German Federal Ministry for Economic Cooperation and Development (BMZ) through the sector project PROFILE at GIZ - Deutsche Gesellschaft für Internationale Zusammenarbeit. 2013.
Ssengooba F, McPake B, Palmer N: Why performance-based contracting failed in Uganda–an "open-box" evaluation of a complex health system intervention. Soc Sci Med. 2012, 75 (2): 377-383. 10.1016/j.socscimed.2012.02.050.
Borghi J, Mayumana I, Mashasi I, Binyaruka O, Patouillard E, Njau I, Maestad O, Abdulla S, Mamdani M: Protocol for the evaluation of a pay for performance programme in Pwani region in Tanzania: a controlled before and after study. Implement Sci. 2013, 8: 80-10.1186/1748-5908-8-80.
Falisse JB, Meessen B, Ndayishimiye J, Bossuyt M: Community participation and voice mechanisms under performance-based financing schemes in Burundi. Trop Med Int Health. 2012, 17 (5): 674-682. 10.1111/j.1365-3156.2012.02973.x.
Lutwama GW, Roos JH, Dolamo BL: Assessing the implementation of performance management of health care workers in Uganda. BMC Health Serv Res. 2013, 13: 355-10.1186/1472-6963-13-355.
Deci E, Ryan R: Intrinsic Motivation and Self-determination in Human Behavior. 1985, Plenum Press, New York
Taljaard M, Donner A, Villar J, Wojdyla D, Faundes A, Zavaleta N, Acosta A: Global survey on maternal and perinatal health research at hospital level: the case of Latin America. Paediatr Perinat Epidemiol. 2009, 23 (6): 574-581. 10.1111/j.1365-3016.2009.01072.x.
Kalk A, Paul FA, Grabosch E: `Paying for performance’ in Rwanda: does it pay off?. Trop Med Int Health. 2010, 15 (2): 182-190. 10.1111/j.1365-3156.2009.02430.x.
Schwartz J, Bhushan I: Improving immunization equity through a public-private partnership in Cambodia. Bull World Health Org. 2004, 82: 661-667.
Ridde V, Yaogo M, Kafando Y, Kadidiatou K, Ouedraogo M, Bicaba A, Haddad S: Targeting the worst-off for free health care: a process evaluation in Burkina Faso. Eval Program Plann. 2011, 34 (4): 333-342. 10.1016/j.evalprogplan.2011.03.007.
Souares A, Savadogo G, Dong H, Parmar D, Sie A, Sauerborn R: Using community wealth ranking to identify the poor for subsidies: a case study of community-based health insurance in Nouna. Burkina Faso Health Soc Care Community. 2010, 18 (4): 363-368.
Ridde V, Rossier C, Soura A, Bazié F, Kadio K: A community-based approach to indigent selection is difficult to organize in a formal neighbourhood in Ouagadougou, Burkina Faso: a mixed methods exploratory study. Int J Equity Health. 2014, 13 (31): 1-11.
Simporé L, Ridde V, Queuille L, Hema A: Évaluation de l'efficacité du ciblage communautaire des indigents bénéficiaires de la gratuité des soins dans les districts sanitaires de Dori et de Sebba au Burkina Faso. Santé maternelle et accès aux soins en Afrique de l'Ouest Contributions de jeunes chercheurs. 2013, Harmattan, Paris, 209-230.
Ridde V, Jacob JP: Les indigents et les politiques de santé en Afrique. Expériences et enjeux conceptuels. 2013, Academia-L'Harmattan, Louvain-La-Neuve
De Allegri M, Sauerborn R, Kouyate B, Flessa S: Community health insurance in sub-Saharan Africa: what operational difficulties hamper its successful development?. Trop Med Int Health. 2009, 14 (5): 1-11. 10.1111/j.1365-3156.2009.02262.x.
Fonteneau B: L'émergence de pratiques d'économie sociale en matiére de financement de la santé au Burkina Faso. 2000, Hoger Instituut voor de Arbeid, Leuven
Gnawali DP, Pokhrel S, Sié A, Sanon M, De Allegri M, Souares A, Dong H, Sauerborn R: The effect of community-based health insurance on the utilization of modern health care services: evidence from Burkina Faso. Health Policy. 2009, 90 (2-3): 214-222. 10.1016/j.healthpol.2008.09.015.
Jütting J: Do Community-based health insurance schemes improve poor people's access to health care? Evidence from rural Senegal. World Dev. 2004, 32 (2): 273-288. 10.1016/j.worlddev.2003.10.001.
Chankova S, Sulzbach S, Diop F: Impact of mutual health organizations: evidence from West Africa. Health Policy Plan. 2008, 23 (4): 264-276. 10.1093/heapol/czn011.
Spaan E, Mathijssen J, Tromp N, McBain F, Ten Have A, Baltussen R: The impact of health insurance in Africa and Asia: a systematic review. Bull World Health Org. 2012, 90 (9): 633-712. 10.2471/BLT.12.102301.
Parmar D, Reinhold S, Souares A, Savadogo G, Sauerborn R: Does community-based health insurance protect household assets? Evidence from rural Africa. Health Serv Res. 2012, 47 (2): 819-839. 10.1111/j.1475-6773.2011.01321.x.
Lu C, Chin B, Lewandowski J, Basinga P, Hirschhorn LR, Hill K, Murray M, Binagwaho A: Towards universal health coverage: an evaluation of Rwanda Mutuelles in its first eight years. PLoS One. 2012, 7 (6): e39282-10.1371/journal.pone.0039282. doi:10.1371/journal.pone.0039282,
Haddad S, Ridde V, Yacoubou I, Mak G, Gbetié M: An evaluation of the outcomes of mutual health organizations in Benin. PLoS ONE. 2012, 7: 10 e47136-doi:10.1371/journal.pone.0047136,
De Allegri M, Kouyaté B, Becher H, Gbangou A, Pokhrel S, Sanon M, Sauerborn R: Understanding enrolment in community health insurance in sub-Saharan Africa: a population-based case-control study in rural Burkina Faso. Bull World Health Organ. 2006, 84 (1): 852-858.
Jehu-Appiah C, Aryeetey G, Spaan E, de Hoop T, Agyepong I, Baltussen R: Equity aspects of the National Health Insurance Scheme in Ghana: who is enrolling, who is not and why?. Soc Sci Med. 2011, 72 (2): 157-165. 10.1016/j.socscimed.2010.10.025.
Onwujekwe O, Onoka C, Uzochukwu B, Okoli C, Obikeze E, Eze S: Is community-based health insurance an equitable strategy for paying for healthcare? Experiences from southeast Nigeria. Health Policy. 2009, 92 (1): 96-102. 10.1016/j.healthpol.2009.02.007.
Parmar D, De Allegri M, Savadogo G, Sauerborn R: Do community-based health insurance schemes fulfil the promise of equity? A study from Burkina Faso.Health Policy Plan 2013. 2013, Jan 9 [Epub ahead of print].,
Mills A, Ataguba JE, Akazili J, Borghi J, Garshong B, Makawia S, Mtei G, Harris B, Macha J, Meheus F, McIntyre D: Equity in financing and use of health care in Ghana, South Africa, and Tanzania: implications for paths to universal coverage. Lancet. 2012, 380 (9837): 126-133. 10.1016/S0140-6736(12)60357-2.
Agyepong I, Nagai RA: "We charge them; otherwise we cannot run the hospital" front line workers, clients and health financing policy implementation gaps in Ghana. Health Policy. 2011, 99: 226-233. 10.1016/j.healthpol.2010.09.018.
Schneider P: Trust in micro-health insurance: an exploratory study in Rwanda. Soc Sci Med. 2005, 61 (7): 1430-1438. 10.1016/j.socscimed.2004.11.074.
Barimah K, Mensah J: Ghana's National Health Insurance Scheme: insights from members, administrators and health care providers. J Health Care Poor Underserved. 2013, 24 (3): 1378-1390. 10.1353/hpu.2013.0144.
Ridde V, Haddad S, Yacoubou M, Yacoubou I: Exploratory study of the impacts of mutual health organizations on social dynamics in Benin. Soc Sci Med. 2010, 71 (3): 467-474. 10.1016/j.socscimed.2010.03.047.
Robyn P, Bärnighausen T, Souares A, Savadogo G, Bicaba B, Sié A, Sauerborn R: Does enrollment status in community-based insurance lead to poorer quality of care? Evidence from Burkina Faso. Int J Equity Health. 2013, 12: 31-10.1186/1475-9276-12-31.
Ettenger A, Bärnighausen T, Castro A: Health insurance for the poor decreases access to HIV testing in antenatal care: evidence of an unintended effect of health insurance reform in Colombia. Health Policy Planning. 2013, 29 (3): 352-358. 10.1093/heapol/czt021.
Turcotte-Tremblay A-M, Haddad S, Yacoubou I, Fournier P: Mapping of initiatives to increase membership in mutual health organizations in Benin. Int J Equity Health. 2012, 11 (1): 74-10.1186/1475-9276-11-74.
Johri M, Ridde V, Heinmueller R, Haddad S: User-fee elimination reduces maternal and child mortality: 1 year ex post impact evaluation and modelling study in Burkina Faso. Bull World Health Organ. 2014, 92: 706-715. 10.2471/BLT.13.130609.
Atlas of African Health Statistics 2014. Health Situation Analysis of the African Region. 2014
Enquête démographique et de santé et à indicateurs multiples (EDSBF-MICS IV), Rapport préliminaire, Burkina Faso, 2010. 2011, NSD, Ouagadougou
Haddad S, Nougtara A, Ridde V: Les inégalités d'accès aux services de santé et leurs déterminants au Burkina Faso. Santé, Société et Solidarité. 2004, 2: 199-210. 10.3406/oss.2004.1012.
Bicaba A: Évaluation finale de la phase test du financement basé sur les résultats dans les districts sanitaires de Boulsa, Leo et Titao. Rapport Final. 2013, SERSAP, DEP/MS, Ouagadougou
Ridde V, Haddad S: Pragmatisme et réalisme pour l'évaluation des interventions de santé publique. Rev Epidemiol Sante Publique. 2013, 61 (Supplement 2): S95-S106. 10.1016/j.respe.2013.03.037.
Maclure M: Explaining pragmatic trials to pragmatic policymakers. J Clin Epidemiol. 2009, 62: 476-478. 10.1016/j.jclinepi.2008.06.021.
Patsopoulos NA: A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011, 13: 217-224.
Pawson R: The Science of Evaluation: a Realist Manifesto, Volume 216. 2013, Sage Publications, Thousand Oaks, CA
Blaise P, Marchal B, Lefèvre P, Kegels G: Au-delà des méthodes expérimentales, l'approche réaliste en évaluation. Réduire les inégalités sociales de santé. Edited by: Potvin L, Moquet MJ, Jones C. 2010, NPES coll Santé en action, Saint-Denis, 285-296.
Ridde V, Dagenais C: Approches et pratiques en évaluation de programme. Deuxiéme éditon. 2012, Presses de l'Université de Montréal, Montréal
Patton MQ, LaBosSiére F: L'évaluation axée sur l'utilisation. Approches et pratiques de l'évaluation de programme. Edited by: Ridde V, Dagenais C, Ridde V, Dagenais C. 2012, Presses de l'Université de Montréal, Montréal, 145-160. 2
Beaudry J, Gauthier B: L'èvaluation de programme. Recherche sociale: de la problèmatique à la collecte de données. 2éme ed. 1992, Presses de l'Université du Québéc, (QC), Canada, 425-452.
Wholey JS: Evaluability Assessment. 2004, Jossey-Bass, San Francisco
Bamberger M, Rugh J, Mabry L: RealWorld Evaluation. Working Under Budget, Time, Data, and Political Constraints Thousand Oaks. 2006, Sage Publications, CA
Ridde V: Policy implementation in an African State: an extension of the Kingdon's multiple-streams approach. Public Adm. 2009, 87 (4): 938-954. 10.1111/j.1467-9299.2009.01792.x.
Dobson D, Cook TD: Avoiding type III error in program evaluation: results from a field experiment. Eval Program Plann. 1980, 3 (4): 269-276. 10.1016/0149-7189(80)90042-7.
Ridde V, Druetz T, Poppy S, Kouanda S, Haddad S: Implementation fidelity of the national malaria control program in Burkina Faso. PLoS One. 2013, 8 (7): e69865-10.1371/journal.pone.0069865.
Perez D, Lefevre P, Castro M, Sánchez L, Toledo ME, Vanlerberghe V, Van der Stuyft P: Process-oriented fidelity research assists in evaluation, adjustment and scaling-up of community-based interventions. Health Policy Plan. 2011, 26: 413-422. 10.1093/heapol/czq077.
Hasson H, Blomberg S, Dunér A: Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implementation Sci. 2012, 7: 23-10.1186/1748-5908-7-23.
Patton MQ: Utilization-Focused Evaluation. 2008, Sage Publications, Thousand Oaks-London-New Delhi
Perez D, Lefevre P, Castro M, Toledo ME, Zamora G, Bonet M, Van der Stuyft P: Diffusion of community empowerment strategies for Aedes aegypti control in Cuba: a muddling through experience. Soc Sci Med. 2013, 84: 44-52. 10.1016/j.socscimed.2013.02.003.
Samb O, Ridde V, Queuille L: Quelle pérennité pour les interventions pilotes de gratuité des soins au Burkina Faso. Revue Tiers Monde. 2013, 215 (uillet-septembre): 73-91. 10.3917/rtm.215.0073.
Ridde V, Pluye P, Queuille L: Evaluer la pérennité des programmes de santé publique: un outil et son application en Haïti. Rev Epidemiol Sante Publique. 2006, 54 (5): 421-431. 10.1016/S0398-7620(06)76740-2.
Schell S, Luke D, Schooley M, Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, Bunger AC: Public health program capacity for sustainability: a new framework. Implementation Sci. 2013, 8 (1): 15-10.1186/1748-5908-8-15.
Simmons R, Fajans P, Ghiron L: Scaling up health service delivery: from pilot innovations to policies and programmes. 2007, World Health Organization, Geneva
Bennett S, Agyepong I, Sheikh K, Hanson K, Ssengooba F, Gilson L: Building the field of health policy and systems research: an agenda for action. PLoS Med. 2011, 8 (8): e1001081-10.1371/journal.pmed.1001081.
Peters DH, Tran NT, Adam T: Implementation Research in Health: a Practical Guide. 2013, World Health Organization. Alliance for Health Policy and Systems Research, Geneva
Belaid L, Ridde V: An implementation evaluation of a policy aiming to improve financial access to maternal health care in Djibo district, Burkina Faso. BMC Pregnancy Childbirth. 2012, 12 (143): 1-12.
Hawe P, Shiell A, Riley T: Theorising interventions as events in systems. Am J Community Psychol. 2009, 43 (3-4): 267-276. 10.1007/s10464-009-9229-9.
Haddad S, Bicaba A, Feletto M, Taminy E, Kabore M, Ouédraogo B, Contreras G, Larocque R, Fournier P: System-level determinants of immunization coverage disparities among health districts in Burkina Faso: a multiple case study. BMC Int Health Hum Rights. 2009, 9 (Suppl 1): S15-10.1186/1472-698X-9-S1-S15.
Meda C, Konaté L, Ouedraogo HMS, Hercot D, Sombie I: Leadership et vision exercée pour la couverture universelle des soins dans les pays à faible revenu. Cahier Santé. 2011, 21 (3): 178-184.
Yin RK: Case Study Research: Design and Methods. 2009, Sage Publications, Los Angeles, Calif
Creswell JW, Plano Clark VL: Designing and Conducting Mixed Methods Research. 2007, SAGE Publications, Thousand Oaks, Calif
Yin RK: Validity and generalization in future case study evaluations. Evaluation. 2013, 19 (3): 321-332. 10.1177/1356389013497081.
Paillé P, Mucchielli A: L'analyse qualitative en sciences humaines et sociales. 2003, Colin, Paris
Emerson RM, Fretz RI, Shaw LL: Writing Ethnographic Fieldnotes. 1995, University of Chicago Press, Chicago-London
Olivier de Sardan J-P: La rigueur du qualitatif. Les contraintes empiriques de l'interprétation socio-anthropologique. 2008, Academia Bruylant, Louvain-la-Neuve
Langley A, Smallman C, Tsoukas H, Van de Ven AH: Process studies of change in organization and management: unveiling temporality, activity, and flow. Acad Manage J. 2013, 56: 1-13. 10.5465/amj.2013.4001.
The research project is part of the "Community research studies and interventions for health equity in Burkina Faso". We thank the Canadian Institutes of Health Research (CIHR), who funded the program (Grant number ROH-115213). VR holds a CIHR-funded Research Chair in Applied Public Health. AMTT obtained a bursary from the University of Montreal Hospital Research Center and the University of Montreal Public Health Research Institute. MH holds a research scholar award (junior 1) from the Fonds de recherche du Québec - santé. The authors wish to express their gratitude to the representatives of the World Bank and the Ministry of Health's Technical Support Unit in charge of PBF for their collaboration and participation during the discussion workshops.
AS, JL, JLK, and HH received a salary from a grant funded by the World Bank to conduct the baseline of the impact evaluation. The other authors declare that they have no competing interests.
VR and AMTT wrote the original draft with support from AS, JL, and MDA. All authors revised the manuscript as well as read and approved the final version.
About this article
- Performance-based financing
- Community-based health insurance
- User fee exemption
- Burkina Faso
- Implementation study
- Process evaluation
- Case study methodology
- Research protocol