Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Protocol for the evaluation of a social franchising model to improve maternal health in Uttar Pradesh, India

  • Shreya K. Pereira1,
  • Paresh Kumar2,
  • Varun Dutt2,
  • Kaveri Haldar2,
  • Loveday Penn-Kekana3,
  • Andreia Santos1 and
  • Timothy Powell-Jackson1Email author
Implementation Science201510:77

https://doi.org/10.1186/s13012-015-0269-2

Received: 20 April 2015

Accepted: 16 May 2015

Published: 26 May 2015

Abstract

Background

Social franchising is the fastest growing market-based approach to organising and improving the quality of care in the private sector of low- and middle-income countries, but there is limited evidence on its impact and cost-effectiveness. The “Sky” social franchise model was introduced in the Indian state of Uttar Pradesh in late 2013.

Methods/design

Difference-in-difference methods will be used to estimate the impact of the social franchise programme on the quality and coverage of health services along the continuum of care for reproductive, maternal and newborn health. Comparison clusters will be selected to be as similar as possible to intervention clusters using nearest neighbour matching methods. Two rounds of data will be collected from a household survey of 3600 women with a birth in the last 2 years and a survey of 450 health providers in the same localities. To capture the full range of effects, 59 study outcomes have been specified and then grouped into conceptually similar domains. Methods to account for multiple inferences will be used based on the pre-specified grouping of outcomes. A process evaluation will seek to understand the scale of the social franchise network, the extent to which various components of the programme are implemented and how impacts are achieved. An economic evaluation will measure the costs of setting up, maintaining and running the social franchise as well as the cost-effectiveness and financial sustainability of the programme.

Discussion

There is a dearth of evidence demonstrating whether market-based approaches such as social franchising can improve care in the private sector. This evaluation will provide rigorous evidence on whether an innovative model of social franchising can contribute to better population health in a low-income setting.

Keywords

Social franchisingImpact evaluationIndiaStudy protocol

Background

Over the past few decades, India’s maternal mortality ratio has declined substantially from 437 deaths per 100,000 live births in 1992–1993 to 178 deaths per 100,000 live births in 2010–2012 [1, 2]. Despite these improvements, the current state of maternal and child health in India requires urgent attention. India remains the largest contributor to the global burden of maternal deaths, accounting for nearly a quarter of all maternal deaths worldwide [3]. One of the most high profile responses of the Government of India has been to encourage facility births by providing cash incentives to women through the Janani Suraksha Yojana (JSY) scheme. Studies show that the programme has been effective in increasing utilisation of government maternal health services even if the evidence on mortality is contested [4, 5]. However, there are concerns about the public sector and its capacity to meet the increased demand for institutional deliveries. Whether the resources of the private sector should be harnessed to improve maternal health, and other aspects of health, is at the forefront of ongoing debates [6].

India’s private health sector is extensive and incredibly diverse. It ranges from sophisticated tertiary hospitals providing medical care of an international standard to unqualified rural health providers and alternative systems of medicine. The majority of registered doctors work in the private sector, and it is often the first point of contact for a substantial proportion of the population [79]. In Uttar Pradesh, the setting of this study, 31 % of all facility births are in the private sector [10]. Although no worse than the public sector, studies of the private sector in India document poor quality of primary health care and potentially harmful practices [11].

Despite widespread consensus on the growing presence and role of the private sector in low- and middle-income countries, there is limited evidence on the most effective strategies to improve the quality of services [12, 13]. Regulating the quality of the private sector given its size and diversity has proved enormously challenging for the government, and alternative strategies to raise standards must be sought. Innovative approaches currently being used to tackle and institutionalise quality improvement include accreditation [14], contracting out clinical services [15], vouchers [16], and social franchising [17].

Social franchising is the fastest growing market-based approach to organising and improving the quality of care in the private sector in low- and middle-income countries [18]. In 2013, 83 franchises were largely operating in Sub-Saharan Africa and Asia reaching nearly 20 million patients [19]. Social franchises are networks of private providers, operating under contracts with a common agency and providing standardised products and services under a single brand. Social franchises typically have five programmatic goals: quality, health impact, equity, cost-effectiveness and market expansion. Franchise models that link with and encourage referrals between the public and private sector may help reduce health market fragmentation and improve quality of care. Common franchise elements include demand- and supply-side components relating to contract design, training, supervision, branding and advertising.

This study protocol describes the methods to be used in an evaluation of the Sky social franchising model in Uttar Pradesh. The aim of the social franchise model is to increase access to and use of basic obstetric care, emergency obstetric care and family planning services. The evaluation will draw on quantitative and qualitative methods to address three study objectives: (1) to estimate the impact of the social franchising model on the quality and coverage of health services along the continuum of care for reproductive, maternal and newborn health; (2) to understand the scale of the social franchise network, the extent to which various components of the programme are implemented and how impacts are achieved; and (3) to establish the cost-effectiveness and financial sustainability of the programme.

Evidence on social franchising

Our review of the evidence draws on three recent systematic reviews of social franchising in health [17, 18, 20]. The majority of the social franchise programmes focus on reproductive services and family planning products, which together account for a large proportion of the literature on the topic [18, 20]. Before examining the empirical evidence, it is important to note that the methodological rigour of studies on social franchising in low- and middle-income countries is poor. This is demonstrated by the fact that the most thorough review of social franchising, published in the Cochrane library, found no studies eligible for inclusion despite the fact that inclusion criteria were broad enough to permit a range of quasi-experimental methods [17]. None of the reviews uncovered any evidence on the health impact or cost-effectiveness of social franchising.

In a second review, studies of clinical social franchise programmes were included if they provided data on at least one outcome related to quality, health impact, equity, cost-effectiveness and market expansion [18]. Quasi-experimental and qualitative studies were not excluded. The authors included 23 studies whose overall quality was regarded as low. The review found limited and mixed evidence on impact. Social franchising was found to increase client volume and service utilisation, but there was no evidence on the ability of social franchising to expand the availability of health services in currently underserved areas. Over half of the studies measured some aspect of quality but always in relation to family planning services and rarely in a comprehensive manner. A study in Pakistan and Ethiopia found that franchises were of equivalent or lower quality than public clinics but higher quality than non-franchised private providers. In Nepal, both franchised and non-franchised clinics showed similarly poor facility quality.

A third review of social franchising included 15 studies that examined the relationship between franchising and outcomes [20]. Around half focused on quality and utilisation, and a few considered results for providers, client loyalty, client volumes and efficiency. Reproductive health/family planning services research were well represented; other sectors investigated were pharmacy and tuberculosis care. The authors found that franchising is predominantly positively associated with client volumes, physical accessibility and some types of quality, but findings regarding utilisation, customer loyalty and efficiency were mixed. The methodological quality of studies was found to be poor.

In summary, the current scientific evidence and body of knowledge on the impact of social franchising, or on the sustainability of social franchising as a long-term alternative to the public sector, suggest that generalisations about the value of franchising are difficult to make. There is some evidence on the ability of clinical social franchising to increase patient volume and some aspects of quality of care. However, in general, the quality of evidence is sufficiently poor and variation in the types of social franchising models tested so wide that no firm conclusions can be drawn about whether and how social franchises affect service delivery.

Social franchise model

The Sky franchise network includes providers at various levels. SkyCare is the lowest level of the network and consists of informal rural health providers who are typically medically unqualified. SkyCare providers pay a franchise joining fee to the franchisor and receive signage, posters, training manuals and the ability to phone into a central medical facility. The franchisor maintains the central medical facility by employing qualified doctors to conduct remote medical consultations. SkyCare franchisees are given financial incentives by the network to make antenatal care referrals to SkyHealth franchisees. The second level of the network is SkyHealth. These providers are typically qualified traditional medical practitioners trained to provide Ayurvedic, Yoga & Naturopathy, Unani, Siddha and Homoeopathic care (AYUSH). SkyHealth offers telemedicine services and receive financial incentives for completing three antenatal care consultations with a client. At the highest level, nine franchised clinics and three franchisor-owned clinics are staffed by physicians to provide safe delivery and emergency obstetric care.

The programme provides clinical training to private and public sector health providers. The franchisor trains SkyCare in how to conduct mobile phone consultations. SkyHealth is trained to provide antenatal care, recognise and stabilise pregnancy complications, facilitate timely referrals and provide family planning methods and postpartum contraception counselling. Providers from franchised clinics receive training on national and international guidelines to provide emergency obstetric care, general family planning and postpartum intrauterine devices. Public sector providers with the remit of dealing with emergency obstetric cases also receive clinical training in order to manage linkages and referrals from the private sector. Training in the public sector extends to Accredited Social Health Activist (ASHA) working at the community level. An additional file details the training programme by type of provider [see Additional file 1].

The social franchise model takes a total market approach in the sense that it seeks to develop closer links to, and strengthening of, the public sector. The major components of the programme are summarised in Table 1, using an adaptation of the Centre for Health Market Innovations framework for characterising health programmes [21]. Components fall into five major approaches: organising delivery, regulating performance, financing care, changing behaviours and enhancing processes. These approaches include both demand- and supply-side activities to influence health care-seeking behaviours and the quality of healthcare provision. Demand-side activities include incentivising rural health providers for referrals, brand creation, price subsidies for clients below the poverty line and advertising of franchise services. A major demand-side activity of the programme is social marketing, in which the franchisor distributes their own branded medicines—SkyMeds—via a network of shops, pharmacies and franchisees. SkyCare and SkyHealth providers are encouraged to sell SkyMeds, though it is optional. Supply-side activities include clinical training for providers, telemedicine and mobile technologies and the introduction of innovations such as the non-pneumatic anti-shock garment to stabilise women with heavy bleeding in both private and government facilities.
Table 1

Components of the social franchising programme

Organising delivery

Programmes that reduce fragmentation and informality of health care delivery and that may enable financing, regulation, training and new business models

Franchise

A group of providers that operates under the same brand but where outlets are operator-owned and services are standardised by a central franchisor

SkyCare/SkyHealth (stand-alone franchises); franchise clinic/franchise diagnostic (fractional franchises)

Chain

A group of providers that operates under the same brand but where operators are paid employees of a sponsoring organisation

Three franchisor-owned clinics (also called mini-clinics)

Network

A group of providers that are loosely joined to deliver services to specific population groups. Each provider is a separate entity and retains its own branding. Membership in the network may entitle the provider to payments, patient volume, central services or training

Franchisees are linked to a network of shops selling drugs which receive socially marketed products

Regulating performance

Programmes that set standards and enforce or incentivize higher quality care or increased access for target populations

Quality enforcement/monitoring

Programmes that mandate specific clinical practice guidelines, and/or monitor providers over time to ensure quality

Monitoring and supervision of quality standards in franchisees, exit surveys and encourage feedback from competitors

Price regulation

Programmes or regulations that specify prices that must be charged to users for services

Fixed prices for below the poverty line clients at Sky Centres; fixed prices for franchised services at franchised clinics

Financing care

Programmes that mobilise funds for health care and align provider incentives to increase access for targeted groups of patients or to support select health interventions

Links to government health financing mechanisms

Initiatives that link private providers to existing government health financing mechanisms that can contract and reimburse private providers for care provided to specified patient groups

Plan to facilitate linking franchisees and beneficiaries to government cash incentive and insurance schemes. Training of community health workers to link with government schemes

Cross subsidisation

Programmes that charge full-fees for services to patients that are able to afford them and use the profits to subsidise services for the poor

Subsidies for telemedicine for clients below the poverty line off-set to some degree by franchise fee paid per client above the poverty line

Changing behaviours

Programmes designed to change the behaviour of individuals involved in health care transactions

Social marketing

Programmes that aim to change consumer care-seeking behaviours through marketing/advertisement techniques, with or without a branded and/or subsidised product

Branding, advertising, SMS messages, provision of SkyMeds

Community health workers

Programmes that use community health workers to generate demand for products or services

Government community health workers refer women to public and franchised facilities

Provider training

Programmes that seek to improve the quality and/or efficiency of services by training health care workers and/or building the internal capacity of organisations

Training of SkyCentre staff, franchise clinic staff, community health workers and public sector staff. Sky centre staff also trained on telemedicine technology

Other health awareness/education

Programmes that create social awareness and educate the public about specific health topics such as disease prevention and treatment, healthy behaviours, correct use of pharmaceuticals, etc.

Community system to give health messages

Organising delivery

Programmes that reduce fragmentation and informality of health care delivery and that may enable financing, regulation, training and new business models

Enhancing processes

Processes, technologies, or products that facilitate increased efficiency, lower costs, higher quality, and/or improved access

Information and communications technology

Programmes that utilise technology to enable remotely delivered care, communication and exchange of medical information (e.g. telemedicine, call centre, cell phone technology, biometric system, etc.).

Cell phone/smartphone/tablet/telemedicine services through franchisees, including remote diagnostics

Innovative operational processes

Programmes that improve quality, reduce costs or enhance efficiency of services through new business or care processes (e.g. high-volume/low-cost operational models, process standardization).

Telemedicine; getting auxiliary nurse midwives to insert intrauterine devices in rural areas

Mobile health

Programmes that utilise various models of transportation to deliver services to rural and remote populations. (e.g. ambulance services, health worker transport, travelling clinics/products, etc.)

May have Sky ambulance and link to “108” ambulance

Supply chain enhancements

Programmes that reduce costs and improve efficiency of supply chains that move medical products from manufacturer to retailer

Last mile outriders (SkyMeds and diagnostics)

Innovative medical products and equipment

Programmes that design, manufacture and sell new products such as rapid testing kits, nutritional supplements or other medical supplies that reduce costs, improve quality or enable remote care

Non-pneumatic anti-shock garment; stabilisation procedures at lower levels; remote diagnostics; safe delivery kits

Theory of change

A theory of change for the social franchising programme was developed in collaboration with the implementing franchisor in December 2013. The results chain of the Sky social franchise model was used for this purpose and is detailed in an additional file [see Additional file 2]. It shows the sequence of inputs, activities and outputs that are expected to improve outcomes. The results chain is clearly a naïve simplification of reality, but it nevertheless provides a useful framework for understanding how the programme is intended to work as originally designed.

We highlight here the most important pathways that are critical to generating the intended impacts. First, health providers are willing to join the network and expand the range of services they offer to include reproductive and maternal health services. Second, the branding of social franchisees provides a signal of quality that increases patient demand for their services. Third, the monitoring of standards and the prospect of better business performance encourage health providers to improve their quality of care. Fourth, training and the provision of IT such as telemedicine increase skills and knowledge, ultimately leading to better quality of care and appropriate referrals.

Several key assumptions underpin the programme’s success. All else equal, basic economic theory suggests that increasing the quality of a service will raise consumer demand. In healthcare, however, information problems are pervasive and patients may not in fact be able to evaluate the quality of care they received, at least in terms of aspects of care that matter for health [2224]. The extrinsic incentives to improve quality may therefore not be strong. Another important assumption is that lack of provider knowledge and skills are binding constraints to delivering quality healthcare. The evidence here is somewhat mixed. A meta-analysis of studies in low- and middle-income countries shows that training has a modest effect on provider practice [25]. In India, the qualifications of the provider matter for quality but not as much as expected [26].

Methods

Study setting

Uttar Pradesh is India’s fourth largest and most populous state with approximately 199.8 million people living in 18 divisions and 75 districts. If Uttar Pradesh were a country, it would be the fifth largest in the world in terms of population. The three districts in which the social franchising network is located have a population of 8.1 million and vary considerably in terms of demographic and health indicators (see Additional file 3). Kanpur Nagar is predominant urban, with higher literacy and lower mortality than the state average. By contrast, Kannauj and Kanpur Dehat are more typical of the state as a whole. Largely rural, they have poor literacy and high rates of maternal and child mortality that are comparable with the less developed countries in the world.

Across the continuum of care, large discrepancies in maternal and child health indicators are observed between the three districts [see Additional file 3]. For example, coverage of at least three visits of antenatal care in Kanpur Nagar is the highest at 51 % compared to 15 and 32 % in Kannauj and Kanpur Dehat, respectively. Despite government schemes to improve rates of institutional births, 54 % of deliveries occur at home in Uttar Pradesh (57 % in Kannauj, 40 % in Kanpur Nagar and 52 % in Kanpur Dehat). Of the home deliveries, 11 %, 53 % and 29 % were conducted by skilled health personnel in Kannauj, Kanpur Nagar and Kanpur Dehat, respectively.

Study design

The impact study is designed as a prospective controlled before and after study in which the comparison group comprises matched areas both within the intervention districts and in neighbouring districts where social franchising is not introduced. The overall design is shown in Fig. 1. The primary sampling unit for much of the data collection is a cluster, defined as a ward (urban) or a village (rural) according to the most recent census. The impact evaluation involves the selection of study clusters to form three arms. Group A contains clusters with a social franchisee in the three intervention districts. Group B comprises clusters with no social franchisee in the same three districts. Group C is taken from neighbouring districts that do not have any social franchise network operating within them.
Fig. 1

Study design and data collection

The selection of study areas was done 14 months after the first health providers were contracted at which time there were 50 SkyHealth and 343 SkyCare providers in the social franchise network. The selection of study clusters proceeds according to the following steps. First, we link every social franchisee to the census area in which it is located and select, at random, 60 intervention clusters (Group A). Second, we select similar controls by matching without replacement the intervention clusters to 60 comparison areas within the same three districts (Group B) [27]. As can be seen in Fig. 2, we impose a buffer zone of 0.5 km around intervention clusters to limit problems of contamination. We perform exact matching on district and urban residences and then within each strata, select pairs of clusters (nearest neighbour) with the smallest distance based on a Mahalanobis metric that is computed using census data on total population, % under 6 years, % females under 6 years, % literate females, % scheduled tribe, % scheduled caste, % cultivator and % “other” workers. Finally, we perform the same matching procedure to select 60 comparison areas in neighbouring districts (Group C).
Fig. 2

Map of study clusters in the three intervention districts

The selection of study clusters provides variation in the social franchise that facilitates identification of its impact. Variation over time is generated in two ways. The 2-year recall period of the household survey means that we have almost 12 months of baseline data even in areas where social franchising is introduced. Moreover, we anticipate that over time, some of the Group B study clusters will become intervention areas as the social franchise network expands generating further variation over time. Geographical variation in the placement of the social franchise is generated by our selection of comparison areas.

Data collection

The evaluation relies primarily on several tools that are administered over two rounds of data collection: a household survey of women who recently gave birth and a health provider survey. The first round of data collection was in January 2015, and the second round is planned for March 2016. This means the impact of the social franchise intervention is assessed approximately 2 years after its start. Information on the precise timing of the introduction of social franchising in each cluster is based on administrative data provided by the franchisor triangulated with responses to the health provider survey. It is worth noting that the research outlined in this protocol is closely coordinated with several other data collection activities—direct observations of births and a case study of three social franchise models that will provide additional insights.

The household survey is administered to women as a cross-section at two points in time and serves as the main source of data on our study outcomes. Eligible respondents include all women who gave birth in the previous 24 months (first round) or 18 months (second round), including those who had a stillbirth or whose child died since birth. Eligible women are identified through a census of households, conducted 1 month before the household survey. Every member of the household is listed and then, for women aged 15 to 49 years, a series of questions probe whether she gave birth to a baby that was born alive, born dead or lost before birth. Using this sampling frame, a maximum of 23 eligible women in each cluster are randomly selected for interview. The household survey tool includes the following modules: (1) household listing, (2) general healthcare interactions, (3) household characteristics, (4) wellbeing of husband, (5) pregnancy history, (6) family planning and antenatal care, (7) delivery and postnatal care, (8) child health, (9) interactions with community health workers, (10) information and perceptions of healthcare, and (11) wellbeing, mental health and physical health.

The health provider survey is administered at the same time as the household survey within the same communities. The sampling frame is generated from a census of all health providers within the study clusters conducted 1 month prior to the health provider survey. The census records the type of health provider and its geographic coordinates. For the purposes of the census, we define a health provider as any institution or individual whose primary purpose is to provide healthcare. We exclude drug sellers. Using this list, we randomly select for interview one private health provider (social franchisee in intervention clusters), one government health provider and one accredited social health activist in each cluster. The health provider survey tool includes the following modules: (1) health facility characteristics, (2) key health worker characteristics, (3) reproductive, maternal and newborn services, (4) maternal health knowledge, (5) maternal health practice, (6) motivation, (7) social franchise and business practices, and (8) experience of social franchise network.

Outcomes

The impact of the social franchise model is assessed using a comprehensive set of pre-specified outcomes that are measured using the household data. An extensive list of outcomes, 59 indicators in total, is shown in Tables 2, 3 and 4. They cover the continuum of care from antenatal care through postnatal family planning and include various types of indicators including healthcare utilisation, process of care, healthy behaviour, patient experience, patient information and financial strain. The study outcomes are organised according to conceptually similar groups that are required when implementing methods to deal with many outcomes.
Table 2

Antenatal outcomes of the impact evaluation by domain

Indicator

Type of indicator

1. ANC utilisation

 

Received at least three ANC visits (%)

Use of healthcare

Received ANC visit in first trimester (%)

Use of healthcare

Number of ANC consultations (visits)

Use of healthcare

Received visit from ASHA during pregnancy (%)

Use of healthcare

2. ANC content of care

 

Fully immunised with tetanus toxoid (%)

Process of care

Received iron supplementation during pregnancy (%)

Process of care

Took iron supplementation during pregnancy for at least 100 days (%)

Process of care

Received test results for syphilis received (%)

Process of care

Abdominal examination during ANC (%)

Process of care

Received a drug for intestinal worms during pregnancy (%)

Process of care

Received a drug to prevent malaria (%)

Process of care

Multiple birth pregnancy detected during ANC (%)

Process of care

ANC content of care score of six items (index 0 to 1)

Process of care

3. ANC knowledge and preparedness

 

Mother knowledge of pregnancy complications (index 0 to 1)

Patient knowledge

Mother knowledge of signs of delivery complications (index 0 to 1)

Patient knowledge

Birth preparedness (financial, transport, blood donor, attendant, safe delivery kit) (index 0 to 1)

Healthy behaviour

Table 3

Intrapartum care outcomes of the impact evaluation by domain

Indicator

Type of indicator

1. Delivery care utilisation

 

Gave birth in a health facility (%)

Use of healthcare

Gave birth with a doctor, nurse or midwife (%)

Use of healthcare

Had a caesarean section (%)

Use of healthcare

2. Recommended delivery care practices

 

Delivery attendant used gloves (%)

Process of care

Delivery attendant washed hands with soap (%)

Process of care

Woman had her BP measured (%)

Process of care

Mobility during labour (%)

Process of care

Oral fluids during labour (%)

Process of care

Heart rate of baby monitored with intermittent or continuous auscultation (%)

Process of care

Use of anti-shock garment (%)

Process of care

3. Harmful or ineffective delivery care practices

 

Shaving pubic hair (%)

Process of care

Enema given (%)

Process of care

Lithotomy position during labour (%)

Process of care

Intravenous fluids during labour (%)

Process of care

4. Delivery care practices frequently over used

 

Urinary catheter (%)

Process of care

Pain control by epidural analgesia (%)

Process of care

Oxytocin augmentation (%)

Process of care

Episiotomy (%)

Process of care

5. Disrespect and abuse

 

Support during labour (%)

Patient experience

Medical procedure performed without consent (%)

Patient experience

Shouted, scolded or humiliated by health worker (%)

Patient experience

Slapped, pinched or hit by health worker (%)

Patient experience

Gave birth with privacy (%)

Patient experience

Refused care for inability to pay (%)

Patient experience

Kept in facility for inability to pay (%)

Patient experience

Felt disrespected or abused during facility stay (%)

Patient experience

6. Economic consequences

 

Out-of-pocket spending on delivery care (NRS)

Financial strain

Borrowed money to pay for delivery care (%)

Financial strain

Household in debt to pay for delivery care (%)

Financial strain

Received JSY cash incentive (%)

Financial strain

Table 4

Postpartum and newborn outcomes of the impact evaluation by domain

Indicator

Type of indicator

1. Postpartum care

Received postpartum care within 48 h of birth (%)

Use of healthcare

Newborn received postnatal care within 48 h of birth (%)

Use of healthcare

2. Newborn content of care

Clean cord care (clean instrument to cut and tie the cord, and nothing put on cord) (%)

Process of care

Thermal care (immediate drying, wrapping, skin to skin and delayed bathing) (%)

Process of care

Baby weighed at birth (%)

Process of care

Baby registered and received certificate (%)

Process of care

3. Neonatal health

 

Neonatal mortality (per 1000 live births)

Health outcome

One-day mortality (per 1000 live births)

Health outcome

Birth weight (kg)

Health outcome

4. Breastfeeding

 

Immediate breastfeeding within 1 h of birth (%)

Healthy behaviour

Colostrum given to baby (%)

Healthy behaviour

Exclusive breastfeeding for 3 days (%)

Healthy behaviour

5. Family planning

 

Modern contraceptive use at 3 months postpartum (%)

Use of healthcare

In contrast to standard surveys on maternal and child health in India [28, 29], we seek to measure a range of intrapartum care practices that may be affected by the social franchise, given its focus on standards. We get at the issue of quality of care by collecting information on recommended delivery care practices, harmful or ineffective practices, frequently over-used practices and disrespect and abuse indicators [3033]. The study will also collect data on a large range of characteristics of the mother and her household. These data are used to control for potential confounding and increase efficiency of our estimates. They are also important in examining the equity impact of the project—to see which socioeconomic segments of the population benefits most from the social franchise.

Sample size calculations

Household sample size calculations are based on an endline comparison of two groups (intervention versus control), using the proportion of women giving birth in a health facility as the primary outcome. On the basis of an observed institutional delivery rate of 50 % at baseline [10] and an assumed coefficient of variation of 0.2, a sample size of 60 intervention and 60 control clusters with a total of 20 women in each cluster are estimated to provide 80 % power to detect an 8 percentage point increase in the rate of institutional deliveries in the intervention group compared with the control at 5 % level of significance. Assuming a coefficient of variation of 0.1 reduces the detectable difference to 6 percentage points. It is anticipated that the study will have power to detect smaller differences once the analysis controls for covariates and utilises data from two survey rounds.

Empirical analysis

The impact evaluation relies primarily on the household data. We use a difference-in-difference strategy to estimate impacts [34]. This involves a comparison of changes in the outcomes over time between the intervention and the comparison groups. The analysis exploits the longitudinal nature of the data generated by the recall period used in the two rounds of the household survey and information on the precise timing of the introduction of social franchising in each study area. Specifically, using individual level data, we regress each outcome on a dummy variable indicating whether social franchising has been introduced in the area at the time of birth, area fixed effects and quarter year fixed effects. Unadjusted estimates are reported as well as those that adjust for household characteristics. Controls for household characteristics include below the poverty line status, urban residence, religion, ethnicity, maternal education, parity, multiple birth and the recall period. We cluster the standard errors at the area level.

We test whether the social franchising model has an effect in two ways: the first analysis compares intervention areas with the two sets of comparison areas pooled together, and the second analysis compares the intervention areas with the comparison areas in adjoining districts without social franchising. The latter may arguably be less prone to selection bias since comparison areas in neighbouring districts are beyond the geographical reach of the project and may offer a more credible counterfactual. If the social franchising model is found to have an effect on any of the main outcomes, we conduct subgroup analyses with respect to below the poverty line status, maternal education and caste. Finally, to assess the so-called parallel trends assumption that underpins the difference-in-difference approach, we exploit the recall period in the household data to verify that trends in each of the outcomes are similar between the three study arms before the introduction of the social franchising model. Evidence of diverging pre-trends would be a cause for concern. Baseline outcomes and characteristics of women are also summarised for each study arm with continuous variables presented as mean (standard deviation) and categorical variables by frequencies (percentage). An additional file describes the empirical strategy of the impact evaluation in further detail [see Additional file 4].

The presence of multiple outcomes leads to the risk of arbitrarily selecting statistically significant outcomes where high values of test statistics arise by chance. Testing each hypothesis one at a time with a fixed significance increases the probability of a type-I error exponentially as the number of outcomes tested grows. We deal with multiple outcomes using several procedures that are implemented for conceptually similar groups of outcomes listed in Tables 2, 3 and 4 [35, 36]. First, we present standardised treatment effects by creating an index for multiple outcomes within each domain and testing for an effect on the index. Implicitly, this weighs each outcome the same within a domain. Second, we present family-wise p values adjusted to account for the multiple outcomes within a domain using the free step-down resampling method of Westfall and Young [37].

Process evaluation

The process evaluation is intended to complement the impact evaluation. Indeed, it will run in parallel and draw on some of the same data sources. We will develop and critically assess a logic model of the project, mapping the pathways and intended effects of each component. We will next describe how the social franchise model evolves and the extent to which various components of the project are implemented on the ground. The process evaluation will then examine the factors that influence private providers’ decision to join the social franchise network. Finally, it will seek to understand how, if at all, the project influenced household decisions about healthcare and the behaviour of health providers.

Quantitative process measures will be collected to understand the extent of implementation, fidelity and scale (Table 5). These will focus on a number of different dimensions of implementation that map closely onto the various components within the project: uptake of social franchising; training of the health providers; information, branding and advertising; interactions with health workers in the social franchise network; and monitoring and feedback on quality and standards.
Table 5

Process measures

Indicator

Survey tool

1. Uptake of social franchising

Proportion of private health providers who join network (%)

Health provider census

Proportion of providers who left network in past year (%)

Health provider survey

Proportion of providers purchasing and selling SkyMeds (%)

Health provider survey

2. Training

Proportion of social franchisees that have received clinical training (%)

Health provider survey

Proportion of social franchisees that have received training in use of technology (%)

Health provider survey

3. Information and marketing

Proportion of women who have ever heard of Sky social franchise network (%)

Household survey

Proportion of social franchisees that have been branded (%)

Health provider survey

4. Contacts with health workers

Proportion of women who had any contact with ASHAs during pregnancy (%)

Household survey

Proportion of individuals who have used telemedicine in past 6 months (%)

Household survey

5. Monitoring and feedback

Proportion of social franchisees that have received supervision visits past 6 months (%)

Health provider survey

Proportion of social franchisees that have received feedback on quality past 6 months (%)

Health provider survey

Qualitative data collection will focus on understanding the process of implementation and how the social franchise model leads to impact. A 6-month period of intensive data collection using ethnographic methods will provide insight into the impact of the social franchise model on the dynamics of health care provision and health seeking behaviour at the village level and provide an important understanding of the context in which the intervention operates [38, 39]. The ethnographic work will seek to understand the intervention as implemented on the ground, the factors influencing providers’ decisions to join the franchise, the influence of the intervention on provider behaviour and stakeholder perceptions of the various providers and components involved in the Sky social franchise. As the ethnographic research process progresses, new hypotheses and questions will develop as new insights occur with increasing familiarity with the context [40]. Participant observation will be carried out by experienced anthropologists who speak the local language. Researchers will keep detailed field notes of informal observations and everyday conversations. If an informant provides more detailed information or partakes in a long discussion, the field worker will ask to digitally record the interview. Field notes and audio recordings of discussions will be transcribed in the original language and then translated.

The sample of villages selected will be a convenience sample informed by the results of the first round of quantitative data collection and drawn from the 60 clusters with a social franchise provider. The ethnographic research will take place in three broad locations, covering the catchment area of SkyHealth providers and their nearby SkyCare providers. Field notes will be double coded by the ethnographers. Preliminary findings and reflections from the ethnographic research will be fed back to the research participants through community meetings in each of the localities as a form of validation.

The second level of qualitative process evaluation will take place through repeat in-depth interviews. A document review and in-depth interviews with franchisor staff will be used to understand the process of implementation as well as the development of the project. Two rounds of interviews will take place with 10 members of staff at various levels in the organisations. In-depth interviews with senior staff will include topics and questions that facilitate understanding of the decision-making behind the intervention and the key factors that shaped its design. Interviews with field staff will include questions about the experience of implementation, the process of engaging with providers and adaptations to the project over time. A context record, which documents information that may impact the implementation, the mechanisms of change and the outcomes under measurement, will be developed. This exercise will be undertaken every 6 months, using short interviews to gather information about any developments, events, setbacks and news that may have impacted implementation of the project.

Economic evaluation

Micro-costing methods are used to estimate the financial and economic costs of setting up, maintaining and running the social franchise. Micro-costing methods using a bottom-up approach that record resource utilisation at the individual service level are employed to assess the cost of services [41, 42]. Three levels of costs are assessed: (i) costs incurred by the franchisor to plan, initiate and run the social franchise; (ii) costs of activities supporting the social franchise; and (iii) costs to the franchisees of participating in the network and providing the franchise services they offer. Data are obtained through administrative records, interviews with the franchisor, interviews with franchisees and informal observations. Costs are then classified according to: (i) start-up, defined as the initial costs related to the set-up of the social franchise; (ii) capital, defined as the costs of inputs that last for more than 1 year, to be annualised using standard methods [43]; and (iii) recurrent, defined as the costs of inputs that are incurred on a regular basis.

Effectiveness data is drawn from the impact evaluation. Since the study is not powered to measure the impact on mortality, it is necessary to model from the multiple study outcomes to the final health outcomes of deaths and disability adjusted life years (DALY) averted. This is based on a decision-tree model to be developed in light of a review of existing modelling tools such as the Lives Saved Tool (LiST) and Impact 2 [44, 45]. Cost-effectiveness ratios are presented as the cost per death averted and cost per DALY, comparing the situation with and without the social franchising programme. A number of commonly used thresholds are used for assessing whether the results can be considered “cost-effective” [46, 47]. A probabilistic sensitivity analysis using Monte Carlo simulation is conducted to test the effect of uncertainties across model parameters [48].

Research ethics and data management

The evaluation study has been approved by the Public Healthcare Society (PHS) Ethics Review Board in India and the London School of Hygiene & Tropical Medicine in the UK. The study design has also received government clearance from the National Health Mission in the State of Uttar Pradesh.

Informed consent is obtained before administering all surveys. Information sheets are read and given to respondents and written or verbal consent is sought prior to interview. The research activities are unlikely to cause any harm since they involve no invasive procedures or examinations. In this respect, it is important to note that the research team are external; they have no responsibility for the implementation of the project or for the services delivered by health providers in the network. The research activities involving data collection through the household survey and health provider survey are not anticipated to cause any harm. There will be no direct benefit to the study participants. The main cost will be the time given by the interviewees. Some of the interviews with households will involve women whose baby may have recently died. The field workers will be trained to deal with such cases sensitively. The household survey includes a mental health screening questionnaire, known as the K10. In some instances, women with severe depression or mental health disorders may be identified through the use of this tool. In such circumstances, researchers are trained to facilitate referral of the individual to an appropriate source of care that is closest to where the woman lives.

Data from the household and health provider surveys are collected through computer-assisted personal interviews. To the extent possible, privacy is maintained during interviews with participants. The study makes every effort to minimise the risk of breaches of confidentiality, particularly in relation to data management and the linking of datasets. The research intends to link data from different sources at the cluster level. This can only be done by the principal investigators at the time of analysis. The quantitative data are to be made publicly available at the end of the project through an established data repository. The data will not contain any global positioning system (GPS) information, identifiers or names that would allow identification of an individual or cluster. In-depth interviews will be conducted in private to maintain confidentiality. Qualitative data are to be securely kept. Audio files are downloaded onto a password protected computer. Data recorded on paper and audio files will be destroyed after the data are analysed and results are reported. In the reporting of the qualitative data, quotations are anonymised such that it is not possible to identify the individual.

Discussion

The social franchising model in Uttar Pradesh seeks to increase access to and use of basic obstetric care, emergency obstetric care and family planning services. The approach is novel in its focus on maternal health services, its effort to engage with low level and, in some cases, informal healthcare providers and its use of technology such as telemedicine. Whether such an approach to social franchising represents the way forward to improving utilisation and quality of maternal health services is unclear. The evaluation of this model will thus provide an important contribution to the existing literature.

The study’s contribution to knowledge will be strengthened by certain attributes of the study design. The matched difference-in-difference approach provides better causal inference than that obtained in previous studies of social franchising which have rarely used control groups. The large number of outcomes gives us the opportunity to capture the full range of effects across the continuum of care generated by this multifaceted health system intervention. By accounting for multiple inferences, we deal with the risk of specifying many outcomes. Finally, we complement the impact evaluation with a cost-effectiveness analysis and an examination of the implementation process to understand how the programme worked or failed to work and its potential for scaleup.

The findings will need to be interpreted with several potential limitations in mind. First, we must rely on women to recall delivery care practices during childbirth to come to conclusions as to the impact of the social franchising model on quality of care. Measures of quality that use standardised patients are the gold standard but cannot be used in the case of childbirth [26, 49]. Second, we anticipate issues to do with recall. Some indicators place a heavy burden on the woman’s ability to recollect events up to 2 years ago and are therefore likely to suffer from recall problems. In our adjusted estimates of impact, we control for the recall period and the issue is only a concern for the evaluation insofar as recall bias differs between study arms. Third, we note the potential for bias in our impact estimates given that the offer to join the social franchise network is not randomised. To attempt to limit selection problems, we use matched control areas to provide a credible a counterfactual as possible.

There is a dearth of evidence demonstrating whether market-based approaches such as social franchising can improve care in the private sector. Yet expansion of social franchising models in developing countries has been rapid. There is therefore a critical need for robust evaluations of different social franchising models in a wide range of settings to understand whether they contribute to better population health.

Abbreviations

ASHA: 

Accredited Social Health Activist

AYUSH: 

Ayurveda, Yoga & Naturopathy, Unani, Siddha, and Homoeopathy

DALY: 

Disability adjusted life years

GPS: 

global positioning system

JSY: 

Janani Suraksha Yojana

LiST: 

Lives Saved Tool

PHS: 

Public Healthcare Society

Declarations

Acknowledgements

The research in this publication was supported by funding from Merck Sharp & Dohme Corp. (“MSD”), a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA, through its MSD for Mothers programme. Funding was used for general financial support, including staff salaries, travel and overhead. MSD had no role in the design, collection, analysis and interpretation of data, in writing of the manuscript or in the decision to submit the manuscript for publication. The content of this publication is solely the responsibility of the authors and does not represent the official views of MSD or MSD for Mothers.

Authors’ Affiliations

(1)
Department of Global Health and Development, London School Hygiene and Tropical Medicine
(2)
Sambodhi Research and Communications Limited
(3)
Department of Infectious Disease Epidemiology, London School Hygiene and Tropical Medicine

References

  1. International Institute for Population Sciences (IIPS). National Family Health Survey (MCH and Family Planning), India 1992–93. Bombay: IIPS; 1995.Google Scholar
  2. Office of the Registrar General. Special Bulletin on Maternal Mortality in India 2010-12. New Delhi, India: Government of India; 2013.Google Scholar
  3. Kassebaum NJ, Bertozzi-Villa A, Coggeshall MS, Shackelford KA, Steiner C, Heuton KR, et al. Global, regional, and national levels and causes of maternal mortality during 1990–2013: a systematic analysis for the Global Burden of Disease Study. Lancet. 2013;384(9947):980–1004.Google Scholar
  4. Hunter BM, Bisht R, Chakravarthi I, Murray SF. Demand-side Financing and Promotion of Maternal Health. 2014.Google Scholar
  5. Murray SF, Hunter BM, Bisht R, Ensor T, Bick D. Effects of demand-side financing on utilisation, experiences and outcomes of maternity care in low- and middle-income countries: a systematic review. BMC Pregnancy Childbirth. 2014;14:30.PubMedPubMed CentralGoogle Scholar
  6. Sharma DC. Concern over private sector tilt in India’s new health policy. Lancet. 2015;385(9965):317.PubMedGoogle Scholar
  7. Planning Commission of India. High Level Expert Group Report on Universal Health Coverage for India. New Delhi: Planning Commission of India; 2011.Google Scholar
  8. Sengupta A, Nundy S. The private health sector in India. BMJ. 2005;331(7526):1157–8.PubMedPubMed CentralGoogle Scholar
  9. Saksena P, Xu K, Elovainio R, Perrot J. Health Services Utilization and Out-of-Pocket Expenditure in Public and Private Facilities in Low Income Countries. Geneva: World health report; 2010.Google Scholar
  10. Office of the Registrar General & Census Commissioner. Annual Health Survey 2011-2012 Fact Sheet, Uttar Pradesh. Office of the Registrar General & Census Commissionerth ed. New Delhi, India: Government of India; 2011–2012.Google Scholar
  11. Das J, Gertler PJ. Variations in practice quality in five low-income countries: a conceptual overview. Health Aff (Millwood). 2007;26(3):w296–309.Google Scholar
  12. Mills A, Brugha R, Hanson K, McPake B. What can be done about the private health sector in low-income countries? Bull World Health Organ. 2002;80(4):325–30.PubMedPubMed CentralGoogle Scholar
  13. Patouillard E, Goodman CA, Hanson KG, Mills AJ. Can working with the private for-profit sector improve utilization of quality health services by the poor? A systematic review of the literature. Int J Equity in Health. 2007;6:17.Google Scholar
  14. Greenfield D, Braithwaite J. Health sector accreditation research: a systematic review. Int J Qual Health Care. 2008;20(3):172–83.PubMedGoogle Scholar
  15. Lagarde, M. and N. Palmer. The impact of contracting out on health outcomes and use of health services in low- and middle-income countries. Cochrane Database Syst Rev. 2009(4): p. Cd008133.Google Scholar
  16. Bellows NM, Bellows BW, Warren C. Systematic Review: the use of vouchers for reproductive health services in developing countries: systematic review. Trop Med Int Health. 2011;16(1):84–96.PubMedGoogle Scholar
  17. Koehlmoos, TP, Gazi R, Hossain SS, Zaman K. The effect of social franchising on access to and quality of health services in low- and middle-income countries. Cochrane Database Syst Rev. 2009(1): p. Cd007136.Google Scholar
  18. Beyeler N, York De La Cruz A, Montagu D. The impact of clinical social franchising on health services in low- and middle-income countries: a systematic review. PLoS One. 2013;8(4), e60669.PubMedPubMed CentralGoogle Scholar
  19. Viswanathan R, Schatzkin E, Sprockett A. Clinical Social Franchising Compendium: An Annual Survey of Programs: Findings from 2013. San Francisco: San Francisco: The Global Health Group, Global Health Sciences, University of California; 2014.Google Scholar
  20. Nijmeijer KJ, Fabbricotti IN, Huijsman R. Is franchising in health care valuable? A systematic review. Health Policy Plan. 2014;29(2):164–76.PubMedGoogle Scholar
  21. Centre for Health Market Innovations. CHMI Definitions. [cited 2015; Available from: http://healthmarketinnovations.org/chmi-definitions.
  22. Arrow KJ. Uncertainty and the welfare economics of medical care. 1963. Bull World Health Organ. 2004;82(2):141–9.PubMedPubMed CentralGoogle Scholar
  23. Chalkley M, Malcomson JM. Contracting for health services when patient demand does not reflect quality. J Health Econ. 1998;17(1):1–19.PubMedGoogle Scholar
  24. Mooney G, Ryan M. Agency in health care: getting beyond first principles. J Health Econ. 1993;12(2):125–35.PubMedGoogle Scholar
  25. Rowe AK, Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D. Health Care Provider Performance Review, et al. Systematic review of strategies to improve health care provider performance in low- and middle-income countries. 2014.Google Scholar
  26. Das J, Holla A, Das V, Mohanan M, Tabak D, Chan B. In urban and rural India, a standardized patient study showed low levels of provider training and huge quality gaps. Health Aff (Project Hope). 2012;31(12):2774–84.Google Scholar
  27. Stuart EA. Matching methods for causal inference: a review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics. 2010;25(1):1–21.Google Scholar
  28. International Institute for Population Sciences (IIPS). National Family Health Survey, India 2005–06. Bombay: IIPS; 2007.Google Scholar
  29. Office of the Registrar General & Census Commissioner, Annual Health Survey (AHS). Government of India: New Delhi, India; 2011-2012.Google Scholar
  30. Nagpal J, Sachdeva A, Sengupta Dhar R, Bhargava VL, Bhartia A. Widespread non-adherence to evidence-based maternity care guidelines: a population-based cluster randomised household survey. BJOG. 2015;122(2):238–47.PubMedGoogle Scholar
  31. Organization WH. Care in Normal Birth: A Practical Guide. Geneva: World Health Organization; 1996.Google Scholar
  32. Kruk, M.E, Kujawski S, Mbaruku G, Ramsey K, Moyo W, Freedman LP. Disrespectful and abusive treatment during facility delivery in Tanzania: a facility and community survey. Health Policy and Planning. 2014: p. czu079.Google Scholar
  33. Warren C, Njuki R, Abuya T, Ndwiga C, Maingi G, Serwanga J, et al. Study protocol for promoting respectful maternity care initiative to assess, measure and design interventions to reduce disrespect and abuse during childbirth in Kenya. BMC Pregnancy Childbirth. 2013;13(1):21.PubMedPubMed CentralGoogle Scholar
  34. Angrist JD, Pischke J-S. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton: Princeton university press; 2009.Google Scholar
  35. Anderson ML. Multiple inference and gender differences in the effects of early intervention: a reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects. J Am Stat Assoc. 2008;1481–95:103(484).Google Scholar
  36. Kling JR, Liebman JB, Katz LF. Experimental analysis of neighborhood effects. Econometrica. 2007;75(1):83–119.Google Scholar
  37. Westfall PH. Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment, vol. 279. New York: John Wiley & Sons; 1993.Google Scholar
  38. Hawe P, Shiell A, Riley T. Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. J Epidemiol Community Health. 2004;58(9):788–93.PubMedPubMed CentralGoogle Scholar
  39. Evans C, Lambert H. Implementing community interventions for HIV prevention: insights from project ethnography. Soc Sci Med. 2008;66(2):467–78.PubMedGoogle Scholar
  40. DeWalt, K.M. and B.R. DeWalt. Participant observation: a guide for fieldworkers. 2010: Rowman Altamira; 2010.Google Scholar
  41. Gyldmark M. A review of cost studies of intensive care units: problems with the cost concept. Crit Care Med. 1995;23(5):964–72.PubMedGoogle Scholar
  42. Jegers M, Edbrooke DL, Hibbert CL, Chalfin DB, Burchardi H. Definitions and methods of cost assessment: an intensivist’s guide. ESICM section on health research and outcome working group on cost effectiveness. Intensive Care Med. 2002;28(6):680–5.PubMedGoogle Scholar
  43. Terris-Prestholt F, Fern Terris-Prestholt AS, Sweeney S, Kumaranayake L. The Cost Effectiveness Analysis Guidelines for Syphilis Screening Strategies. 2011.Google Scholar
  44. Walker N, Tam Y, Friberg I. Overview of the Lives Saved Tool (LiST). BMC Public Health. 2013;13 Suppl 3:S1.PubMedPubMed CentralGoogle Scholar
  45. Weinberger M, Pozo-Martin F, Boler T, Fry K, Hopinks K. Impact 2: An Innovative Tool for Estimating the Impact of Reproductive Health Programmes. London: Marie Stopes International; 2012.Google Scholar
  46. Shillcutt SD, Walker DG, Goodman CA, Mills AJ. Cost-effectiveness in low- and middle-income countries: a review of the debates surrounding decision rules. Pharmacoeconomics. 2009;27(11):903–17.PubMedPubMed CentralGoogle Scholar
  47. Revill P, Walker S, Madan J, Ciaranello A, Mwase T, Gibb DM, et al. Using Cost-Effectiveness Thresholds to Determine Value for Money in Low- and Middle-Income Country Healthcare Systems: Are Current International Norms Fit for Purpose? York: Centre for Health Economics Research Paper 98; 2014.Google Scholar
  48. Drummond MF, O'Brien BJ, Stoddart GL, Torrance W. Methods for the Economic Evaluation of Health Care Programmes, in Methods for the Economic Evaluation of Health Care Programmes. Oxford: Oxford University Press; 1997.Google Scholar
  49. Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283(13):1715–22.PubMedGoogle Scholar

Copyright

© Pereira et al.; licensee BioMed Central. 2015

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement