- Open Access
- Open Peer Review
Process evaluation of a social franchising model to improve maternal health: evidence from a multi-methods study in Uttar Pradesh, India
- Loveday Penn-Kekana†1,
- Timothy Powell-Jackson†1Email author,
- Manon Haemmerli1,
- Varun Dutt2,
- Isabelle L. Lange1,
- Aniva Mahapatra3,
- Gaurav Sharma1,
- Kultar Singh2,
- Sunita Singh3,
- Vasudha Shukla2 and
- Catherine Goodman1
© The Author(s). 2018
- Received: 9 April 2018
- Accepted: 6 September 2018
- Published: 24 September 2018
A prominent strategy to engage private sector health providers in low- and middle-income countries is clinical social franchising, an organisational model that applies the principles of commercial franchising for socially beneficial goals. The Matrika programme, a multi-faceted social franchise model to improve maternal health, was implemented in three districts of Uttar Pradesh, India, between 2013 and 2016. Previous research indicates that the intervention was not effective in improving the quality and coverage of maternal health services at the population level. This paper reports findings from an independent external process evaluation, conducted alongside the impact evaluation, with the aim of explaining the impact findings. It focuses on the main component of the programme, the “Sky” social franchise.
We first developed a theory of change, mapping the key mechanisms through which the programme was hypothesised to have impact. We then undertook a multi-methods study, drawing on both quantitative and qualitative primary data from a wide range of sources to assess the extent of implementation and to understand mechanisms of impact and the role of contextual factors. We analysed the quantitative data descriptively to generate indicators of implementation. We undertook a thematic analysis of the qualitative data before holding reflective meetings to triangulate across data sources, synthesise evidence, and identify the main findings. Finally, we used the framework provided by the theory of change to organise and interpret our findings.
We report six key findings. First, despite the franchisor achieving its recruitment targets, the competitive nature of the market for antenatal care meant social franchise providers achieved very low market share. Second, all Sky health providers were branded but community awareness of the franchise remained low. Third, using lower-level providers and community health volunteers to encourage women to attend franchised antenatal care services was ineffective. Fourth, referral linkages were not sufficiently strong between antenatal care providers in the franchise network and delivery care providers. Fifth, Sky health providers had better knowledge and self-reported practice than comparable health providers, but overall, the evidence pointed to poor quality of care across the board. Finally, telemedicine was perceived by clients as an attractive feature, but problems in the implementation of the technology meant its effect on quality of antenatal care was likely limited.
These findings point towards the importance of designing programmes based on a strong theory of change, understanding market conditions and what patients value, and rigorously testing new technologies. The design of future social franchising programmes should take account of the challenges documented in this and other evaluations.
- Process evaluation
- Social franchising
- Maternal health
India’s private health sector is extensive and diverse. It ranges from sophisticated tertiary hospitals providing medical care of an international standard to alternative systems of medicine and unqualified rural health providers. The majority of registered doctors work in the private sector, which is often the first point of contact for a substantial proportion of the population [1–3]. Regulating the private sector in India has proved challenging, and alternative strategies that encourage private providers to raise standards are required . However, there remains limited evidence on the most effective strategies to improve the quality of private sector services [5–8].
One proposed strategy is social franchising, an organisational model that applies some of the principles of commercial franchising to support the provision of branded, quality-assured services of social importance, such as healthcare, via a network of private providers . Although there is considerable variation across programmes, Viswanathan et al.  identify a set of core characteristics that most have in common: (1) a third party administrator, typically an NGO which manages the brand and supervises the network providers through regular visits and audits; (2) the use of protocols and guidelines under which providers must operate; (3) a focus on the sale of healthcare services, in addition to healthcare commodities; (4) the aim of providing quality-assured health services to the most under-served populations; and (5) the aim of achieving self-sustainability both from the franchisor and franchisees’ perspectives. The social franchising model has been applied to a wide range of health-related services and is one of the fastest growing private sector interventions in recent years, with 83 active healthcare social franchising programmes identified in 2015 in low- and middle-income countries (LMICs), 37 of which began between 2007 and 2012 .
Although considerable resources are being channelled into social franchising in LMICs, there is limited rigorous evidence on the effectiveness of clinical social franchising. Moreover, while impact evaluations are needed to measure the effect of complex interventions, process evaluations also have an important role to play in examining “the context and implementation of an intervention, the mechanisms through which it may affect outcomes, and the response of the intervention target population” . When conducted alongside impact evaluations, process evaluations can help explain the “black box” behind complex and multi-faceted interventions, leading to an understanding of the mechanisms underlying their impact (or lack of impact) and thus to the provision of more relevant policy recommendations.
The focus of this paper is the Matrika social franchising model in Uttar Pradesh, India. The programme aimed to improve quality and coverage of maternal health services in three districts of the state. The main element of the programme was the “Sky” social franchise. In previous research, we examined the population impact of the programme using a rigorous quasi-experimental design . Findings from the impact study show that the multi-faceted programme was not effective in improving the quality and coverage of maternal health services . It had no significant effect on facility births, the primary outcome. Across an additional 56 pre-specified outcomes, measuring healthcare use, content of care, patient experience, patient knowledge, healthy behaviours, and financial strain, there was no difference between intervention and comparison areas. There was some evidence of an increase in recommended delivery care practices.
This paper reports findings from a process evaluation, conducted alongside the impact evaluation, with the aim of explaining why the Matrika social franchise programme did not have a population impact. Using a theory of change and multiple sources of data, we examine implementation of the programme, the mechanisms of impact and contextual factors to understand why the programme did not improve the quality and coverage of maternal health services. Although the Matrika programme was multi-faceted, we focus on its core component, the Sky social franchise network.
Social franchise model
The Matrika programme was a complex multi-faceted intervention which sought to improve maternal health primarily by reducing deaths from post-partum haemorrhage. The programme combined various activities to encourage more women to use services and raise the quality of antenatal care (ANC), obstetric care, and family planning services. The intervention was implemented by World Health Partners (the franchisor) in partnership with Pathfinder International. It officially started in March 2013 and ended in May 2016, with a budget of USD 3,250,000.
The rationale for the programme was primarily based on a pragmatic argument that private health providers, since they are widely accessible and frequently used, should be harnessed to improve the health of women during pregnancy and childbirth. However, it was also recognised that their capacity and links with the public sector required strengthening and that there was also inadequate demand for maternal health services. Hence, there was a need for a multi-pronged approach to address both supply- and demand-side constraints to coverage of quality maternal health services.
The second level was SkyHealth centres. These were small clinics owned by individuals who mostly had a qualification in ayurveda, yoga and naturopathy, unani, siddha, or homoeopathy (AYUSH). Their main role within the franchise was to provide ANC consultation, for which they received a financial incentive from the franchisor, and to channel clients towards appropriate facilities for delivery. ANC consultations were to be free to women, though SkyHealth centres could charge for any commodities provided. Most SkyHealth centres were new to ANC provision. During the design stage, there was an expectation that these providers would also provide delivery care within the franchise, but this was vetoed because of concerns about their skill levels. SkyHealth centres were expected to conduct their ANC consultations using telemedicine over the Internet with doctors in the central medical facility, for which they were required to purchase a computer and other equipment. A “medical diagnostic box” allowed them to measure and communicate clinical information, such as blood pressure and pulse, to the central medical facility. SkyHealth centres were also integrated into the public supply chain of iron and folic acid for provision during ANC. At the highest level, Franchise Clinics1 were private hospitals offering delivery and emergency obstetric care under a fee structure set by the franchisor. The franchisor gave branded signage to providers at all levels and marketed the Sky brand through various channels such as wall paintings and radio spots.
To improve and standardise quality of care, clinical training of two to three days and regular quality improvement (mentoring) visits were given to SkyHealth centres and Franchise Clinics (not SkyCare). Although not part of the social franchise network, public facilities in the same districts were identified to act as referral destinations for delivery care and were also given training. SkyHealth centres were trained to provide ANC, recognise and stabilise pregnancy complications, facilitate timely referrals, and provide post-partum contraception counselling. They were also trained in how to operate the telemedicine equipment. Training of Franchise Clinics and public providers covered the same topics as well as emergency obstetric care.
To increase the demand for maternal health services, there were village-level information activities such as wall paintings, billboards, radio spots, and film shows. There was also a 1-day training of ASHAs on birth preparedness, recognition of danger signs during pregnancy, and appropriate sources of ANC and delivery care, with the intention that ASHAs would encourage women to use both public facilities and facilities within the Sky network.
By the end of the programme, 365 SkyCare providers, 50 SkyHealth centres, and 8 Franchise Clinics were part of the network. Clinical training lasting 2 to 3 days was given to 58 private providers, and there were 225 quality improvement visits in 50 private facilities. One day of training was given to 2149 ASHAs.
Uttar Pradesh is the most populous state in India with almost 200 million people in 2011, now estimated to be over 220 million. The population is predominantly rural (77%). Maternal mortality remains high in Uttar Pradesh, with the most recent estimate at 285 deaths per 100,000 live births . The proportion of women giving birth in a health facility has risen rapidly in the past decade, from 39% in 2005–2006 to 79% in 2015–2016 . The public sector accounted for much of the increase, rising from 7 to 45% over the same period. Women with four or more ANC visits increased from 37% in 2005–2006 to 51% in 2015–2016.
The Sky social franchise was implemented in three adjoining districts (Kanpur Nagar, Kanpur Dehat, and Kannauj) in Uttar Pradesh. With Kanpur as its largest city, Kanpur Nagar district is the most populated of the three, with 4.6 million people. With 1.7 and 1.8 million inhabitants, respectively, Kannauj and Kanpur Dehat districts are predominately rural. The private market for maternal healthcare in the study areas is largely made up of small, individually owned hospitals and clinics located in urban and peri-urban areas . Most facilities are owned by doctors and, to a lesser extent, AYUSH providers.
Our approach was guided by the Medical Research Council process evaluation framework, which argues that “an understanding of the causal assumptions underpinning the intervention and use of evaluation to understand how interventions work in practice are vital in building an evidence base that informs policy and practice” . The framework gives prominence to three components of a process evaluation: (1) implementation, (2) mechanisms of impact, and (3) context.
The theory of change provided our analytical framework. Specifically, we analysed the data and organised the results according to the six key mechanisms in Fig. 2, as a means of explaining the success or failure of the programme in generating the desired outputs, ultimately with a view to understanding why the social franchise had no measurable effect on population outcomes. We explored the potential reasons for the impact evaluation results by comparing our findings with the theory of change, paying particular attention to the influence of implementation, mechanisms, and context. The success of the Sky social franchise depended on these mechanisms operating in practice, which in turn required a number of assumptions to hold. Some of these assumptions were identified a priori, while others were explored inductively.
We undertook a multi-methods study , drawing on both quantitative and qualitative primary data. We relied primarily on the quantitative data to assess the extent to which components of the intervention were implemented. Both qualitative and quantitative data were used to understand mechanisms and the role of contextual factors. While these two types of data collection were done by different groups of researchers, the development of the study protocol and the design of tools involved input from all the researchers .
Description of data collection methods
Tools and date of data collection
Household surveys (two rounds—Jan 2015 and May 2016)
3600 (round 1) and 3452 (round 2) households in 180 study clusters. Three types of cluster were surveyed. Group A contained clusters with a SkyCare or SkyHealth provider in the three intervention districts. Group B comprised clusters with no social franchisee in the same three districts. Group C was taken from neighbouring districts that did not have any social franchise network operating within them
Eligible respondents included all women aged 15–49 years who gave birth in the previous 24 months (round 1) or 18 months (round 2), including those who had a stillbirth or whose child died since birth. Eligible women were identified through a census of households, conducted 1 month before the household survey
- Source of care received for ANC, delivery care, and family planning
- Whether the woman was accompanied by an ASHA for ANC and delivery care
- Community awareness of the Sky brand
Health provider surveys (two rounds—Jan 2015 and May 2016)
Using a census of all health providers within the study clusters, we randomly selected for interview one private health provider (social franchisee in intervention clusters), one government health provider, and one ASHA in each cluster. In the second round, we sought to re-interview the same providers or, if not available, a random replacement of the same type. Complete interviews were obtained from 454 health providers in round 1 and 446 interviews in round 2
For the purposes of the census, we defined a health provider as any institution or individual whose primary purpose is to provide healthcare. We excluded drug sellers
- Branding of the facility
- Use of the telemedicine and mobile phone consultations
- Training, supervision, and monitoring
- Health provider knowledge of ANC and actual practice of ANC based on respondents’ recall of their most recent ANC consultation
Clinical observations of antenatal care consultations (Feb–Aug 2016)
A purposive sample of six facilities (4 SkyHealth and 2 Franchise Clinics) was selected to reflect variation amongst facilities within the network. This was not intended to be a representative sample that would allow generalisation across the whole network
25 observations of ANC visits using telemedicine
Social franchise user survey (Apr–Jun 2016)
15 health facilities were selected based on stratified random sampling (9 SkyHealth facilities, 6 Franchise Clinics). 760 women were selected from facility records
Eligible women were those who had received ANC or delivery care from a social franchise facility within the previous year and, in the case of ANC, had given birth by the time of the survey
- Socio-economic characteristics of women and their household
- Pathway of care
- Experience with telemedicine
- Perception of services and costs
Qualitative research: semi-structured interviews and participant observations (including conversations and other interactions) (2016)
From the 15 focal sites that were randomly sampled for the social franchise user survey, we purposively selected 6 “intensive” sites to reflect differences in number of clients and types of services offered: 2 Franchise Clinics and 4 SkyHealth centres
We used a combination of clinic lists and the help of ASHAs to identify women for interview. We purposefully sampled women to reflect a representation of different profiles based on age, parity, level of education, and urban, peri-urban or rural residence
Participant observations and semi-structured interviews with 30 women; 21 ASHAs; 15 SkyCare providers; 11 SkyHealth directors who were still part of the Sky franchise and 5 who had either not joined, left, or been asked to leave; 3 central medical facility staff and 9 franchisor staff
- Topics with franchisees included motivation for joining social franchise, how being part of the social franchise impacted their business and the care they provide, how sustainable the social franchise model is, differences between private and public sectors, and perspectives on their integration
- ASHAs: understanding of social franchise model, traditional responsibilities of an ASHA and how tasks with the social franchise fit into their role, incentives and payment systems, practices of working with public and private sectors
- Women: care-seeking behaviours and decision making in pregnancy and childbirth, perceptions of the interactions with social franchise providers (including telemedicine), ASHAs, and the public sector
- Programme staff: conceptualisation of the model, challenges and successes during implementation, modifications made, financial and sustainability considerations of the social franchise, nature of interactions with stakeholders and partners
Survey tools were translated from English into Hindi and back-translated, before being piloted in the field. The semi-structured interviews were carried out using topic guides that were pilot tested and adapted in the course of data collection. These adaptations were based on regular discussions of emerging themes between the four members of the qualitative research team and in response to findings from the quantitative data. Interviews were transcribed and translated into English using a professional translation service and checked by one of the native Hindi-speaking members of the research team.
We analysed the quantitative data descriptively to generate indicators of implementation and provide insights as to the context surrounding the intervention, reporting means and proportions, as appropriate. Indicators of implementation were grouped according to social franchise recruitment and branding, community-level measures, and provider-level measures of implementation. Individual indicators of provider antenatal care knowledge and practice were combined to generate summary indices, for which we report the mean for different types of health provider in intervention and comparison areas. All quantitative data were analysed in Stata 14.2.
All qualitative interview transcripts were coded using NVivo 10.0 software according to a priori themes related to our evaluation research questions. Further themes were added throughout analysis after familiarisation with the context and data, and coding was stratified by facility and respondent. A simultaneous evaluation of another social franchise in Rajasthan with similar methods gave our findings a point of comparison, which we used to deepen our understanding of the results in this study. As such, analysis was a continuous process. After undertaking a preliminary thematic analysis of the interviews and field notes , we held a number of reflective meetings including researchers involved in all components of the process evaluation to triangulate across data sources, synthesise evidence, and identify the main findings .
We validated preliminary findings through discussion with the implementers of the intervention. We produced a report which we sent to implementers for comment and discussion before finalising. We fed back more finalised results to policymakers and other key stakeholders in India. Insights generated from these interactions were used to finalise the results reported in this paper.
We highlight key findings under each of the six broad mechanisms identified in the theory of change. In each section, we start by outlining the potential mechanisms of impact involved. We then explain the main findings, bringing together evidence on the degree of implementation, the logic of potential mechanisms of impact, and the influence of the context in which the intervention was implemented.
Competitive nature of the market for ANC meant franchise providers achieved very low market share
Establishing SkyHealth facilities within the social franchise network was designed to increase the availability of ANC in rural areas and to provide a point of contact for women to be referred for delivery care in both the public and private sectors.
Social franchise recruitment and branding
Provider census/survey (January 2015)
Sky providers as proportion of private providers in intervention clusters (%)
Sky providers as proportion of all providers in intervention clusters (%)
Sky providers who would recommend others to join social franchise (%)
Sky providers who are male (%)
Sky provider branded (%)
Consistent with the competitive environment and multiple care-seeking options, women’s pathway of care was found to be fragmented in the sense that they often used more than one provider for ANC. Data from the social franchise user survey showed that women who visited SkyHealth centres made on average 4.7 visits for ANC, and out of these, only 1.5 were visits to SkyHealth centres. The data show that these women relied more on the public sector for ANC, particularly healthcare provided at Anganwadi centres, which distributed iron and folic acid as well as tetanus toxoid vaccination at the village level for free. The fragmented care seeking was exacerbated by ASHAs being incentivised to bring women to various types of facilities and sectors. SkyHealth providers often paid ASHAs to bring clients to them, as is standard practice in the rest of the private sector , something that was informally acknowledged by the Sky franchisors who labelled it as reimbursement for ASHA transport costs. Qualitative research revealed that ASHAs would sometimes organise ANC trips for women involving visits to both government and SkyHealth facilities on the same day. Women interviewed in the qualitative research and those observed during the fieldwork often came to SkyHealth providers with ASHAs directly from the public sector.
The low market share suggests that the price subsidies for ANC provided in SkyHealth centres were not sufficient to attract women in reasonable numbers to their facilities. We also note that the vast majority of Sky providers were male when women are known to have a preference for female providers (Table 2).
Awareness of the franchise was low and the brand was not seen as a signal of quality
Branding and other promotional activities sought to raise demand for ANC and delivery care offered by facilities in the social franchise network. The mechanism depended critically on patients knowing about the brand and recognising it as a signal of quality.
Community-level measures of implementation
Household survey (May 2016)
A. Awareness of social franchise amongst women in areas with franchise provider
Heard of Sky brand (%)
Recognise SkyCare logo (%)
Recognise SkyHealth logo (%)
Reported knowing a provider in village who was part of franchise (%)
B. Awareness of social franchise amongst women in districts without franchise
Heard of Sky brand (%)
Recognise SkyCare logo (%)
Recognise SkyHealth logo (%)
Reported knowing a provider in village who was part of franchise (%)
C. Maternal health seeking accompanied by ASHAs
Women using ANC accompanied by ASHAs in intervention districts (%)
Women using ANC accompanied by ASHAs in comparison districts (%)
Women giving birth in facility accompanied by ASHAs in intervention 2003districts (%)
Women giving birth in facility accompanied by ASHAs in comparison districts (%)
These data were supported by the participant observations at facilities which demonstrated that the prominence of Sky branding varied greatly. A common complaint from providers was that the franchisor had not delivered on their promise to market the brand. What marketing had been done was said to be insufficient and often to have been poorly implemented. Some talked about how they had to take the initiative into their own hands and produce their own leaflets and organise health camps in villages (health promotion activities and check-ups in villages).
I am working on a zero balance here also. If we do not do anything of our own, on the side, then we would, I mean WHP can make us completely jobless. They were given the duty of getting the clients for the centre, now they should get them. SkyHealth provider
It was being focused again and again that you increase the number of check-ups [ANC visits] but if I could increase the number of check-ups by myself I would have increased it. But the advertisement – that was not done in the village. Advertise in the villages and make the people aware – that happened once or twice. Van came, vehicle came, but after that was happened was nothing. If a car passes by advertising – the people come down to know what is being advertised. SkyHealth provider
Qualitative findings also suggest that the branding itself was not seen as a signal of quality as women did not know the brand. In 30 interviews with women, and when talking to many more through the participant observations, not one woman mentioned how the Sky brand had attracted her to the facility. They typically reported that they had not heard of the brand. Instead, they had come to the facility on the advice of the ASHA or because they knew the health provider working there. Providers at SkyHealth centres felt that women came because of their personal reputation and not because the brand had attracted patients.
Mechanisms for encouraging women to attend ANC services were ineffective
To establish links between SkyHealth centres and the community, SkyCare providers and ASHAs were given training and offered financial incentives to encourage women to use ANC.
Provider-level measures of implementation
Provider survey (June 2016)
A. Incentive, targets, and referrals
SkyHealth receive incentives for ANC consultations (%)
SkyCare receive incentives for ANC referrals (%)
SkyHealth referred ANC clients for delivery care in past 3 months (%)
SkyCare referred women for ANC in past 3 months (%)
SkyHealth report having targets for ANC (%)
SkyCare report having targets for ANC (%)
SkyHealth providers ever given telemedicine consultation (%)
Mean number of telemedicine consultations in last month
Telemedicine equipment works on day of interview (%)
Sky providers ever trained (%)
Duration of Sky provider training (days)
ASHA received training in past 12 months in intervention districts (%)
ASHA received training in past 12 months in comparison districts (%)
D. Supervision and monitoring
Sky provider ever received supervision visit from WHP (%)
Sky provider received supervision visit in the last 6 months (%)
Mean number of supervision visits in the last 6 months (%)
SkyHealth received feedback on quality of care in the last 6 months (%)
The qualitative research revealed that most SkyCare providers had little or no experience working in the area of maternal health. They only interacted with pregnant women if they were sick and wanted curative care during their pregnancy. Another challenge was that SkyCare providers saw themselves as providers of care and not health educators. As a result, referring women to another facility or getting women to have a phone consultation with the central medical facility was seen as undermining their reputation as someone able to cure health problems and potentially leading to a loss of clients. As a result, some SkyCare providers saw SkyHealth providers as competition.
The women will say to you, ‘you are a doctor.’ Why are you sending me to someone else or making me talk on the phone? SkyCare provider
The relationship between the programme and ASHAs was also problematic. The franchisor initially recruited ASHAs as SkyCare providers, but this caused tension with the state government who felt that these ASHAs were a government resource. As a result, the ASHAs were quickly replaced by rural health providers, and instead, the role of ASHAs was limited to encouraging women to attend ANC and delivery care. SkyCare providers complained that working with ASHAs was challenging and that it became complicated when it came to who should get the incentives from SkyHealth providers for referring women—the ASHA or the SkyCare providers. Some SkyCare providers talked about how they had tried to organise health camps in the village with the SkyHealth provider, hoping to create linkages between the two providers. However, in order to get women to attend the health camp, they had to work with ASHAs, who would afterwards claim money for bringing women.
Finally, we found that the training of ASHAs in the past 12 months was no more common in the intervention districts than in the comparison districts. In other words, ASHAs were receiving training from a range of sources outside the programme such that Matrika was not filling a well-identified gap (Table 4).
Referral linkages were not sufficiently strong between SkyHealth providers and Matrika delivery providers
The Matrika model relied on SkyHealth centres both to provide ANC and to refer patients to the higher-level Franchise Clinics for delivery care. One of the major challenges impeding this mechanism in practice was recruiting Franchise Clinics into the network. Indeed, 11 Franchise Clinics were expected to be referral destinations for pregnant mothers, but during data collection, it was found that only six of them were active in the Matrika programme. Moreover, the precise mechanism through which SkyHealth providers would be successful in encouraging ANC patients to give birth in Matrika delivery facilities was vague. We found from the social franchise user survey that none of the women attending SkyHealth facilities for ANC delivered in Franchise Clinics. Provider survey data showed that less than half the SkyHealth providers had referred any clients for delivery care in the past 3 months (Table 4). At one of the Franchise Clinics, the director reported not receiving any referrals from SkyHealth and SkyCare providers or ASHAs in the 2 years that he had been part of the franchise.
We wanted to get associated with someone whose network is doing well through which we could expand the hospital. I thought at this level –SHC [SkyHealth Centre] – the patient there who comes and if they have complications, they will be admitted here. But this never happened. Franchise Clinic director
Clearly, one of the major barriers to attending Franchise Clinics for delivery was the cost of care. Data from the social franchise user survey showed that the socio-economic status of users was higher at Franchise Clinics (70% in wealth quintile 5 for Uttar Pradesh as a whole and 17% in wealth quintile 4) than at SkyHealth providers where the distribution was more evenly spread across the wealth quintiles (48.5% in the top two quintiles and 16.3% in the two lowest quintiles). Given the relatively high price of delivering in Franchise Clinics (median cost of 117 USD for a normal delivery), it seemed unlikely that SkyHealth users could or would contemplate delivering there when the public sector offered free care. Additionally, the number of these facilities ended up being too low to cover a geographical area large enough for the referral mechanism to function.
Evidence on the influence of quality improvement activities (training and supervision) on quality of care was mixed
There were a number of hypothesised pathways through which improving the capacity of health providers was anticipated to have impact. The clinical training of private providers in the social franchise network was intended to raise quality of ANC, delivery care, and post-partum family planning through increases in knowledge and skills. Quality improvement visits were also expected to increase quality of ANC and delivery care in both private and public facilities.
Provider antenatal care knowledge and practice
Provider survey (June 2016)
A. Provider ANC knowledge (index score, 61 items)
Comparison of SkyHealth and other private AYUSH providers
Private AYUSH (not SkyHealth) in intervention villages
Private AYUSH in comparison villages
Comparison of ASHAs in intervention and comparison districts
ASHA in intervention districts
ASHA in comparison districts
B. Provider ANC practice (index score, 9 items)
Comparison of SkyHealth and other private AYUSH providers
Private AYUSH (not SkyHealth) in intervention villages
Private AYUSH in comparison villages
Comparison of ASHAs in intervention and comparison districts
ASHA in intervention districts
ASHA in comparison districts
However, other data suggest that monitoring and supervision visits were unlikely to have improved quality of care in Sky facilities. Qualitative responses were mixed about the adequacy of visits from WHP staff to support clinics in the SkyHealth facilities. While they acknowledged that franchisor staff visited them regularly, they said that generally these visits did not concern quality—rather WHP staff came to monitor data collection (i.e., names of maternal health patients who attended the clinic). It is possible that these visits were in fact monitoring visits and that the clinic directors simply did not perceive them to be such. In terms of routine quality of care monitoring and improvement, facility heads did not suggest that they were disappointed with the irregularity of supervision, training, feedback, and activities. In general, observations and interviews suggested that most clinic heads were satisfied with their own capacities and viewed the technical management of any clinical staff to be their own domain. For the most part, they did not show themselves to be eager for skill and knowledge improvement.
Telemedicine was an attractive feature for clients but problems in the implementation of the technology meant its effect on quality of ANC was likely limited
Telemedicine was intended to link the patient to a qualified and experienced doctor at the central medical facility, thereby ensuring quality ANC services at SkyHealth facilities. Availability of telemedicine was also supposed to increase demand for ANC.
Various findings raise questions about the benefits of telemedicine in practice. Although telemedicine was seen as an attractive feature of what SkyHealth centres had to offer, we found important flaws in the implementation of the technology. First, multiple sources of data suggest that the telemedicine equipment did not always work. Data from the health provider survey show that only 70% of SkyHealth providers had ever used the telemedicine equipment (Table 4). Only 30% of SkyHealth providers reported that the equipment was working on the day of interview, and the average number of teleconsultations per month was 2.0 (range 0 to 20). The most common problem was internet connectivity (data not shown). The qualitative research raised similar concerns. When talking to women, ASHAs, and providers and observing telemedicine consultations, a number of problems were clear. There were often technical problems or glitches that meant the consultation did not happen or women had to wait a long time, sometimes a number of hours. Women were often a bit bewildered about “who” the doctor was and who they were talking to. In a number of clinics, there was a communication problem, as the central medical facility staff could not understand the dialect that women were speaking. Women felt it was hard to talk to someone they could just see on a screen.
The doctor [on the computer] wasn’t talking to me, she was talking to the other doctor [provider at the clinic] and that doctor was talking to me. The doctor [provider at the clinic] was asking me things and the doctor on the computer. But I couldn’t speak properly to her [the doctor on the computer] as she was from the city. So I would tell the doctor [in the clinic] and he would tell the other doctor. I could see her face but I did not pay attention as I was looking at the doctor at the facility. SkyHealth user
ASHAs also complained the telemedicine often did not work, that the consultations took too long and women became impatient.
Women are worried they had to wait at the facility for many hours. Some women would shout at me saying ‘we have spent some much time here, there is not internet, we have a lot of work at home.’ Some women would get angry at us, they would say they are losing their time because of me. Gradually I lost interest due to the refusals. ASHA
ANC practices observed during telemedicine consultations in SkyHealth facilities
Activities performed by healthcare worker
Washed hands before procedures
Put on examination gloves
Weighed the client
Took the client’s blood pressure
Examined hands for oedema
Performed or referred for urine test
Tested for proteinuria
Tested for glucose
Checked for sign of anaemia
Performed or referred for anaemia test
Palpated client’s abdomen for fundal height
Listened for foetal heartbeat
Performed or referred for syphilis test
This process evaluation of the Sky social franchise was conducted alongside an impact evaluation, which measured a wide range of study outcomes, covering healthcare utilisation, quality of care, patient experience, healthy behaviours, health status, and financial strain . The findings from the impact evaluation showed that there was no evidence that the multi-faceted project had a population impact on the vast majority of outcomes measured, with the exception of a small effect on recommended delivery care practices. Notably, the programme was shown to have no effect on ANC utilisation, ANC content of care, and facility births.
Our approach in the process evaluation was to develop a theory of change to map out the key mechanisms linking inputs and outputs and then to critically assess the logic of the proposed mechanisms, their extent of implementation, and whether they worked in practice. Using the framework provided by the theory of change, we reported six key findings that help explain why the social franchise model was not effective in improving the quality and coverage of maternal health services. First, the franchisor achieved its health provider recruitment targets but the competitive nature of the market for antenatal care meant social franchise providers achieved very low market share. Moreover, the fact that most providers were male may have made the SkyHealth centres a less attractive option. Second, despite the Sky providers being branded, community awareness of the franchise was low and the brand was not perceived as a signal of quality. Third, mechanisms for encouraging women to attend antenatal care services were ineffective. Fourth, referral linkages were not sufficiently strong between antenatal care providers in the franchise network and delivery care providers. Fifth, evidence on the influence of quality improvement activities (training and supervision) on quality of care was mixed. Finally, telemedicine was perceived by clients as an attractive feature, but problems in the implementation of the technology meant its effect on quality of antenatal care was likely limited.
The study had a number of limitations. First, the evaluation focused on the Sky social franchise component of what was a multi-faceted intervention and did not, for example, examine project activities in the public sector. Second, the process evaluation included multiple sources of data that were collected at different points in time in an evolving programme. Indeed, some components of the programme that did not work as intended were recognised by the franchisor, who made adjustments during the course of implementation. At the same time, a strength of our study was the triangulation of different data sources that enabled us to have an in-depth understanding of the design, implementation, and context of the programme. Moreover, the quantitative and qualitative research teams coordinated closely to ensure that the two types of data built on each other and avoided duplication. Third, the analysis and interpretation of data from the process evaluation took place after preliminary findings from the impact evaluation were known. It has been noted that an approach in which the two types of evaluation are conducted independently, blinded from each other’s findings, may be less prone to bias . Finally, some of our quality of care measures were based on self-reports with potential recall problems, and while direct observations are preferable from a methodological standpoint, we conducted only a small number.
The findings pertain to this particular social franchising model, and we note that other social franchising programmes for maternal health are designed quite differently. For example, the Merrygold social franchise in neighbouring Rajasthan seeks to engage only with higher volume delivery care providers in the private sector. On the other hand, there were mechanisms within Matrika that are common to other social franchise models, such as the training, quality improvement support, branding and marketing, and demand generation using community health workers, meaning that our findings raise a number of questions for most clinical social franchising programmes. Although our findings on quality of care were inconclusive, the training of two to three days was shorter than what evidence suggests is necessary to improve practice [21–23]. There is the question of whether patients recognise the brand of the social franchise as a signal of quality, particularly in contexts where the reputation of the provider is paramount. Other social franchise studies similarly reported that branding and marketing activities had little impact on influencing care-seeking behaviour [24, 25]. Relying on government community health workers to generate demand for social franchise services may be optimistic, as documented elsewhere [26, 27]. Also, to assess whether telemedicine can improve quality of care, one must consider the appropriateness of the technology with respect to specific health services and the available infrastructure. Finally, we note that firms in a commercial franchise model are incentivised to maintain quality standards through the threat of expulsion from the franchise and the subsequent loss of business. This mechanism was not a key feature of the Sky franchise model, in its design or implementation. It is debatable whether social franchisors can in practice leverage such incentives, especially when they are looking to meet recruitment targets and, if not, what the implications are for their success.
The Sky franchise model for maternal health in Uttar Pradesh was adapted from an earlier programme focused on child health in Bihar. At the time, the model was seen as quite successful based on programme implementation data, though a rigorous evaluation later found no impact . Moreover, our results highlight some of the pitfalls of transporting mechanisms from one disease area to another without appropriate adaptation. Maternal health services, and particularly delivery care, require a more sophisticated level of skill and equipment than most ambulatory care for childhood diseases, meaning that improving quality was much more challenging and that village-level providers were ultimately not deemed appropriate as delivery providers. In addition, the challenge of maintaining continuity of care during pregnancy and identifying women at high risk was perhaps under-appreciated.
The broader markets for maternal care and curative care are also quite different, highlighting the importance of contextual factors in explaining intervention impact. An underlying assumption of the Sky model, perhaps transferred from the Bihar model, was that as rural health providers are often the only health providers in the vicinity of a village and are frequently used, it made sense to work with these providers to train them to perform ANC and encourage them to refer women for delivery. However, while these rural health providers were also widely used in Uttar Pradesh, they were not the usual choice for maternal healthcare or for health promotion activities, and it was difficult to shift these care-seeking patterns. In addition, the competitive nature of the market explained the low market share achieved by social franchise providers, a finding that is consistent with an evaluation of the franchising model in Bihar . Public sector providers had a distinct competitive advantage through a number of existing policies, including (1) the Janani Suraksha Yojana (JSY) incentive scheme, which provided financial incentives to ASHAs and women when the women sought care in public hospitals for ANC and delivery ; (2) free ambulance services for transport to public facilities ; and (3) free public ANC care. Other influential contextual factors included difficulties in recruiting female health providers into the network and the existing infrastructure that meant the telemedicine technology did not function as well as intended.
The evaluation has clear lessons for project design. Specifically, at the design stage of a project, it can be extremely useful for implementers to develop a theory of change and critically probe each of the intended pathways of impacts. This process can shed light on the logic and coherence of the programme. There were a number of components of the Sky social franchise that were poorly conceived. In some cases, it is with the benefit of hindsight that we can say this—e.g., the market for ANC was more competitive than first assumed. But in other cases, components were not sufficiently thought through at the outset—e.g., women using SkyHealth providers were unlikely to be able to afford the prices charged for delivery care at Franchise Clinics. Probing the intended pathways of impact would also require formative research and small-scale piloting prior to implementation in order to understand market conditions, what patients value, and how new technologies perform.
Although caution should always be exercised when generalising findings, our focus on mechanisms in process evaluation should give the study greater external validity. At the very least, the findings should place a higher burden of proof on policymakers and funders who propose investing in social franchising for maternal health. Relative to other studies on social franchising, the evaluation of this social franchise is at the rigorous end of the spectrum; the existing evidence on social franchising in LMICs remains weak , and engagement with the private sector through social franchising appears to be moving in advance of evidence. However, there is also a lack of evidence on alternative effective and cost-effective strategies to improve quality amongst these providers, indicating that further innovation and experimentation should be encouraged. As institutional delivery rates continue to rise, the question of how to improve quality should be at the forefront of policymakers’ minds. This is an area of research that needs much more attention, combining the expertise of clinicians, social scientists, management scientists, and other disciplines.
The findings presented in this process evaluation offer key insights into why the social franchise model did not improve the quality and coverage of maternal health services. They point towards the importance of designing programmes based on a strong theory of change, understanding market conditions and what patients value, and rigorously testing new technologies. The design of future social franchising programmes should take account of the challenges documented in this and other evaluations.
Franchise Clinics consisted of privately run facilities as well as two facilities owned and administered by WHP
We wish to express our gratitude to the numerous women and health providers who gave up their valuable time to participate in the study and share with us their experiences. We thank the implementing partners, World Health Partners and Pathfinder International, for facilitating as well as participating in the research.
This research is supported by funding from MSD, through its MSD for Mothers programme. MSD has no role in the design, collection, analysis, and interpretation of data; in the writing of manuscripts; or in decisions to submit manuscripts for publication. The content of all publications is solely the responsibility of the authors and does not represent the official views of MSD. MSD for Mothers is an initiative of Merck & Co., Inc., Kenilworth, NJ, USA.
Availability of data and materials
Data from the two rounds of the household survey of women are available from LSHTM’s DataCompass repository (https://doi.org/10.17037/DATA.263). Data from the two rounds of the health provider survey will be available from LSHTM’s DataCompass repository. Full transcriptions of the qualitative data are not available as they may contain quotes and identifiable information that could compromise the identity of participants.
LP-K and TP-J developed the study protocol of the process evaluation. TP-J, VD, KS, and VS were responsible for the household survey of women and the health provider survey. LP-K, ILL, AM, and SS were responsible for the qualitative research and the clinical observations. MH was responsible for the social franchise user survey. TP-J, MH, and GS contributed to the analysis of the quantitative data. LP-K, ILL, AM, and SS contributed to the analysis of the qualitative data. MH, TP-J, LP-K, and CG conceptualised and drafted the manuscript. All authors commented on the manuscript and approved the final version.
Ethics approval and consent to participate
The study received ethics approval from the Indian Council of Medical Research (HMSC/2014/10/HSR), Public Healthcare Society in India (10/Nov/2013), the London School of Hygiene and Tropical Medicine (London, UK; 8610 and 9083), and Gene Bandhu Ethics Committee (ECG001/2015). Written informed consent was obtained from participants.
Consent for publication
LP-K, TP-J, MH, ILL, and CG report receiving a research grant from Merck Sharp and Dohme (MSD) through its MSD for Mothers programme. All other authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Planning Commission of India. High level expert group report on universal health coverage for India. New Delhi: Government of India; 2011.Google Scholar
- Saksena P, Xu K, Elovainio R, Perrot J. Health services utilization and out-of-pocket expenditure at public and private facilities in low-income countries, vol. 2010; 2010.Google Scholar
- Sengupta A, Nundy S. The private health sector in India. BMJ. 2005;331:0–1.View ArticleGoogle Scholar
- Sheikh K, Saligram PS, Hort K. What explains regulatory failure? Analysing the architecture of health care regulation in two Indian states. Health Policy Plan. 2015;30(1):39–55.View ArticleGoogle Scholar
- Mills A, Brugha R, Hanson K, McPake B. What can be done about the private health sector in low-income countries? Bull World Health Organ. 2002;80(4):325–30.PubMedPubMed CentralGoogle Scholar
- Montagu D, Goodman C. Prohibit, constrain, encourage, or purchase: how should we engage with the private health-care sector? Lancet. 2016;388:613–21.View ArticleGoogle Scholar
- Montagu D, Goodman C, Berman P, Penn A, Visconti A. Recent trends in working with the private sector to improve basic healthcare: a review of evidence and interventions. Health Policy Plan. 2016;31:1117–32.View ArticleGoogle Scholar
- Patouillard E, Goodman CA, Hanson KG, Mills AJ. Can working with the private for-profit sector improve utilization of quality health services by the poor? A systematic review of the literature. Int J Equity Health. 2007;6:17.View ArticleGoogle Scholar
- Viswanathan R, Behl R, Seefeld CA. Clinical social franchising compendium: an annual survey of programs: findings from 2015. San Francisco: The Global Health Group, Global Health Sciences, University of California, San Francisco; 2016.Google Scholar
- Rath S, Nair N, Tripathy PK, Barnett S, Rath S, Mahapatra R, Gope R, Bajpai A, Sinha R, Costello A, et al. Explaining the impact of a women's group led community mobilisation intervention on maternal and newborn health outcomes: the Ekjut trial process evaluation. BMC Int Health Hum Rights. 2010;10:25.View ArticleGoogle Scholar
- Pereira S, Kumar P, Dutt V, Haldar K, Penn-Kekana L, Santos A, Powell-Jackson T. Protocol for the evaluation of a social franchising model to improve maternal health in Uttar Pradesh, India. Implement Sci. 2015;10(1):77.View ArticleGoogle Scholar
- Tougher S, Dutt V, Pereira S, Haldar K, Shukla V, Singh K, Kumar P, Goodman C, Powell-Jackson T. Effect of a multifaceted social franchising model on quality and coverage of maternal, newborn, and reproductive health-care services in Uttar Pradesh, India: a quasi-experimental study. Lancet Glob Health. 2018;6(2):e211–21.View ArticleGoogle Scholar
- Office of the Registrar General. Special Bulletin on Maternal Mortality in India 2011–13. New Delhi: Government of India; 2016.Google Scholar
- IIPS. National Family Health Survey IV (2015–16): State Fact Sheet Uttar Pradesh. Mumbai: International Institute for Population Sciences; 2017.Google Scholar
- Goodman C, Gauthan M, Iles R, Bruxvoort K, Subharwal M, Gupta S, Jain M. The nature of competition faced by private providers of maternal health services in Uttar Pradesh, India. London: London School of Hygiene and Tropical Medicine; 2017.Google Scholar
- Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.View ArticleGoogle Scholar
- Morse JM. Principles of mixed methods and multimethod research design. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in social & behavioral research. Thousand Oaks: Sage; 2003. p. 189–208.Google Scholar
- WHO. Antenatal care: report of a technical working group. Geneva: World Health Organization; 1994. WHO/FRH/MSM/ 96.98Google Scholar
- WHO. Integrated management of pregnancy and childbirth. Geneva: World Health Organization; 2003.Google Scholar
- Creswell J. Qualitative inquiry and research design: choosing among five traditions. Thousand oaks: Sage Publications; 1998.Google Scholar
- Rowe A, Rowe S, Peters D, Holloway K, Chalker J, Ross-Degnan D. Health care provider review. In: National Academies of Sciences, editor. Improving quality of care in low- and middle-income countries: workshop summary. Washington: The National Academies Press; 2015.Google Scholar
- Mohanan M, Giardili S, Das V, Rabin TL, Raj SS, Schwartz JI, Seth A, Goldhaber-Fiebert JD, Miller G, Vera-Hernandez M. Evaluation of a social franchising and telemedicine programme and the care provided for childhood diarrhoea and pneumonia, Bihar, India. Bull World Health Organ. 2017;95(5):343–352E.View ArticleGoogle Scholar
- Das J, Chowdhury A, Hussam R, Banerjee AV. The impact of training informal health care providers in India: A randomized controlled trial. Science. 2016;354(6308):aaf7384.View ArticleGoogle Scholar
- Lönnroth K, Aung T, Maung W, Kluge H, Uplekar M. Social franchising of TB care through private GPs in Myanmar: an assessment of treatment results, access, equity and financial protection. Health Policy Plan. 2007;22:156–66.View ArticleGoogle Scholar
- Sieverding M, Briegleb C, Montagu D. User experiences with clinical social franchising: qualitative insights from providers and clients in Ghana and Kenya. BMC Health Serv Res. 2015;15:49.View ArticleGoogle Scholar
- Nguyen PH, Menon P, Keithly SC, Kim SS, Hajeebhoy N, Tran LM, Ruel MT, Rawat R. Program impact pathway analysis of a social franchise model shows potential to improve infant and young child feeding practices in Vietnam. J Nutr. 2014;144(10):1627–36.View ArticleGoogle Scholar
- Ravindran TS. Public-private partnerships in maternal health services. Econ Pol Wkly. 2011;43–52.Google Scholar
- Mohanan M, Babiarz KS, Goldhaber-Fiebert JD, Miller G, Vera-Hernandez M. Effect of a large-scale social franchising and telemedicine program on childhood diarrhea and pneumonia outcomes in India. Health Aff (Millwood). 2016;35(10):1800–9.View ArticleGoogle Scholar
- Powell-Jackson T, Mazumdar S, Mills A. Financial incentives in health: new evidence from India’s Janani Suraksha Yojana. J Health Econ. 2015;43(0):154–69.View ArticleGoogle Scholar
- Babiarz KS, Mahadevan SV, Divi N, Miller G. Ambulance service associated with reduced probabilities of neonatal and infant mortality in two Indian states. Health Aff (Millwood). 2016;35(10):1774–82.View ArticleGoogle Scholar