- Open Access
The last one heard: the importance of an early-stage participatory evaluation for programme implementation
Implementation Science volume 9, Article number: 137 (2014)
The systematic involvement of project beneficiaries in community maternal and child health programmes remains low and limited, especially during the formative stages of the project cycle. Understanding how positive and negative feedbacks obtained from communities can subsequently be used to inform and iterate existing programmes is an important step towards ensuring the success of community health workers for maternal and child health programming and, ultimately, for improving health outcomes.
The study took place over a period of 4 weeks in North Rukiga, Kabale District of southwestern Uganda. Using a cross-sectional qualitative study that employed an epistemological approach of phenomenology, nine focus group discussions and eight in-depth interviews were conducted with a total of 76 female participants across six different sites. Women were identified as either users or non-users of the maternal and child health programme. Purposeful sampling was employed to recruit women from six different locations within the programme catchment area. Translated and transcribed transcripts were subjected to a bottom-up thematic analysis using NVivo 10 Software, whereby themes were arrived at inductively.
Predominant themes emerging from the focus groups and key informant interviews identified early trends in programme strengths. Beneficiaries reported confidence in both the programme and the relationships they had forged with community health workers, exhibited pride in the knowledge they had received, and described improved spousal involvement. Beneficiaries also identified a number of programme challenges including barriers to adopting the behaviours promoted by the programme, and highlighted issues with programme dependency and perceived ownership. It also emerged that community health workers were not reaching the entire population of intended programme beneficiaries.
This research provides support for the importance of an early-stage participatory evaluation of beneficiaries’ perceptions of newly initiated health programmes. Our results support how evaluations conducted in the early phases of programme implementation can provide valuable, timely feedback as well as yield recommendations for programme adjustment or re-alignment, and in turn, better meet end-user expectations. Potential reasons for the observed lack of community participation in early stages of programme implementation are considered.
A growing body of literature advocates for the important contribution of formative evaluations in the implementation of health programmes ,. These early-stage evaluations provide in-depth, contextually relevant, and timely information on factors that can potentially enhance or impede implementation success. However, formative evaluations are rarely utilised, or if they are, seldom are they documented for peer review. What emerges is an increasing need to report methodologies for conducting these evaluations to ensure their findings are applied to practice ,. In addition, and despite the Paris Declaration on Aid Effectiveness’ recommendation to increase the use of local systems for programme design, implementation, and monitoring and evaluation, local involvement in health programmes remains low and limited during the formative stages of a project cycle ,. An overemphasis on measuring clinical outcomes, most of which are aligned to the indicators outlined in the Millennium Development Goals (MDGs), has led to the neglect of other important factors. Namely, the key characteristics and processes through which effective outcomes are achieved and the opinions of the end-users with regard to how effective they perceive the intervention to be .
Progress towards meeting the MDGs 4 and 5 remains particularly slow in under-resourced areas of low- and middle-income countries (LMICs), where 99% of maternal and child deaths occur . Each year, a quarter of a million women die from complications related to pregnancy and childbirth and 7.2 million children die before they reach their fifth birthday ,. Faced with these stark figures, most countries are not expected to meet MDGs 4 and 5 by 2015 ,. Uganda is one such country ,. The most recent Ugandan Demographic Health Survey reported a maternal mortality ratio (MMR) of 438 per 100,000 live births and an under-5 mortality of 90 per 1,000 live births . The primary causes of child mortality in Uganda remain malaria, neonatal diseases, pneumonia, and diarrhoea, comprising 25%, 23%, 19%, and 17% of child deaths, respectively . Apart from the high under-5 mortality rates, there are many other areas of concern for child health in Uganda, with 32% of children under-5 being stunted . Seventy-three percent and 20% of children under-5 in Uganda are iron and vitamin A deficient, respectively . Only 52% of children between 12 and 23 months are fully immunised, falling short of the 80% target . Despite neonatal deaths accounting for 39% of all infant deaths , 86% do not receive any postnatal visit, with only 2% of newborns receiving care within the recommended first hour after birth and only 9% receiving care within the first 24 h . The 2011 Demographic Health Survey also identified appropriate breastfeeding practices for children 0–24 months as problematic, as only 63% of children under 6 months are exclusively breastfeed, and only 6% of children 6–23 months are fed according to Infant and Young Child Feeding (IYCF) recommendations . As a result, Uganda continues to rank among the world’s countries with the poorest maternal and child health indicators.
Evidence-based, cost-effective interventions for reducing maternal and child morbidity and mortality are well known throughout the literature ,. When implemented at the community level, these simple interventions are estimated to prevent between 40% and 72% of under-5 deaths ,. Most of these interventions hinge on specific changes in health behaviour and include practices such as accessing health care and vaccinations at appropriate times, exclusive and appropriate breastfeeding, sleeping under an insecticide treated net, and improved hand-washing and water storage behaviours . Unfortunately, the low adoption of these practices among both women and children has significantly hampered their success in improving maternal and child health outcomes . The uptake of these evidence-based interventions is often hindered by a lack of early-stage engagement of the very communities programmes are meant to serve . What emerges too often is a system where programmes are dictated by top-down processes, which, some argue, shifts the ownership and accountability away from communities and beneficiaries . Results from previous RCTs in Uganda, for example, demonstrate that strengthening community monitoring and accountability mechanisms can significantly increase both health outcomes and service utilisation . Another second factor contributing to the low coverage of effective community-level interventions is a global shortfall of approximately 4.3 million health workers ,. Community health workers (CHWs)  have the potential to act as an important bridge between communities and formal health systems  and are effective in providing essential MCH services during the critical days surrounding childbirth for both mother and baby in resource poor settings -.
Understanding how positive and negative feedback can be obtained from communities and how such feedback can subsequently be used to inform and iterate existing programmes is a crucial step towards ensuring the successful adoption of MCH programming. Changes in health behaviours should be achieved through incremental improvement and the incorporation of learning through community participation . Community participation, interchangeably used here with engagement, is considered as the process through which community members influence CHW for MCH programmes, with the goal of enhancing community well-being . Though several challenges for its practice exist , community participation is recognised within Alma-Ata as best practice in health programmes and has been linked to increasing ownership and quality of services, which can subsequently impact on the success of activities -.
An effective method of promoting community participation is through the involvement of programme beneficiaries throughout the monitoring and evaluation (M&E) process. Exploring the factors that contribute to the success and/or failure of community members, such as CHWs, engaging in MCH programmes provides an important opportunity for shared learning on what policies and implementation strategies are most effective in the improvement of MCH outcomes at community level. By studying users’ views of a health service, it becomes possible to identify areas for improvement and strengthening as well as acceptance of the initiative. Involving project beneficiaries in the evaluation of programmes can enhance individual self-determination , empower individuals ,, and assist in addressing inherent power imbalances between project beneficiaries and staff . Atkinson and Haran  state that assessing user satisfaction can increase compliance, explore knowledge transfer and patient involvement, inform policy design, and compare alternatives. Furthermore, user satisfaction can predict acceptance of a proposed strategy, which often influences future utilisation, and compliance of care ,.
Understanding a health service from the user’s point of view has important and widely applicable implications for the future implementation of CHW for MCH programmes in Uganda and for the broader public health policy arena. The purpose of our research is to better understand mothers’ perceptions and acceptability to a newly initiated CHW for MCH strategy at the initial stages of programme implementation through a participatory formative evaluation. By conducting this type of investigation, it is anticipated that the beneficiaries become an active part of the monitoring and evaluation process. Moreover, the implication is that findings can potentially contribute to inform programme iteration for health-care policy and decision makers from an early stage so that appropriate, timely changes can be made. Finally, a greater understanding of the perceptions of target beneficiaries may lead to specific changes that make the programme more acceptable, therefore increasing utilisation and ultimately improving MCH outcomes. This research is timely given an increasing demand for a shift in methods and standards of evaluation. Programmes are increasingly moving from output evaluations to the evaluation of results and impact, while developing more collaborative approaches to programme evaluations ,.
1 timed and targeted counselling (ttC)
The research described focuses around a newly initiated maternal and child health programme, Access to Infant and Maternal Health (AIM-Health). The AIM-Health employs a behaviour change communication (BCC) approach to improve health and nutrition outcomes for pregnant women and children under-2 through the use of timely and appropriately targeted messages. These messages originate from evidence-based, cost-effective practices ,, and are highlighted in both WHO and UNICEF guidelines as best practice to target the primary causes of maternal and child morbidity and mortality in LMICs . The messages focus on 7 key interventions for pregnant and lactating mothers and 11 key interventions for children under-2, which is known as “7–11” within World Vision’s programmatic approach. The 7–11 interventions are subsequently delivered through a minimum of ten household counselling visits made by a CHW at specific times. The counselling approach is based on the American College of Nurse-Midwives home-based life saving skills (HBLSS) technique .
This study took place over a period of 4 weeks in North Rukiga, Kabale District of southwest Uganda. This area was chosen since it was approximately 2 months into a newly initiated 5-year CHW programme led by World Vision Ireland. The AIM-Health programme is currently being implemented across ten settings in five different sub-Saharan African contexts. In Uganda, the programme is run in collaboration with World Vision Uganda and the Ugandan Ministry of Health (UMOH). All research occurred in the North Rukiga county of Kabale District in southwest Uganda, which consists of primarily of subsistence farmers and comprises two sub-counties, Kashambya and Rwamucucu, totaling 13 parishes. The southwest region of Uganda has some of the worst MCH indicators for the country, with only 25% of deliveries occurring in a health facility, and 19% of women giving birth with no assistance whatsoever . This region also has the lowest percentage of postnatal care (PNC) with only 18% attendance .
We conducted a cross-sectional qualitative study that used the epistemological approach of phenomenology through in-depth interviews (IDIs) and focus group discussions (FGDs) to understand a woman’s lived experiences with the CHW programme to date. Village chairpersons acted as mobilisers using purposive sampling to gain access and disseminate research materials to potential participants.
A semi-structured interview guide was developed for both FGDs and IDIs to assist in eliciting individual responses; however, it was not followed rigidly as phenomenological research is concerned with experiences and issues of importance to the individuals and is based on personal knowledge and subjectivity . Women were given a preference of attending an FGD or an individual interview. FGDs are recognised as a strong methodology when discussing potentially sensitive topics , and the researchers felt this method may help women be more forthcoming about both their pregnancy experiences, as well as any problems they were facing with the MCH programme. Individual key-informant interviews were also offered as a methodology since using both FGD and in-depth interviews increases comprehensiveness in collection and allows for a more reflexive analysis .
Participants were first asked a question not specific to the programme (i.e. “What are some of the challenges that pregnant and nursing women face in your community?”). During the interview, the researcher encouraged the participants to control the flow of the conversation. Though attempts were made to avoid direct questions, the researcher was required to ask additional probing questions when conversations stalled. Interviews were conducted in the local language (Rukiga) by a female health professional, trained in interview techniques, and who was familiar with the AIM-Health programme. A member of the research team was also present to capture the tone, mannerisms, and note the body language of the participants. Recorded discussions were subsequently transcribed, translated, and the content verified by a bilingual Rukiga and English speaker. In total, nine FGDs and eight in-depth interviews were conducted with a total of 76 participants across six different sites, three from each of the sub-counties (Table 1).
Participants were pregnant and/or breastfeeding women residing in World Vision’s programme area of North Rukiga and therefore potential beneficiaries of the AIM-Health project. Women were self-identified as either users or non-users of the service provided by CHWs as part of AIM-Health, had to be at least 18 years of age, and had to have completed the informed consent process. Women were considered “users” of 7–11 if they had been visited at least once by a trained CHW. Contrastingly, a “non-user” was defined as a woman who had not been visited by a CHW trained as part of the 7–11 strategy despite living in the programme target areas. Purposeful sampling using community “mobilisers” was employed to recruit only AIM-Health potential beneficiaries from six different locations within the programme catchment area. Community mobilisers included either pregnant women or community nurse/midwives working out of a local health facility. Mobilisers were given participant information leaflets to either distribute or read to potential participants in their communities 7 days prior to data collection, which included information on participant requirements, methodology, and purpose of study. Interested participants were asked to reconvene at a particular date and time if they wanted to participate in the study. Specific times were allocated for both the FGDs and IDIs, with participants choosing which time and therefore which interview method they wanted to participate in. Upon arrival at the study site, the self-identified as either pregnant and/or breastfeeding women were asked their age and were asked for their preference of type of interview type. The six different locations were identified in consultation with UMOH officials, who chose them as they felt these locations best represented the population of North Rukiga.
Translated and transcribed data were subjected to a bottom-up thematic analysis using NVivo 10 Software. No thematic groups were determined prior to data collection and analysis. A “theme” was therefore characterised as a recurrent underlying concept that offers a general insight from the entire data range . Themes were arrived at inductively, as no hypothesis or thematic groups were present prior to analysis . The analysis process was continuous and relied on the researcher’s growing familiarity with the data . The analytical approach used was constructivism whereby the researcher employs a transactional and subjective approach and uses the interaction between participant and herself as well as her own personal knowledge and experience to analyse the data . All transcripts were read thoroughly, and two main thematic groups (positive experiences and negative experiences) were included in the first level of coding. These groups were subsequently coded further to form sub-groups, with a third level of analysis occurring in specific themes that were rich in data. Though the research team attempted to reduce preconceptions and individual convictions during the collection and analysis phase, phenomenology and constructivism both recognise that analysis is a cognitive process and as such, it is subject to the researcher’s own view, understanding, conceptual orientations and experiences of the world, and is therefore undoubtedly affected by what they witness ,.
Ethical approval for this research was obtained from the Health Policy and Management/Centre for Global Health Research Ethics Committee, Trinity College Dublin and the Higher Degrees Research and Ethics Committee, Makerere University School of Public Health, Uganda. Aligned with best practice for illiterate participants, both written and verbal informed consent was obtained from participants.
A total of 76 pregnant or breastfeeding women participated in this study, of which 17 had not been visited by a CHW despite living within the programme catchment area despite having previously expressed a desire to be enrolled (Table 1). The majority (n = 70) was unemployed and only 23% had at least some secondary education. Seventy-two participants reported attending at least one ANC visit, with 2 non-users and 2 users having never sought ANC services. The mean reported walking time to a health facility of was 75 min (s = 76.6). Twenty-five participants reported living over 1.5 h walking distance from the health facility, despite the acceptable distance of 5 km (assumed to take approximately 1-h walking), as defined by the Government of Uganda . For 70 (92.1%) of respondents, the nearest health centre was a Health Facility II, which is not equipped with ANC or delivery services. Table 2 includes demographic information for all study participants. The perceptions of both users and non-users towards the MCH programme highlighted areas of encouragement regarding the programme, some unexpected initial trends, as well as areas of concern that if not addressed, may hamper the programme’s success.
Predominant themes emerging from the focus groups and interviews that highlighted early trends in programme strengths were as follows:
Confidence in and importance of CHWs and 7–11
The value ascribed to both the work of the CHWs and the MCH programme initiatives was high for all (users and non-users), as they identified the potential benefits to both their own and their families’ health. When asked what changes were noticed since the initiation of the programme, user participants identified a number of positive changes ranging from improvements in malnutrition to household water and sanitation. As one respondent states:
“We didn’t have a toilet, I see my man is trying to construct one. We didn’t have a bathroom, I see the man is trying to build one. We didn’t even have time to sweep. I see I’m now trying to sweep.” (FGD2, 37, user)
In addition, women viewed the CHWs as agents of change that are able to impart knowledge that communities would absorb and explained that these would then have positive health impacts. As one non-user described:
“The change which would come, like those women who deliver from home, like when they [the CHW] tell them that they have to go to the hospital people would deliver from the hospital, but because some of them have not been told, they might get problems which they would not have got.” (FGD9, 27, non-user)
Pride in health knowledge
Throughout the interviews, feelings of pride and empowerment in one’s health knowledge and ability to carry out the AIM-Health programme were evident. Observation methods used during the focus groups noted that women began to act as teachers to other participants, passing on recently learned health messages with confidence and enthusiasm. When discussing the MCH messages delivered by CHWs, women who were enrolled in the programme often went into great detail about maternal and child health care practices. On more than one occasion, when the women were asked if there was anything they wanted to add, they spontaneously took turns displaying their knowledge of MCH, in an obvious attempt to display what they had learned. In contrast, women who had not been visited by the CHWs did not exude the same level of confidence around pregnancy-related issues and often doubted their abilities, indicating that they needed assistance and requesting future visits from CHWs.
Relationship with CHW
In general, the CHWs that visited women in their homes were highly regarded due to their knowledge, commitment to the job, and their caring nature. The consistency of care offered by the CHW was made apparent when one participant stated:
“These doctors in our village, I see we have a very good relationship with them, a very good relationship with them, because they keep on coming to see how our health is…” (FGD2, 30, user)
Several enrolled participants also identified and recognised situations where CHWs undertook activities that went beyond their job description, for instance supplying food or paying for transport to the hospital, which appeared to further strengthen the bond between the CHW and the mother. One woman exemplifies this when she explains how she sequestered CHWs when dealing with marital disputes, stating:
“I tell them [CHWs] that `for sure, my man is beating me seriously’, or sometimes I tell them he wants to beat me up. They came and cool him down.” (IDI, 22, user)
Increased male involvement, as previously seen in the case of increased participation in the construction of latrines, was a reoccurring theme throughout the discussions. Increased spousal support emerged specifically from women who received home visits by CHWs, as they commonly expressed changes they had noticed in their husband’s behaviours towards their pregnancy and childcare:
“Since this programme came a man would just impregnate you and go but now when you know that you are pregnant, even if it is like one month, he plans to go with you to the hospital and you continue to go for antenatal care with him.” (FGD4, 22, user)
Participants expressed satisfaction with these changes and attributed increased spousal involvement and knowledge to the CHWs involving both them and their husbands in the counselling sessions. Statements like the ones below were common and present in all focus groups and interviews with women who were enrolled in the programme:
“But when they [the CHWs] went for training, when they [CHWs] find you at home with your man, they teach you and you see the man listening to everything and making sure he does it.” (FGD7, 24, user)
Interestingly, women who were not enrolled in the programme commonly brought up issues around their partners’ lack of involvement in or understanding of her pregnancy. Women often acknowledged that they believed being visited would change their husbands’ attitudes towards their pregnancy, as one woman stated:
“…Because they [the CHWs] didn’t come to advise us, when they [husbands] are in our homes with us, it brings a problem very much because when you tell him [the husband] that, `this and this is needed’, like, if a woman is pregnant, she doesn’t need to carry heavy things, he will just look at you, as if he has not heard anything.” (FGD8, 20, non-user)
Disconcerting themes identified through the focus groups and interviews with both enrolled and non-enrolled women are as follows.
Barriers to programme adherence—insufficient resources and hospital level
The most predominate theme in this category is that the project did little to reduce the barriers that impede women from engaging in appropriate and programme-encouraged health-seeking behaviours. Women questioned to what degree knowledge alone could contribute towards the improvement of health outcomes for their families. This was a common theme, which featured strongly in every focus group and all but one in-depth interview. Two main barriers, a lack of financial means and resources and barriers at the point of the health centre, were cited as contributing to the inability of women to engage in the healthy behaviours promoted by the CHW. Securing resources to procure materials and food proved very difficult for many women. As two women stated:
“I saw them coming to advise us when we have given birth, that you should eat well, like meat, yet you don’t have money… We don’t have a way of getting the things they are telling us.” (FGD4, 32, user)
“If they advise me on something which I can manage to do, I’m O.K. But if they are advising me on something which I can’t do, I feel bad.” (FGD7, 20, user)
Women also met barriers to a healthy delivery at the health centre level. This appeared to be especially frustrating for mothers. Issues such as not having their own delivery kits, having no money for transport in the case of a referral, and either the lack of health staff or quality of care received were all identified as barriers encountered at the health centre. Many of these barriers were mentioned by both users and non-users of the programme, with non-users citing husbands as an additional barrier.
Though the AIM-Health programme intends to be a community-based and community-led initiative, overseen by the Ministry of Health and only supported by World Vision Uganda, women almost exclusively referred to it as a World Vision programme. Commonly cited notions, such as “This programme of World Vision…” (FGD6, 32, user) or “These doctors [the CHWs] in the village you have given us…” (FGD2, 37, user), highlight the fact that despite best intentions, community members do not regard the programme as the community-led initiative. A potential repercussion of this lack of ownership is demonstrated when a non-user expressed that though there was a problem with the CHWs not visiting homes, she was unable to impart change, stating, “It makes us feel bad, but we have nothing to do about it.” (FGD8, un, non-user).
When the researcher specifically asked several non-users what they recommend as a possible solution to this situation, many stated that it was World Vision’s responsibility to remove the inactive CHWs and to select and train new CHWs to fill their role.
AIM-Health intends to improve MCH outcomes though capacity building and empowerment, and refrains from providing service delivery; however, another emerging theme was the reliance on the programme to provide essential tangible resources such as Mama Kits (local delivery kits which include soap, razors, gloves, plastic sheeting, etc.), medicine, and bed nets. Though women did value the knowledge imparted to them by the programme, it was often undermined by the lack of resources they had available, which one participant stated, should be supplied to them:
“For sure, if they only give us advise, when the time reaches you don’t have anything in your hands.” (FGD7, 41, user)
Lack of understanding of the roles and responsibilities of the CHWs among participants
Both users and non-users were familiar with the AIM-Health programme active in their communities and due to the high visibility of this programme, came to expect that CHWs would visit the households of pregnant women. However, participants frequently had misinformation, or a lack of information, on the roles and responsibilities of the CHWs. This included misunderstanding that the CHWs act as volunteers and that they receive no financial compensation for their work, the expectation that CHWs could dispense medication as part of their remit, and for non-users, where to report a non-active CHW in their community. The research team was frequently asked to explain the role of CHWs as well as the programme itself.
The popular view that increased user satisfaction can contribute to health service effectiveness should provide sufficient reasoning for both its investigation and its function as a method of early programme evaluation -. Despite this however, user perceptions of health services in LMICs are rarely addressed and even more rarely incorporated into programme iteration procedures . Our results provide additional support for the importance of involving end-users and project beneficiaries from the very early stages of project implementation. The feedback obtained from such an evaluation not only serves to highlight a programme’s strengths and weaknesses at what, in theory, is a more flexible point in the project but can also identify unexpected consequences, which can be further monitored throughout the project cycle.
Highlighted in Alma-Ata as a key principle of Primary Health Care for All, community participation must be continuously promoted to encourage active and reflective participation throughout the project cycle, especially in the early stages of project implementation. It is our view that community-based approaches for MCH require participant involvement and feedback from the early stages of programme implementation and that user perspectives of implementation should be a standard M&E reporting indicators. Making such evaluations standard allows for a more timely reflection on user views of service during the implementation phase  while promoting the involvement of the community throughout the project cycle. Though the appropriate timing of such evaluations is debated within the literature , this paper argues that when attempting to understand users’ perspectives of programmes, evaluations can and should be conducted soon after programme deployment. Though similar evaluations will differ depending on their appropriateness to context, they require flexibility to capture the complex interactions that occur within health promotion programmes . Our research demonstrates that such evaluations should occur early after the initialisation of programmes, as potential users often require little exposure before forming an opinion that can have lasting effects on long-term use.
In Uganda, as in many other LMICs, living within lower wealth quintiles and not having attended any secondary school makes one more vulnerable to poor maternal health care practices ,. The reported ever use of ANC services in the study population is comparable to that of the Ministry of Health 2010 statistics of 95% . A more concerning demographic is the reported travel time to a health facility, an average of 75 min. The fact that the majority of nearby health centres do not provide ANC services indicates that families must travel even farther for MCH care and highlights the importance of CHWs or other outreach services in the area.
This early-stage evaluation also revealed that not all intended programme beneficiaries were being reached by CHWs. Potential participants were required to be residing within the programme area and to be pregnant and/or breastfeeding; they were not required to be programme recipients. The possible situation of non-active CHWs in some programme areas is concerning and may indicate a potentially larger issue with the overall programme structure. In this study, all non-users were located within the same sub-district, and these results were fed back to World Vision promptly as to address this service gap and provide care for all women residing in the area. Though CHW programmes have been identified as a means to increase health coverage and have been shown to be effective in MCH programmes -,, prevalent issues common to many CHW programmes that can impact their success and sustainability are well documented and should be considered during initiative development -.
Strengths of the programme
The CHWs as part of the AIM-Health programme were trusted in North Rukiga, with women expressing confidence in the knowledge and messages relayed to them. Users and non-users indicated that community health workers could bring positive changes to the health behaviours of both women and men. Future formative evaluations of similar programmes may benefit from conducting additional FGDs with male, orhusband’, participants. Recognising the importance of a CHW’s work, as well as the perception that without the CHWs individuals may have more negative health behaviours, highlights the value placed on such work. Viewing your health worker as knowledgeable can increase one’s satisfaction and utilisation of a service . Specifically in Uganda, a user’s perception of their health worker and their training has been shown to increase acceptance and use of service -. The positive attributes ascribed to the health workers and the programme’s initiatives as well as their overall acceptability should assist in keeping women enrolled in the AIM-Health programme, and following the teachings of CHWs during the recommended timeframe, which it can be inferred, will subsequently increase health outcomes for both mother and child.
The pride the women took in their health knowledge may aid in empowering women, making them more confident in their ability to practice health behaviours and hence more willing to adhere to the programme messages and more likely to encourage other women to participate. Understanding how one’s pride in their own abilities, perhaps as a form of increased self-efficacy, may impact health-seeking behaviours or even knowledge dissemination, and what effects this may have on MCH outcomes should be further explored. Though it is unclear if the levels of knowledge on MCH practices differ between users and non-users, there was a clear difference in the level of confidence exuded by the two different groups when it came to exhibiting their health knowledge. Similarly, differences were observed in the women’s willingness to express and share the knowledge they had acquired thus far as part of the programme. The relationship between the CHWs and the women may change the dynamics of the CHW’s perceived responsibilities, as some participants appeared to ask CHWs to weigh in on issues beyond the scope of their training and task description.
The strong relationship with the CHWs as identified by enrolled women transcends the typical patient-provider role and can serve to impact satisfaction of service and subsequent utilisation of services . These strong relationships may be attributed to a positive perception of the CHWs due to their perceived knowledge, their ability to relate to CHWs, and the frequency of visits (minimum ten) that encourages the development of a relationship. On the positive side, this may increase job satisfaction and retention for CHWs, whereby feeling appreciated and having respect and commendation from community members are important motivators for CHWs . On the other hand, CHWs may sometimes feel obliged to provide additional resources, such as food or transport costs, to households out of their own pocket. Not only can they often not afford these resources themselves, but the expectation that they would provide for these extends far beyond their remit. Lastly, there is a chance that CHWs may be asked to become involved in a situation they are ill prepared to handle, such as the case of domestic abuse (something that some of the women indicated they had discussed with the CHW).
The trend of women ascribing increased partner involvement in MCH to the CHWs was an unanticipated outcome of the health programme and further highlights the importance of conducting an early-stage evaluation assessment. Such issues can now be followed up, researched or strengthened through changing programme activities that will more actively and consistently support this important aspect of delivering MCH services. Male involvement in MCH is recognised as beneficial to promoting positive health behaviours and outcomes ,. This is consistent with past research in Uganda, which found that males who were knowledgeable on ANC, and who had received information from health workers, were more likely to accompany their partner to ANC visits . Though recognised as an area with a dearth of research ,, other studies conducted in Uganda have called for more attention to be paid to the role of male head of households (as primary decision maker for the family) and how they may mediate access to MCH services . Taken together, male involvement in MCH services in Uganda remains an important factor to ensure the successful implementation of MCH interventions.
Community ownership of CHW programmes is widely recognised as an indispensable element to success and sustainability  and also has implications for CHW attrition and motivation by increasing support by and accountability to the community . Both users and non-users of the MCH programme seemed unaware that the community health workers are accountable to the communities in which they serve, who are in turn, are ultimately responsible for their selection or exclusion. A lack of ownership is not only in opposition to the fundamentals of community-based health programmes but it also threatens its impacts and future directions and the ability to deliver quality community-level care at scale. The choice of words that many women used to describe the CHWs such as “doctors” may signify several things, including the trust in the CHWs’ abilities and also, more worryingly, a level of power or authority. Further investigation into whether this term was intended to ascribe faith in CHWs, or was developed out of systems of hierarchy and power, and how this may impact relationships could further complement this work. Dependency on the programme to provide tangible products, for example, food, clothes and birthing kits to assist in facilitating a healthy pregnancy, childbirth and early child development years, is concerning as it can impact the sustainability of the programme and is not within the programme’s current mandate. This level of dependency and the expectation of product deliverables also highlight a lack of programme awareness and allude to barriers women face in fulfilling the recommendations by the CHW.
Even when women recognised the importance and expressed a desire to adhere to the programme’s recommendations, they are often unable to fulfil them. Women frequently cited barriers they faced in achieving healthy MCH practices, which are consistent with common barriers sited throughout the literature -. These barriers, at the clinical, outreach, and community levels, have been shown to reduce MCH care uptake throughout the continuum of care for mother and child. More extensive reviews of such barriers are covered elsewhere in the literature ,. Though many of these barriers could be addressed through supply-side instruments, as this programme is not meant to be a service delivery mechanism, other strategies need to be developed to reduce the influence these have on women and children’s capabilities to access care. Inadequate programme understanding or information contributes to a lack of ownership and dependency on the programme. In addition, other areas of concern might be reduced if community members’ expectations were modified through a better understanding of the programme structure, intentions and activities.
The findings from this research and their timely dissemination to both the Ministry of Health and World Vision allowed for changes in implementation to reflect the views of programme users. A policy report resulting from this research was distributed to nine other AIM-Health Programmes running in five sub-Saharan African countries, so project implementers were aware of the challenges for this particular context. As previously stated, an area identified with non-active CHWs was targeted for further training or selection of new CHWs and the importance of spousal involvement in the counselling sessions was emphasised throughout refresher training for CHWs and in local health centres in North Rukiga. Follow-up with World Vision staff has noted that spousal involvement in ANC and PNC continues to increase in the region. Several of the main barriers that women identified throughout the research were also considered for implementation. This includes the plan for motorcycle ambulances for pregnant woman and children, which has been initiated to assist in reducing barriers of transportation in emergency situations, as well as increasing essential delivery materials in health centres. The programme has also scaled up their nutrition interventions, recognising under-nutrition as a major contributor to morbidity and mortality in the area, as well as the multiple barriers families face in providing adequate nutrition for children under-2.
The authors acknowledge that this study is not without limitations. As with most interview methods, there is a possibility of social desirability and respondent bias. Though appropriate measures were taken to ensure the authenticity of the data, some may have been lost in translation as IDIs and FGD were not conducted in the primary researcher’s native language and required translation. Characteristics of both the lead researcher and translator may be considered a limitation, as the researcher was not from the area and participants may have interpreted both as coming from positions of power.
This article reports on the investigation of user perceptions of a newly initiated MCH programme in southwest Uganda that delivers evidence-based interventions through community health workers. Our results exemplify how early-phase evaluations can serve to highlight programme strengths and expectations and also unveil previously unanticipated consequences of programme implementation that merit further investigation, follow-up, and programmatic support. This research contributes to the existing literature on formative evaluations and the need to further develop these methodologies for conducting such evaluations to ensure their findings are applied towards the iteration of existing health programmes and policy. Furthermore, this study supports the need for early-stage formative evaluations of beneficiaries’ perceptions towards health programmes as an appropriate community participatory and evaluation method that should be integrated into standard M&E reporting processes, an approach that is too frequently omitted in development projects. The reasons for the observed lack of community participation might lie, in part, in the increasing pressure on NGOs and their implementing partners to prove their value-add, resulting in their accountability focus being more towards donors rather than beneficiaries, and from a concern that funding implementation research may divert funding from NGO core programming activities. Amidst the static of rigid programming, complicated reporting structures, imposed donor agency ideologies, and frequent top-down approaches to “development”, the community’s voice is too often the last one heard.
BG, EM, and NMT all participated in the design of the research study. GM assisted in research facilitation and provided contextual support. BG collected and analysed the data. BG and FV drafted and made revisions to the manuscript. All authors read and approved the final manuscript.
Access to Infant and Maternal Health
behavioural change communication
community health worker
low- and middle-income country
maternal and child health
Millennium Development Goal
monitoring and evaluation
maternal mortality ratio
Ministry of Health
Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, Kimmel B, Sharp ND, Smith JL: The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med. 2006, 21 (S2): S1-S8. 10.1007/s11606-006-0267-9.
Wallace CM, Legro MW: Using formative evaluation in an implementation project to increase vaccination rates in high-risk veterans: QUERI series. Implement Sci. 2008, 3: 22-10.1186/1748-5908-3-22.
Lahai MA: Participatory Evaluation: Perceptions of Local People on Long-Term Impact of Development Interventions in Northern Ghana. 2009, University of Amsterdam, Amsterdam
Wallace T, Bornstein L, Chapman J: The Aid Chain: Coercion and Commitment in Development NGOs. 2006, IT Publishing, London
Gould N: Integrating qualitative evidence in practice guideline development: meeting the challenge of evidence-based practice for social work. Qual Soc Work. 2010, 9: 93-109. 10.1177/1473325009355623.
Lawn JE, Cousens S, Zupan J: 4 million neonatal deaths: When? Where? Why?. Lancet. 2005, 365: 891-900. 10.1016/S0140-6736(05)71048-5.
Lozano R, Wang H, Foreman KJ, Rajaratnam JK, Naghavi M, Marcus JR, Dwyer-Lindgren L, Lofgren KT, Phillips D, Atkinson C, Lopez AD, Murray CJL: Progress towards millennium development goals 4 and 5 on maternal and child mortality: an updated systematic analysis. Lancet. 2011, 378: 1139-1165. 10.1016/S0140-6736(11)61337-8.
Rajaratnam JK, Marcus JR, Flaxman AD, Wang H, Levin-Rector A, Dwyer L, Costa M, Lopez AD, Murray CJL: Neonatal, postneonatal, childhood, and under-5 mortality for 187 countries, 1970-2010: a systematic analysis of progress towards millennium development goal 4. Lancet. 2010, 375: 1988-2008. 10.1016/S0140-6736(10)60703-9.
Ministry of Finance (MOF): Millennium Development Goals Report for Uganda 2010; Special theme: Accelerating Progress Towards Improving Maternal Health. Kampala: Ministry of Finance; 2010.
Ayiko R, Antai D, Kulane A: Trends and determinants of under-five mortality in Uganda. East Afr J Public Health. 2009, 6: 136-140.
Uganda Bureau of Statistics (UBOS) and Macro International Inc: Uganda Demographic and Health Survey 2011. Calverton: UBOS and Macro International Inc; 2012.
Ministry of Health: The Operational Framework for Nutrition in the National Child Survival Strategy. Kampala: Ministry of Health, Government of Uganda; 2009.
Ministry of Health: Health Sector Strategic and Investment Plan: Promoting People’s Health to Enhance Socio-economic Development 2010/11-2014/15. Kampala: Ministry of Health, Government of Uganda; 2010.
Uganda Bureau of Statistics (UBOS) and Macro International Inc: Uganda Demographic and Health Survey 2006. Calverton: UBOS and Macro International Inc; 2007.
Campbell OM, Graham WJ: Strategies for reducing maternal mortality: getting on with what works. Lancet. 2006, 368: 1284-1299. 10.1016/S0140-6736(06)69381-1.
Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS: How many child deaths can we prevent this year?. Lancet. 2003, 362: 65-71. 10.1016/S0140-6736(03)13811-1.
Althabe F, Bergel E, Cafferata ML, Gibbons L, Ciapponi A, Aleman A, Colantonio L, Palacios AR: Strategies for improving the quality of health care in maternal and child health in low- and middle-income countries: an overview of systematic reviews. Paediatr Perinat Epidemiol. 2008, 22 (Suppl 1): 42-60. 10.1111/j.1365-3016.2007.00912.x.
Darmstadt GL, Bhutta ZA, Cousens S, Adam T, Walker N, De Bernis L: Evidence-based, cost-effective interventions: how many newborn babies can we save?. Lancet. 2005, 365: 977-988. 10.1016/S0140-6736(05)71088-6.
ACNM: Life-Saving Skills Manual for Midwives. 4th edition. Silver Spring: American College of Nurse-Midwives; 2011.
Haines A, Sanders D, Lehmann U, Rowe AK, Lawn JE, Jan S, Walker DG, Bhutta Z: Achieving child survival goals: potential contribution of community health workers. Lancet. 2007, 369: 2121-2131. 10.1016/S0140-6736(07)60325-0.
MacLachlan M, Carr SC, McAuliffe E: The Aid Triangle: Recognizing the Human Dynamics of Dominance, Justice and Identity. 2010, Zed, London
Easterly W: The White Man’s Burden: Why the West’s Efforts to Aid the Rest Have Done so Much Ill and So Little Good. 2006, Penguin, London
Björkman M, Scensson J: Power to the people: evidence from a randomized field experiment on a community-based monitoring in Uganda. Q J Econ. 2009, 124 (2): 735-769. 10.1162/qjec.2009.124.2.735.
Chen L, Evans T, Anand S, Boufford JI, Brown H, Chowdhury M, Cueto M, Dare L, Dussault G, Elzinga G: Human resources for health: overcoming the crisis. Lancet. 2004, 364: 1984-1990. 10.1016/S0140-6736(04)17482-5.
Task Force for Scaling Up Education and Training for Health Workers: Scaling Up, Saving Lives. Geneva: Global Health Workforce Alliance/World Health Organization; 2008.
World Health Organization: Strengthening the Performance of Community of Health Workers in Primary Health Care. Geneva: WHO; 1989.
The Earth Institute: One Million Community Health Workers: Technical Task Force Report. New York: Columbia University; 2011.
Lehmann U, Sanders D: Community Health Workers: What Do We Know About Them? The State of the Evidence on Programmes, Activities, Costs and Impact on Health Outcomes of Using Community Health Workers. 2007, World Health Organization: Evidence and Information for Policy, Geneva
Bhutta ZA, Lassi ZS, Pariyo G, Huicho L: Global Experience of Community Health Workers for Delivery of Health Related Millennium Development Goals: A Systematic Review, Country Case Studies, and Recommendations for Integration into National Health Systems. 2010, WHO/GHWA, Geneva
Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X, Van Wyk BE, Odgaard-Jensen J, Johansen M, Aja GN, Zwarenstein M: Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev. 2010, 3: CD004015-
Christopher JB, Le May A, Lewin S, Ross DA: Thirty years after Alma-Ata: a systematic review of the impact of community health workers delivering curative interventions against malaria, pneumonia and diarrhoea on child mortality and morbidity in sub-Saharan Africa. Hum Resour Health. 2011, 9: 27-10.1186/1478-4491-9-27.
Lewin S, Dick J, Pond P, Zwarenstein M, Aja G, Van Wyk B, Bosch-Capblanch X, Patrick M: Lay health workers in primary and community health care. Cochrane Database Syst Rev. 2005, 1: CD004015-
Gilmore B, McAuliffe E: Effectiveness of community health working in delivering preventive interventions for maternal and child health in low- and middle- income countries: a systematic review. BMC Public Health. 2013, 13: 847-10.1186/1471-2458-13-847. doi:10.1186/1471-2458-13-847
Paul S: Community Participation in Development Projects: The World Bank Experience. 1987, World Bank, Washington
Rifkin SB: Lessons from community participation in health programmes: a review of the post Alma-Ata experience. Int Health. 2009, 1: 31-37. 10.1016/j.inhe.2009.02.001.
Cummings FH: Role of participation in the evaluation and implementation of development projects. Knowledge Pol. 1997, 10: 24-33. 10.1007/BF02912484.
Michener VJ: The participatory approach: contradiction and co-option in Burkina Faso. World Dev. 1998, 26: 2105-2118. 10.1016/S0305-750X(98)00112-0.
MacQueen KM, McLellan E, Metzger DS, Kegeles S, Strauss RP, Scotti R, Blanchard L, Trotter RT: What is community? An evidence-based definition for participatory public health. Am J Public Health. 2001, 91: 1929-1938. 10.2105/AJPH.91.12.1929.
Whitmore E: Empowerment in program evaluation: a case example. Can Social Work Rev. 1990, 7: 215-229.
McDonald D: Developing guidelines to enhance the evaluation of overseas development projects. Evaluat Progr Planning. 1999, 22: 163-174. 10.1016/S0149-7189(99)00009-9.
Nichols L: Participatory program planning: including program participants and evaluators. Evaluat Progr Planning. 2002, 25: 1-14. 10.1016/S0149-7189(01)00044-1.
Parkinson S: Power and perceptions in participatory monitoring and evaluation. Evaluat Progr Planning. 2009, 32: 229-237. 10.1016/j.evalprogplan.2009.03.003.
Atkinson S, Haran D: Individual and district scale determinants of users’ satisfaction with primary health care in developing countries. Soc Sci Med. 2005, 60: 501-513. 10.1016/j.socscimed.2004.05.019.
McPake B: User charges for health services in developing countries: a review of the economic literature. Soc Sci Med. 1993, 36: 1397-1405. 10.1016/0277-9536(93)90382-E.
Bernhart MH, Wiadnyana I, Wihardjo H, Pohan I: Patient satisfaction in developing countries. Soc Sci Med. 1999, 48: 989-996. 10.1016/S0277-9536(98)00376-1.
Essential Interventions, Commodities and Guidelines for Reproductive Maternal, Newborn, and Child Health: A Global Review of Key Interventions Related to Reproductive, Maternal, Newborn and Child Health. 2011, The Partnership for Maternal health, Newborn and Child Health and the Aga Khan University, Geneva
Husserl E: Logical Investigations. 1970, Humanities Press, New York
Green J, Thorogood N: Qualitative Methods for Health Research. 2004, Sage, London
Mays N, Pope C: Qualitative research in health care: assessing quality in qualitative research. BMJ. 2000, 320: 50-52. 10.1136/bmj.320.7226.50.
Bradley EH, Curry LA, Devers KJ: Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007, 42: 1758-1772. 10.1111/j.1475-6773.2006.00684.x.
Appleton JV, King L: Constructivism: a naturalistic methodology for nursing inquiry. Adv Nurs Sci. 1997, 20: 13-22. 10.1097/00012272-199712000-00003.
Miles MB, Huberman AM: Qualitative Data Analysis: An Expanded Sourcebook. 1994, Sage, Thousand Oaks
Atkinson S, Medeiros RL: Explanatory models of influences on the construction and expression of user satisfaction. Soc Sci Med. 2009, 68: 2089-2096. 10.1016/j.socscimed.2009.03.042.
WHO: Evaluation in Health Promotion: Principles and Perspectives. Geneva: World Health Organization Regional Publication, European Series; No. 92; 2001.
King ME, Behrman JR: Timing and duration of exposure in evaluations of social programs. World Bank Res Obs. 2009, 24 (1): 55-82. 10.1093/wbro/lkn009.
Springett J: Appropriate approaches to the evaluation of health promotion. Critical Pub: Health. 2001, 11 (2): 139-151. 10.1080/09581590110039856.
Tann CJ, Kizza M, Morison L, Mabey D, Muwanga M, Grosskurth H, Elliot AM: Use of antenatal services and delivery care in Entebbe, Uganda: a community survey. BMC Pregn Childbirth. 2007, 7: 23-10.1186/1471-2393-7-23.
Bhutta ZA, Ahmed T, Black RE, Cousens S, Dewey K, Giugliani E, Haider BA, Kirkwood B, Morris SS, Sachdev H, Shekar M: What works? Interventions for maternal and child undernutrition and survival. Lancet. 2008, 371: 417-440. 10.1016/S0140-6736(07)61693-6.
Rahman SM, Ali NA, Jennings L, Seraji MHR, Mannan I, Shah R, Al-Mahmud AB, Bari S, Hossain D, Das MK, Baquil AH, Arifeen SE, Winch PJ: Factors affecting recruitment and retention of community health workers in a newborn care intervention in Bangladesh. Hum Resour Health. 2010, 8: 12-10.1186/1478-4491-8-12.
Strachan DL, Källander K, Ten Asbroek AH, Kirkwood B, Meek SR, Benton L, Conteh L, Tibenderana J, Hill Z: Interventions to improve motivation and retention of community health workers delivering integrated community case management (iCCM): stakeholder perceptions and priorities. Am J Trop Med Hyg. 2012, 87 (Suppl 5): 111-119. 10.4269/ajtmh.2012.12-0030.
Bhattacharyya K, Winch P, LeBan K, Tien M: Community Health Worker Incentives and Disincentives: How They Affect Motivation, Retention and Sustainability. 2001, BASICS/USAID, Arlington
Hermann K, Van Damme W, Pariyo GW, Schouten E, Assefa Y, Cirera A, Massavon W: Community health workers for ART in sub-Saharan Africa: learning from experience - capitalizing on new opportunities. Hum Resour Health. 2009, 7: 31-10.1186/1478-4491-7-31.
Wild K, Barclay L, Kelly P, Martins N: Birth choices in Timor-Leste: a framework for understanding the use of maternal health services in low resource settings. Soc Sci Med. 2010, 71: 2038-2045. 10.1016/j.socscimed.2010.09.012.
Kiwanuka SN, Ekirapa EK, Peterson S, Okui O, Rahman MH, Peters D, Pariyo GW: Access to and utilisation of health services for the poor in Uganda: a systematic review of available evidence. Trans R Soc Trop Med Hyg. 2008, 102: 1067-1074. 10.1016/j.trstmh.2008.04.023.
Nabbuye-Sekandi J, Makumbi FE, Kasangaki A, Kizza IB, Tugumisirize J, Nshimye E, Mbabali S, Peters DH: Patient satisfaction with services in outpatient clinics at Mulago hospital, Uganda. Int J Qual Health Care. 2011, 23: 516-523. 10.1093/intqhc/mzr040.
Kiguli J, Ekirapa-Kiracho E, Okui O, Mutebi A, MacGregor H, Pariyo GW: Increasing access to quality health care for the poor: community perceptions on quality care in Uganda. Patient Prefer Adherence. 2009, 3: 77-85.
Alio AP, Salihu HM, Kornosky JL, Richman AM, Marty PJ: Feto-infant health and survival: does paternal involvement matter?. Matern Child Health J. 2010, 14: 931-937. 10.1007/s10995-009-0531-9.
Sternberg P, Hubley J: Evaluating men’s involvement as a strategy in sexual and reproductive health promotion. Health Promot Int. 2004, 19: 389-396. 10.1093/heapro/dah312.
Tweheyo R, Konde-Lule J, Tumwesigye NM, Sekandi JN: Male partner attendance of skilled antenatal care in peri-urban Gulu district, Northern Uganda. BMC Pregn Childbirth. 2010, 10: 53-10.1186/1471-2393-10-53.
Ditekemena J, Koole O, Engmann C, Matendo R, Tshefu A, Ryder R, Colebunders R: Determinants of male involvement in maternal and child health services in sub-Saharan Africa: a review. Reprod Health. 2012, 9: 32-10.1186/1742-4755-9-32.
Roth DM, Mbizvo MT: Promoting safe motherhood in the community: the case for strategies that include men. Afr J Reprod Health. 2001, 5: 10-21. 10.2307/3583426.
Vallières F, Hansen A, McAuliffe E, Cassidy EL, Owora P, Kappler S, Gathuru E: Head of household education level as a factor influencing whether delivery takes place in the presence of a skilled birth attendant in Busia, Uganda: a cross-sectional household study. BMC Pregn Childbirth. 2013, 13: 1-1. 10.1186/1471-2393-13-48.
Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, Pielemeier NR, Mills A, Evans T: Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet. 2004, 364: 900-906. 10.1016/S0140-6736(04)16987-0.
Kerber KJ, De Graft-Johnson JE, Bhutta ZA, Okong P, Starrs A, Lawn JE: Continuum of care for maternal, newborn, and child health: from slogan to service delivery. Lancet. 2007, 370: 1358-1369. 10.1016/S0140-6736(07)61578-5.
Kinney MV, Kerber KJ, Black RE, Cohen B, Nkrumah F, Coovadia H, Nampala PM, Lawn JE: Sub-Saharan Africa’s mothers, newborns, and children: where and why do they die?. PLoS Med. 2010, 7: e1000294-10.1371/journal.pmed.1000294.
The authors would like to thank World Vision Uganda and World Vision Ireland for their support and participation towards this research. Specifically, we would like to thank Sheila Garry and Magnus Conteh for their active contribution to the review of the original research report. We would also like to highly recognise the World Vision Uganda team from Kabale, especially James Muhumuza, and would like to thank Gloria Nahabwe for her wonderful translation work and help throughout the entirety of the data collection process. The team is grateful for the support of Irish Aid and the people of Ireland for funding this work through the “Access to Infant and Maternal Health Programme”, a collaborative initiative with World Vision Ireland and the Centre for Global Health, Trinity College Dublin. We especially want to thank the study’s participants and their families for taking the time to discuss with us and for being so open and welcoming to this work. Lastly, we would like to thank the two anonymous reviewers who provided important and relevant revisions to the original manuscript.
The authors declare that they have no competing interests.
About this article
Cite this article
Gilmore, B., Vallières, F., McAuliffe, E. et al. The last one heard: the importance of an early-stage participatory evaluation for programme implementation. Implementation Sci 9, 137 (2014). https://doi.org/10.1186/s13012-014-0137-5
- Maternal and child health
- Community health workers
- Programme implementation
- Project cycle