This article has Open Peer Review reports available.
A mixed methods protocol to evaluate the effect and cost-effectiveness of an Integrated electronic Diagnosis Approach (IeDA) for the management of childhood illnesses at primary health facilities in Burkina Faso
© Blanchet et al. 2016
Received: 15 April 2016
Accepted: 23 July 2016
Published: 4 August 2016
Burkina Faso introduced the Integrated Management of Childhood Illnesses (IMCI) strategy in 2003. However, an evaluation conducted in 2013 found that only 28 % of children were assessed for three danger signs as recommended by IMCI, and only 15 % of children were correctly classified. About 30 % of children were correctly prescribed with an antibiotic for suspected pneumonia or oral rehydration salts (ORS) for diarrhoea, and 40 % were correctly referred. Recent advances in information and communication technologies (ICT) and use of electronic clinical protocols hold the potential to transform healthcare delivery in low-income countries. However, no evidence is available on the effect of ICT on adherence to IMCI. This paper describes the research protocol of a mixed methods study that aims to measure the effect of the Integrated electronic Diagnosis Approach innovation (an electronic IMCI protocol provided to nurses) in two regions of Burkina Faso.
The study combines a stepped-wedge trial, a realistic evaluation and an economic study in order to capture the effect of the innovation after its introduction on the level of adherence, cost and acceptability.
The main challenge is to interconnect the three substudies. In integrating outcome, process and cost data, we focus on three key questions: (i) How does the effectiveness and the cost of the intervention vary by type of health worker and type of health centre? (ii) What is the impact of changes in the content, coverage and quality of the IeDA intervention on adherence and cost-effectiveness? (iii) What mechanisms of change (including costs) might explain the relationship between the IeDA intervention and adherence?
Despite a large reduction in under-5 child mortality (from 180 per 1000 live births in 1990 to 83 per 1000 live births in 2015), sub-Saharan Africa failed to reach the Millennium Development Goal 4 target of 60 deaths per 1000 live births . In 1999, the World Health Organization (WHO) developed the Integrated Management of Childhood Illness (IMCI) strategy . This strategy provides an algorithm to guide health workers through a systematic clinical assessment of sick children with the aim of improving the diagnostic classification and the treatment of these children [3–5] and hence reducing mortality [4, 6].
However, effective implementation of IMCI is often constrained by poor adherence to the guidelines [7–9]. Previous studies have reported that adherence to the guidelines decreases over time due to inadequate initial training, shortage of staff and insufficient supervision [10, 11]. Takada et al.  have noted that healthcare workers typically find the IMCI chart booklet burdensome and try to work from memory, resulting in a decrease in quality of care. In addition, healthcare workers may omit sections resulting in incomplete assessments [4, 13]. Chaudhary  demonstrated that the adherence of health workers improved with supervision. However, regular supervision of health workers after training is often lacking [8, 13], partly due to the lack of resources.
Burkina Faso introduced the IMCI strategy in 2003. However, an evaluation conducted in 2013 found that only 22 % of nurses working in primary care facilities had been trained in IMCI . Only 28 % of children were assessed for three danger signs as recommended by IMCI, and only 15 % of children were correctly classified . About 30 % of children were correctly prescribed with an antibiotic for suspected pneumonia or oral rehydration salts (ORS) for diarrhoea, and 40 % were correctly referred .
Recent advances in information and communication technologies (ICT) holds the potential to transform healthcare delivery in low-income countries [15–17]. However, no evidence is available on the effect of ICT on adherence to IMCI. Small-scale studies conducted in Tanzania found ICT to be well accepted by healthcare workers during IMCI consultations [7, 16]. However, the experience with using such technology on a large scale is limited.
In 2010, Terre des Hommes (TdH), a Swiss non-governmental organisation, together with the Ministry of Health (MoH), launched the Integrated electronic Diagnosis Approach (IeDA) intervention with the objective of improving adherence to IMCI guidelines in public primary health centres in two regions of Burkina Faso. In this paper, we present the design of a mixed methods evaluation of this intervention.
Burkina Faso is composed of 13 regions and 63 health districts. The public health system is characterised by a three-tier service structure: (i) at the first level are the districts with 1535 health centres (Centre de Santé et de Promotion Sociale (CSPS)) and the 104 district hospitals (Centre Médical avec Antenne Chirurgicale (CMA)), (ii) at the next level are the nine regional hospitals (Centre Hospitalier Régional (CHR)), and (iii) finally, the third level is comprised of the three national teaching hospitals (Centre Hospitalier Universitaire (CHU)) .
The IMCI approach was implemented only at the first level of the pyramid, i.e. in health centres. These facilities deliver a minimum package of services defined by the Ministry of Health comprising both preventive (e.g. vaccinations, antenatal care, health education, and promotion of proper nutrition, hygiene and safe water) and curative measures (e.g. treatment of common illnesses, minor surgery, supply of essential medicine, maternal and child consultations). CSPSs are governed by a management committee (Comite de gestion) composed of members of the community. The district health management team is in charge of supervising CSPSs and analysing routine data collected in them .
The Electronic Register of Consultations and the IeDA intervention
The ‘electronic register of consultations’ or ‘Registre Electronique de Consultations’ (REC) in French was designed in 2010 by TdH . The REC software, based on the CommCare software language, is installed on the open access CommCare platform developed by Dimagi . The REC guides health workers through the IMCI algorithm. By doing so, it aims to improve adherence of nurses to the clinical protocol and to provide the local health district and the MoH with routine data on the management of childhood illnesses. The first versions of the REC were piloted in 2011 and 2012 in 52 primary health facilities located in two districts in the Nord region and perceived by 90 % of users (nurses) as being a supportive tool during consultations . An additional pilot district, Yako, was added in the Boucle du Mouhoun region in 2014 and 2015. Following the pilot phase, the MoH requested TdH to expand the implementation of the REC to the remaining health districts of both regions.
Development and implementation of the improved versions of the REC in all primary facilities of the two regions (district level).
Provision, with the Ministry of Health, of a 7-day training course on IMCI guidelines including 2 days on the use of the REC to all primary health workers working in each district to be covered (district level).
Development of a quality assurance mechanism through which each district and health centre is encouraged to find appropriate solutions in response to their local needs (district and health centre levels).
Supervise every month every health centre benefiting from the intervention and support health district authorities in their annual supervision of health centres (health centre level).
Development of a health information system based on the data collected through the REC and fed back to the district managers and heads of health centres (district and health centre levels).
The evaluation methods
A stepped-wedge trial to evaluate the effect of IeDA on the adherence to IMCI guidelines in primary health facilities
A cost-effectiveness analysis (CEA) to assess the value for money of the delivery of IeDA
A realistic evaluation to understand the implementation process, the mechanisms by which the IeDA intervention leads to change and to identify factors that may affect these mechanisms at the health centre and community levels
These three studies are taking place in a total of eight health districts across the Nord and Boucle du Mouhoun regions, with the three districts where TdH piloted the REC excluded from the evaluation.
The stepped-wedge trial
The intention was that the intervention would eventually be delivered to all primary health centres in all eight districts. Also, during the period preceding the intervention (i.e. in year 1 step 1), a first round of data collection would be conducted in the eight districts to provide pre-intervention measurements in all districts.
The aim of the trial is to determine whether the IeDA intervention increases adherence to the IMCI algorithm and hence improves classification, referral, prescription and counselling during under-5 child consultations in health centres. This will be determined by observation of child consultations followed by a repeat consultation by study staff in selected CSPS.
Degree of adherence to IMCI guidelines, measured as a quantitative variable
Correct disease classification and prescription, measured as a binary variable
Correct identification of danger signs
Correct classification of children
Prescription of the correct medicine
Correct referral or hospitalisation
Correct counselling delivered to child carer
Selection of primary health centres
The evaluation is being conducted in ten randomly chosen primary health centres per health district. The list of CSPSs selected for the evaluation has not been communicated to the implementing agency to reduce the likelihood that they target these health centres for more intensive support. Ten CSPS per district was chosen to give sufficient contextual variation, while minimising logistical difficulties and ensuring that some CSPS in each district were not selected to allow for selection concealment. Only primary health centres with staff trained in IMCI were considered for selection, and all hospitals were excluded. The intervention effect may vary by health centre workload, and so, the selection was stratified on the 2013 annual under-5 consultations caseload, as provided by the MoH. For the 337 CSPSs with staff trained in IMCI in the eight districts under evaluation, the annual caseload was transformed into a daily rate by dividing it by 261 days (i.e. the number of working days). Out of the 337 CSPSs, 13 were excluded, either because of missing caseload data (six CSPS) or because of a zero recorded caseload (seven CSPS). Out of the remaining 324 CSPSs, a further 54 CSPSs were excluded because they had caseloads of less than three cases per working day, which would have limited the number of consultations that could be observed in these facilities. Although these 54 CSPSs represented 17 % of the remaining 324 CSPSs, they only represent 4 % of the total caseload in 2013. In each district, five CSPSs were then randomly selected among those with fewer than seven cases per working day and five CSPSs were randomly selected among those with seven cases or more per working day (on average). Seven cases were chosen for the cutoff as this ensured there were at least nine CSPSs in each of the two strata in each district. Data will be collected in the same ten CSPSs per district during each step of the stepped-wedge trial during a 1- to 2-day visit (see the ‘Data collection’ section below).
Sample size calculation
Ten health centres in eight districts will be visited every step.
A harmonic mean of ten children will present at each health facility over the course of a 1- to 2-day visit.
Sampling ten children per facility per round carries a design effect of 2.
The coefficient of variation between clusters is 0.3.
Then, there is 90 % power to detect an increase in the correct diagnosis and treatment from 25 to 33 %. If there is only a harmonic mean of four children at each health centre, there will be 98 % power to show an increase from 25 to 40 %.
An international organisation will be implementing a performance-based financing (PBF) intervention in four of the eight districts during the study, which may influence the performance of the health workers. In addition, two of the eight districts will act as ‘control’ clusters for their study, and so, it would be better if the IeDA intervention does not start in both these clusters before starting in any other cluster.
An international donor is prepared to provide support for two of the five districts in the Boucle du Mouhoun region, but they needed to start doing this soon.
There needs to be some balance in the randomisation order across the two regions.
There must be one PBF intervention district and one PBF non-intervention district (either a control district or one excluded from the study) in each pair of randomised districts, i.e. in the first and second districts to receive the intervention, the third and fourth districts, the fifth and sixth districts and the seventh and eight districts. The difference in the average order of the PBF intervention districts and the non-intervention districts must be less than one. The first three districts must not include both PBF control districts.
At least one of the first two districts must be in the Boucle du Mouhoun region.
At least one of the first three districts and at least one of the last three districts must be from the Nord region.
Applying these criteria to the 40,320 possible randomisations resulted in 4224 acceptable allocations. For these 4224, no district was in a given position in the random order for less than 6.9 %, or more than 17.0 %, of acceptable allocations (292 and 720 acceptable allocations, respectively). If two districts were always next to each other in the order, then they should be treated as one district in the analysis, but among the 4224 acceptable allocations, the least and most common pairing of districts occurred in 6.8 and 42.8 % of acceptable allocations, respectively. We therefore judged that the restricted randomisation had not resulted in an overly constrained set of possible allocations and one of the 4224 acceptable allocations was chosen at random. This was done in Stata v13, by sorting the 4224 acceptable allocations on a variable containing random numbers generated by the ‘uniform()’ command and then choosing the allocation with the smallest random number; this was done by a statistical co-investigator (JL). This allocation will be communicated to the TdH and the MoH gradually, so they only know the next two districts to receive the intervention.
As outcome measures, the trial is assessing the adherence to IMCI guidelines and the correctness of the illness classification, prescription, referral (if necessary) and counselling.
Each selected CSPS is visited for 1 or 2 days at each ‘step’ of the trial by a team of two staff from the evaluation team. One member of staff observes all under-5 children consultations performed by the local health workers while the other reassesses children after their consultation with the local health worker. Our teams notify the head of the health centre the afternoon of the day before our visit.
The first member of the evaluation team records information regarding each consultation in a detailed observation form. Three observation forms were designed, one per age category defined by the IMCI guidelines (i.e. 0–7 days, 7 days–2 months, 2 months–5 years) and pre-tested to include all questions or tasks that a healthcare worker should perform when following the IMCI guidelines. Each observation form is divided into sections corresponding to the different sections in the IMCI protocol for each age category. For instance, the observation form to observe consultations of children from 2 months to 5 years old includes sections for the assessment of danger signs: cough or fast/difficult breathing, diarrhoea, fever, ear problems, anaemia, nutritional status, HIV, immunisation and vitamin A uptake, referral, treatment and counselling. Observations are passive, and the observer never intervenes during the consultation. Information recorded in the observation form also includes whether the nurse referred to the IMCI chart booklet during the consultation or used the REC (if the district benefited from the intervention), as would be expected if IMCI was being implemented correctly. If the health centre has more than one nurse conducting child consultations, the observer follows the first child who is called for consultation.
Following observation of a consultation, the second member of the evaluation team, expert in IMCI, re-assesses the child independently using the REC to provide a ‘gold standard’ against which to compare local health workers’ classifications and treatment decisions. In case of discrepancies between the health worker and the IMCI expert, the final management decision is determined following discussions between the two of them.
In addition to observations and repeat assessments, a questionnaire, based on the WHO Service Availability and Readiness Assessment questionnaire , was developed to document the context in which the trial is conducted. This questionnaire is completed at each visit to a CSPS and captures information on the resources available for management of childhood illnesses according to the IMCI guidelines.
Two evaluation teams (each comprised of two members) were recruited for data collection. They had all previously been trained in IMCI by the Ministry of Health and had at least 5 years of experience of IMCI in their health centre. In addition, they underwent 2 weeks of training on the study methods, provided by the investigators.
The stepped-wedge design can be analysed in different ways . In the primary analysis, each step will be analysed using methods appropriate for a parallel cluster randomised trial, using district-level summaries of the outcome to calculate a difference measure. These difference measures will be combined across steps using inverse variance weighting. Hypothesis testing will be based on bootstrapping or a permutation test.
In the secondary analysis, the method described by Hussey and Hughes  will be used. As the outcome is likely to be relatively common, the risk difference will be modelled using generalised estimating equations at the level of the child consultation, with fixed effects for intervention versus control phase, time step and district and clustering at the level of the health centre.
Potential imbalances at the level of the child and the health centre between intervention and control phases will be investigated and any imbalances will be adjusted for. A priori, adjustment will also be made for district-level summaries of the outcome at baseline.
A detailed statistical analysis plan will be prepared before the end of data collection.
The aim of this sub-study is to estimate the value for money of implementing IeDA compared to current practice (implementing standard IMCI, based on the paper-based IMCI protocol) in the diagnosis of under-5 children in primary healthcare centres in Burkina Faso. The CEA, embedded within the IeDA stepped-wedge trial, will estimate both the cost per consultation and the incremental cost per child correctly managed associated with delivering IeDA compared to delivering standard IMCI. The perspective for the CEA is that of the healthcare provider, i.e. the MoH. For both standard IMCI and IeDA, costing will be undertaken using a micro-costing approach . For each of the two interventions, the main components of the cost per consultation are as follows: (i) programme costs, i.e. costs associated with the implementation of the intervention that are incurred at the administrative level outside the point of delivery of healthcare, and (ii) patient costs or costs associated with the implementation of the intervention which are incurred at the point of delivery of healthcare . To obtain an adequate estimate of the cost per consultation for each of the two interventions, programme costs will be allocated to patient costs using established cost allocation procedures .
The cost per consultation of IeDA and standard IMCI will be estimated using an observational before-and-after study design in the district of Ouahigouya, one of the districts included in the stepped-wedge trial. In this district, the costs per consultation of standard IMCI will be estimated during the 6-month period prior to IeDA implementation. The cost per consultation for IeDA will be estimated during the 6-month period after IeDA implementation.
For both arms, all resource consumption affecting programme costs at health centre and district level will be measured prospectively using surveys. This data will be supplemented with data from health centre and district accounts. All resource consumption affecting programme costs at national level will be measured retrospectively via interviews with agents who have participated in the implementation of both interventions (mostly MoH officers and, in the case of IeDA, also TdH coordinators). Resource consumption affecting patient costs will be measured using data recorded in the REC for every consultation in the ten Ouahigouya health centres selected for the stepped-wedge trial which is monitored in the period between 6 months before and 6 months after IeDA implementation. For each of the two interventions, the number of consultations monitored in the Ouahigouya health centres during this period is scheduled to be 80. Data collected in the REC for each patient consultation includes (i) duration of the consultation and (ii) medications prescribed in the consultation. An additional form will be added to the REC to measure consumables used during each patient consultation. To minimise bias in the measurement of resource consumption due to seasonal effects on healthcare delivery, data on resource consumption from all the consultations monitored in the stepped-wedge trial (whether standard IMCI or IeDA) 3 months before and 3 months after IeDA implementation in Ouahigouya will be used in the estimation of patient costs. Once resource consumption is measured, costs will be calculated using unit prices for each resource consumed. Depending on the specific type of resource, unit prices will be obtained from MoH databases, MoH price lists or supplier catalogues.
Regression analysis will be used to estimate the mean difference in costs per consultation between IeDA and standard IMCI. Cost-effectiveness modelling will be used to assess the incremental cost-effectiveness of IeDA versus standard IMCI.
For Pawson and Tilley , issues of context and mechanism are crucial elements to consider in any realist evaluation as they help to explain ‘what works, for whom and in what circumstances’. For these authors, ‘what works’ is not of itself a helpful question as: ‘programs work (have successful “outcomes”) only insofar as they introduce the appropriate ideas and opportunities (“mechanisms”) to groups in the appropriate social and cultural conditions (“contexts”)’. Focusing on the realist dimensions enables us to examine the particular significance of ‘mechanism’ and ‘context’ more closely.
We will use longitudinal research, using a case study design , to enable us to understand the impact of key features or mechanisms for IeDA users and the context in which adherence to protocols is achieved (or not) over time.
In the case studies, the task of the research team will be to identify different context and mechanism configurations. These case studies will be purposively selected in relation to the performance of the health centre (in terms of adherence to the protocol) and the capacity of health centre staff to reflect on their experience. Informed by data from the stepped-wedge trial (high-performing and low-performing health centres), new cases identified from the list of health centres randomly selected in the trial will be added to our purposive sample. Four intervention health centres with high performance (positive cases) and four intervention health centres (deviant cases) with low performance will be selected.
Non-participant and participant observation of nursing and multi-disciplinary activities related to the use of IeDA (e.g. supervision from district managers, relation with pharmacist)
Post-observation interviews guided by issues arising from observations
Key stakeholder interviews exploring views in general about the use, influences on use and impact of IeDA. We will record the interviews and code and analyse them using the software NVivo
Interviews with patients about their experience of IeDA
Survey questionnaire administered to REC users to understand their experience using the tool and perceptions on the benefits or constraints of REC
Review of relevant documentation (e.g. reports, statistics)
Field notes written during and after each visit
In the realist evaluation sub-study, the analysis will consist of identifying patterns and variables in one case study  through a ‘pattern matching’ process  and verifying the presence of the same pattern in other case studies where it was expected to obtain the same pattern, what Yin called ‘theoretical replication’ [38, p. 116]. Cross comparisons will also be made with deviant cases where other patterns are expected to be found for predicted reasons.
To enable comparisons, data analysed in the trial on the level of adherence, resource availability and health facility size and compared across areas and health centres will be used to identity well-performing heath centres and poorly performing health centres. This will allow qualitative cross-case analysis taking into account similarities and differences between health centres.
With the IeDA study, we aim for a holistic evaluation in which outcome, process and cost data are integrated in the analysis of the findings to maximise our capacity to understand the effect of the introduction of a complex intervention into a specific context.
In integrating outcome, process and cost data, we focus on three key questions: (i) How does the effectiveness and the cost of the intervention vary by type of health worker and type of health centre? (ii) What is the impact of changes in the content, coverage and quality of the IeDA intervention on adherence and cost-effectiveness? (iii) What mechanisms of change (including costs) might explain the relationship between the IeDA intervention and adherence?
To explore whether more intensive implementation of the IeDA intervention can generate better adherence outcomes, data collected on implementation parameters (e.g. fidelity and dose) will be used to categorise and rank health centres. We will calculate the correlation between outcome and the fidelity and dose of the IeDA intervention.
In addition, process and cost data will be integrated into the trial analysis to identify new hypotheses on factors influencing the performance of health centres. Two factors that may play an important role in performance are (1) the characteristics of health workers (e.g. their experience in IMCI, the time allocated to each consultation) and (2) the characteristics of health centres (e.g. team structure, leadership, resources available).
The Ethics Committee of the Ministry of Health of Burkina Faso and the Ethics Review Committee at the London School of Hygiene & Tropical Medicine have approved the trial. Risks and burdens to the patients and the nurse as a consequence of the study are minimal.
Every nurse at the health centre level who will be observed will be informed about the study and the procedures and written consent will be requested. Carers of children will give written informed consent to provide authorisation for one of the researchers to observe the consultation conducted by the nurse and for the other researcher to conduct a repeat consultation.
An information sheet in the local language will be read to all participants (carers who accompany children, nurses, MoH Officials, NGO and United Nations Officers). Participants will be encouraged to ask questions about the research. If consent is given, they will be asked to sign either their name or, if unable to do so, provide their verbal assent and print their thumb print, on the attached consent form.
The IeDA study is funded by Terre des Hommes, Switzerland.
Availability of data and materials
The authors make, upon request, associated protocols and data collection tools available to the readers.
KB wrote the first draft of the paper, and every co-author contributed to the revised version of the paper, which was finalised by KB, JL, FPM and SC. All authors read and approved the final manuscript.
Conflict of interest declared as the funds available for the study from the international donor are transferred through the implementing agency Terre des Hommes.
Consent for publication
Participants to the study provided written consent to publish the results of the study.
Ethics approval and consent to participate
The IeDA study was approved both by the Ethics Review Committee of the Ministry of Health in Burkina Faso and the London School of Hygiene & Tropical Medicine. Every participant to the study provided written consent to the researchers.
The trial has been registered at clinicaltrials.gov (NCT02341469) and the trial protocol can be requested from the corresponding author.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- United Nations Inter-agency Group for Child Mortality Estimation, Levels and trends in child mortality—Report 2015. WHO. New York; 2015.Google Scholar
- Black RE, Morris SS, Bryce J. Where and why are 10 million children dying every year? Lancet. 2003;361(9376):2226–34.View ArticlePubMedGoogle Scholar
- Boss LP, Toole MJ, Yip R. Assessments of mortality, morbidity, and nutritional status in Somalia during the 1991-1992 famine. Recommendations for standardization of methods. JAMA. 1994;272(5):371–6.View ArticlePubMedGoogle Scholar
- Rakha M, et al. Does implementation of the IMCI strategy have an impact on child mortality? A retrospective analysis of routine data from Egypt. BMJ Open. 2013;3(1). doi:10.1136/bmjopen-2012-001852.
- Nguyen D, et al. Does Integrated Management of Childhood Illness (IMCI) training improve the skills of health workers? A systematic review and meta-analysis. PLoS One. 2013;8(6):e66030.View ArticlePubMedPubMed CentralGoogle Scholar
- Jones G, et al. How many child deaths can we prevent this year? Lancet. 2003;362(9377):65–71.View ArticlePubMedGoogle Scholar
- Derenzi B, et al. e-IMCI: improving pediatric health care in low-income countries. Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008. Florence, Italy: April 5-10 2008.Google Scholar
- Horwood C, et al. Experiences of training and implementation of the integrated management of childhood illness (IMCI) in South Africa: a qualitative evaluation of the IMCI case management training course. Soc Sci Med. 2009;70(2):313–20.View ArticlePubMedGoogle Scholar
- Bryce J, Victora CG, Black RE. The unfinished agenda in child survival. Lancet. 2013. 382(9897): 1049-59.Google Scholar
- Chaudhary N, Mohanty P, Sharma M. Integrated management of childhood illness (IMCI) follow-up of basic health workers. Indian J Paediatr. 2005;72(9):735–9.View ArticleGoogle Scholar
- Rowe A, et al. The rise and fall of supervision in a project designed to strengthen supervision of Integrated Management of Childhood Illness in Benin. Health Policy Plan. 2010;25(2):125–34.View ArticlePubMedGoogle Scholar
- Takada S, Oudavong B, Kurolwa C. The successes and challenges of the IMCI training course in Lao PDR. Southeast Asian J Trop Med Public Health. 2007;38(1):178–87.Google Scholar
- Mugala N, et al. Barriers to implementation of the HIV guidelines in the IMCI algorithm among IMCI trained health workers in Zambia. BMC Paediatrics. 2010;10(93).Google Scholar
- Kouanda L, Baguiya A. Evaluation de la qualite des soins prodigues aux enfants de moins de cinq (05) ans dans les formations sanitaires des regions du nord et du centre-nord du Burkina Faso. Burkina Faso: WHO, UNICEF, UNFPA; 2013.Google Scholar
- Peters T. Using cell phones to improve treatment in Cape Town tuberculosis patients: an evaluation. Cape Town: Bridges Organisation; 2005.Google Scholar
- Mitchell M, et al. Perceived improvement in integrated management of childhood illness implementation through use of mobile technology: qualitative evidence from a pilot study in Tanzania. J Health Commun. 2012;17(sup1):118–27.View ArticlePubMedGoogle Scholar
- Chib A, van Velthoven MH, Car J. mHealth adoption in low-resource environments: a review of the use of mobile healthcare in developing countries. J Health Commun. 2015;20(1):4–34.View ArticlePubMedGoogle Scholar
- DSS/DGISS. Enquete SARA. Ouagadougou, Burkina Faso: Ministere de la Sante; 2012.Google Scholar
- Ministere de la Sante. Plan national de developpement sanitaire 2011-2020. Ouagadougou: Ministere de la Sante Burkina Faso; 2011.Google Scholar
- Deflaux G. Registre Electronique de Consultations (REC)—cahier des charges. Ouagadougou, Burkina Faso: Terre des Hommes; 2010.Google Scholar
- Deflaux G, et al. Computerization of medical consultation for children under five years of age in rural areas of Burkina Faso. In: Bolay J-C et al., editors. Technologies for sustainable development. Swistzerland: Springer; 2014.Google Scholar
- Yameogo P, et al. REC: Résultats d’utilisation d’un logiciel informatique pendant la consultation des enfants de moins de 5 ans selon la stratégie PCIME. Burkina Faso: Terre des hommes; 2011.Google Scholar
- Beard E, et al. Stepped wedge randomised controlled trials: systematic review of studies published between 2010 and 2014. Trials. 2015;16(1):1–14.View ArticleGoogle Scholar
- Hargreaves JR, et al. Five questions to consider before conducting a stepped wedge trial. Trials. 2015;16(1):1–4.View ArticleGoogle Scholar
- Rowe AK, et al. A multifaceted intervention to improve health worker adherence to integrated management of childhood illness guidelines in Benin. Am J Public Health. 2009;99(5):837–46.View ArticlePubMedPubMed CentralGoogle Scholar
- Rowe AK, et al. Evaluating health worker performance in Benin using the simulated client method with real children. Implement Sci. 2012;7:95.View ArticlePubMedPubMed CentralGoogle Scholar
- Horwood C, et al. An evaluation of the quality of IMCI assessments among IMCI trained health workers in South Africa. PLoS One. 2009;4(6):e5937.View ArticlePubMedPubMed CentralGoogle Scholar
- Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–91.View ArticlePubMedGoogle Scholar
- Copas AJ, et al. Designing a stepped wedge trial: three main designs, carry-over effects and randomisation approaches. Trials. 2015;16(1):1–12.View ArticleGoogle Scholar
- World Health Organisation, Service Availability and Readiness Assessment (SARA): an annual monitoring system for service delivery - Reference Manual. Service availability and readiness assessment (SARA): an annual monitoring system for service delivery—reference manual. Geneva: Health Statistics and Information Systems, WHO; 2013.Google Scholar
- Davey C, et al. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358.View ArticlePubMedPubMed CentralGoogle Scholar
- Drummond, et al. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press; 2015.Google Scholar
- Johns BEA. Programme costs in the economic evaluation of health interventions. Cost Effectiveness Resour Allocation. 2003;1(1):1–10.Google Scholar
- Pawson R, Tilley N. Realistic evaluation. Thirdth ed. London: Sage Publications; 1997.Google Scholar
- Yin R. Case study research: design and methods. London: Sage Publications; 2003.Google Scholar
- Blanchet K, James P. How to do (or not to do)…a social network analysis in health systems research. Health Policy Plan. 2012;27:438–46.View ArticlePubMedGoogle Scholar
- Blanchet K, James P. The role of social networks in the governance of health systems: the case of eye care systems in Ghana. Health Policy Plan. 2013;28(2):143–56.View ArticlePubMedGoogle Scholar
- Yin R. Case study research: design and methods, Applied Social Research Methods Series. London: Sage Publications; 2003.Google Scholar
- Trochim W. Outcome pattern matching and program theory. Eval Program Plan. 1989;12:355–66.View ArticleGoogle Scholar