Ideal and reality: do countries adopt and follow recommended procedures in comprehensive multiyear planning guidelines for national immunization programmes?
Implementation Science volume 10, Article number: 48 (2015)
Meticulous steps and procedures are proposed in planning guidelines for the development of comprehensive multiyear plans for national immunization programmes. However, we know very little about whether the real-life experience of those who adopt these guidelines involves following these procedures as expected. Are these steps and procedures followed in practice? We examined the adoption and usage of the guidelines in planning national immunization programmes and assessed whether the recommendations in these guidelines are applied as consistently as intended.
We gathered information from the national comprehensive multiyear plans developed by 77 low-income countries. For each of the 11 components, we examined how each country applied the four recommended steps of situation analysis, problem prioritization, selection of interventions, and selection of indicators. We then conducted an analysis to determine the patterns of alignment of the comprehensive multiyear plans with those four recommended planning steps.
Within the first 3 years following publication of the guidelines, 66 (86%) countries used the tool to develop their comprehensive multiyear plans. The funding conditions attached to the use of these guidelines appeared to influence their rapid adoption and usage. Overall, only 33 (43%) countries fully applied all four recommended planning steps of the guidelines.
Adoption and usage of the guidelines for the development of comprehensive multiyear plans for national immunization programmes were rapid. However, our findings show substantial variation between the proposed planning ideals set out in the guidelines and actual use in practice. A better understanding of factors that influence how recommendations in public health guidelines are applied in practice could contribute to improvements in guidelines design. It could also help adjust strategies used to introduce them into public health programmes, with the ultimate goal of a greater health impact.
Whether considered as a staged or an iterative process, public health policy and practice involve the use of guidelines in planning and implementing interventions [1,2]. These instruments provide the principles, procedures, and techniques used in virtually every phase of the implementation circle, from initiation to formulation, implementation, and finally, evaluation of the outcomes and impacts [3-7]. Even though systematic effort to convert basic research knowledge to evidence-based guidelines and practice is relatively new in public health compared to clinical medicine [8,9], there is increasing body of scientific literature and research on the use of basic public health science and new discoveries in public health policies and practice guidelines [9-15]. Additionally, instruments such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework are increasingly being used by the public health community to appraise the quality of evidence for public health recommendations and production of public health guidelines [16,17].
The production of public health guidelines and their effective use are two complementary aspects. We know much about the types and number of public health guidelines being produced. For example, the World Health Organization (WHO) alone produces at least 20 public health guidelines annually. These guidelines include the global framework for immunization monitoring and surveillance , the updated guidelines for evaluating public health surveillance systems , the guidelines for intensified tuberculosis case-finding and isoniazid preventive therapy for people living with HIV in resource-constrained settings , a comprehensive multiyear planning tool for immunization programmes , and the United Nations Children’s Fund (UNICEF) cholera toolkit , among others. However, there is very little information on whether or how these tools are actually used in public health practice in countries with varying political, economic, and social systems .
The question of whether these tools are applied as consistently as intended is not often addressed in public health research. Despite this gap in knowledge, the public health community generates and disseminates public health guidelines without, in most instances, critically assessing them and building on the experience gained. Sometimes, the guidelines being released or promoted by different public health agencies are more or less the same, except for minor differences in terminology and certain details. An assessment of how recommendations and procedures of public guidelines are followed in public health programmes is necessary if the instruments are to add value to the health of populations [24,25].
Policy context for the study
Guidelines for the development of comprehensive multiyear plans (cMYPs) for national immunization programmes were developed by the WHO, UNICEF, and partners to support countries in implementing the Global Immunization Vision and Strategy (GIVS). The GIVS is a global immunizational policy framework that was introduced in 2005 in order to enhance the impact of immunization programmes and further reduce infant mortality due to vaccine-preventable disease, particularly in low-income countries [26,27]. These guidelines represent a new approach to immunization planning, guided by the need to simplify and harmonize a large number of immunization-planning activities at the national level. They aim to assist countries in streamlining the immunization planning process at the national level into a single comprehensive and budgeted plan and to minimize the duplication of efforts and transaction costs for countries and partners .
The guidelines are based on the rational model of planning [28,29], which follows a logical process beginning with situation analysis, followed by ranking and prioritization of the problems identified, evaluation and selection of possible interventions, and assessment of performance-monitoring indicators and milestones [21,30,31]. The guidelines have been in use since 2005, and the WHO and other partners have played an active role in facilitating their adoption and use through training workshops [32-34] and technical assistance. Since the instrument was introduced, the GAVI Alliance (Gavi) has required countries that applied for specific windows of funding to support their application with a cMYP developed based on the guidelines . The gross national income (GNI) per capita threshold that was set to establish eligibility for Gavi financial support was US$1,000 between 2000 and 2010; thereafter, it was adjusted to US$1,500 .
In this study, we examined the patterns of adoption and alignment of the cMYPs with the recommendations and procedures of the guidelines in national immunization programmes. The purpose of our study is to partially fill the gap in knowledge between recommendations and procedures of public health guidelines and whether they are followed among the target audience in public health practice.
A document review was conducted on 77 countries that had adopted the guidelines for the development of cMYPs for national immunization programmes. The WHO financing and planning database  was used to identify countries that have developed at least one cMYP based on the guidelines. This database is a repository of copies of cMYPs developed by countries and shared with the WHO. As of December 2012, 77 countries had developed and submitted a copy of at least one cMYP, including 54 that had developed a second cMYP. In total, 77 copies of the first cMYPs were retrieved from the database for review and data extraction.
The detailed data that were extracted from the cMYPs included information on how the planning steps proposed by the guidelines were applied by the countries in each of the 11 national immunization planning components identified. Those planning components include the following 1) vaccine supply and stock management, 2) cold chain, 3) human resources, 4) vaccination implementation, 5) immunization safety, 6) adverse events following immunization (AEFI), 7) waste management, 8) surveillance and laboratory analysis, 9) coordination and supervision, 10) immunization promotion, and 11) government funding.
A descriptive analysis was performed using the extracted data to determine adoption patterns and the extent to which countries followed the recommended planning steps in the guidelines, as the tool was used by different countries in the development of the cMYPs. Alignment of the cMYPs with the recommended planning steps was assessed based on inclusion or omission of the steps, as well as inclusion of a minimum set of information for the included steps. A step was considered followed if a cMYP included the step and also, for the first (situation analysis) step, at least one performance gap was included; for the second (problem prioritization) step, at least one national immunization programme priority was specified; for the third (intervention selection) step, at least one national immunization programme intervention was specified; and for a chosen intervention in the fourth (indicator selection) step, at least one performance indicator was specified.
Among the 77 countries studied here, there was a rapid increase in the number of countries that adopted the guidelines and developed their first cMYPs based on those guidelines, from 4 (5%) in 2005 to 66 (86%) by the end of 2007. From 2008, 54 (70%) countries that reached the end of the time period covered by their first cMYP developed a second one; only one country developed a second cMYP in 2008 and by 2011, a total of 54 countries had developed a second cMYP using the guidelines (Figure 1).
Since 2005, Gavi-eligible countries have been required to support their applications for specific windows of funding with a cMYP developed using the guidelines. As a result, all 72 (94%) countries that received Gavi support during the period studied developed a cMYP based on the guidelines. As Gavi eligibility is based on GNI per capita, the majority of countries that used the guidelines were low-income countries. Another three countries (4% of those studied) used the guidelines to develop a cMYP, although they were not eligible for Gavi support; those countries include Namibia, Botswana, and Swaziland. The WHO’s African region, with a high proportion of Gavi-eligible, low-income countries, accounted for the largest numbers of countries studied (N = 40, 52%) (Table 1), followed by Southeast Asia (N = 9, 12%), the Eastern Mediterranean (N = 8, 10%), Europe (N = 7, 10%), the Western Pacific (N = 7, 10%), and the Americas (N = 6, 8%). The European, Western Pacific, and American regions, including predominantly high-income countries, had the lowest number of countries that used the guidelines to develop a cMYP.
We examined how rigorously the four planning steps in the guidelines have been applied by the countries that developed a cMYP (Table 2). The first step, situation analysis, was the most followed among the 77 countries that used the guidelines; 70% to 94% of the countries applied the situation analysis step in all 11 planning components of the national immunization programme. The second planning step, problem prioritization, was the second most applied with 65% to 90% of the 77 countries applying the step in 10 of the 11 planning components. In third position was the third planning step, intervention selection, which showed 48% to 88% of the 77 countries applying the step in 10 of the 11 planning components. The fourth planning step, indicator selection, had the lowest percentage of countries that aligned their plans with the step, with 0% to 7% of the countries applying the step in 10 of the 11 planning components.
The extent to which the four planning steps in the guidelines were followed in the cMYPs developed by the 77 countries was further examined based on individual planning components; 3 categories were identified among the 11 planning components (Table 2). The first category consisted of four components with the highest proportion of countries (>60%) that followed all four planning steps, which included the cold chain, human resources, vaccination implementation, and vaccine supply and stock management planning. Vaccination implementation had the highest number of countries that followed all four planning steps in the development of their cMYPs in this category, with 60 (78%) countries applying all four planning steps. The human resources planning component had the second highest number of countries that followed all four planning steps, with 58 (75%) countries applying all four steps, followed by the cold chain planning component, with 55 (71%) countries applying all four planning steps. Finally, 54 (70%) countries applied all four steps in the vaccine supply and stock management planning component in this category.
The second category consisted of four planning components with 30% to 60% of the countries applying all four planning steps of the guidelines. These planning components included government funding, immunization safety, waste management, and AEFI monitoring planning. Immunization safety had the highest number of countries that followed all four planning steps in this category, with 44 (57%) countries applying all four planning steps, followed by government funding, with 33 (43%) countries applying all four planning steps, and waste management, with 31(40%) countries applying all four planning steps. AEFI monitoring was in last position in this group, with 25 (32%) countries applying all four planning steps.
Finally, the third category contained three planning components with <30% of the countries following all four planning steps. These components included surveillance and laboratory analysis, immunization promotion, and coordination and supervision planning. Eight (10%) countries applied all four planning steps of surveillance and laboratory analysis. None of the countries proposed intervention indicators, the fourth planning step, for immunization promotion, or addressed the coordination and supervision planning components in this category.
The main findings show that a total of 77 countries have adopted and used the guidelines for the development of cMYPs for their national immunization programmes. There was rapid adoption of the instrument, with 66 (86%) countries doing so within the first 3 years of its introduction. Use of the guidelines appeared to be sustained, as 54 (70%) countries had already used the instrument to develop a second cMYP to cover a subsequent time period.
Among the countries that adopted the guidelines, 72 (94%) were low-income countries, with a per capita income not exceeding US$1,000. The African region accounted for the highest percentage of countries that used the guidelines (52%), followed by Southeast Asia (12%). Adoption of the guidelines was minimal among countries with higher incomes, given that only five countries with a per capita income > US$1,000 adopted the guidelines and that none of the high-income countries adopted the guidelines.
The findings that we have presented regarding the patterns of adoption and use of the guidelines suggest that the technical merits of the instrument alone were not the only considerations in a country’s decision to adopt and use the guidelines; other factors, such as economic status and donor funding, and particularly the linkage of an infusion of Gavi donor funds with use of the guidelines in the preparation of national immunization plans, appeared to be influential. Between 2005 and 2010, eligible countries with a per capita income not exceeding US$1,000 were required by Gavi to support their funding applications with cMYPs based on the guidelines, which may explain the increase in adoption mainly among countries with the lowest incomes. Furthermore, certain funding windows were only available within a specific period of time, and the rush to secure funds before the windows closed most likely contributed to the observed accelerated use of the guidelines among these countries. These observations are consistent with studies that have demonstrated the influence of fiscal context and financial incentives on the accelerated adoption and use of new health practice guidelines [36-38]. Even though no funding was given as an incentive, the linkage of funding to use of the guidelines in preparation of the national immunization plans may have influenced the decisions of countries to adopt the guidelines among countries that needed to apply for these funds.
As we have indicated, despite the rapid pace of adoption and use of the guidelines for cMYP development by many countries, the recommended planning steps in the guidelines were not systematically applied in the 77 cMYPs analyzed. Situation analysis, the first planning step, was the most applied among the countries. Subsequent planning steps were applied to a lesser extent, with indicator selection step (the fourth planning step) being the least respected.
Even though the alignment of the cMYPs with all four planning steps in the guidelines, from situation analysis to indicator selection, was generally satisfactory, much higher levels of omission of the intervention and indicator selection planning steps were observed in most of the planning components. This result appears to suggest that countries do not regard the choice of immunization programme interventions and performance monitoring very highly. Less emphasis on performance monitoring has been observed in other public health programmes in resource-poor settings , which may be explained by the fact that, traditionally, the performance and success of national immunization programmes have primarily been measured in terms of the national vaccination implementation and coverage achieved . This idea is consistent with the findings in this study, which showed that the highest number of countries followed all four planning steps in the vaccination implementation planning component compared with the other ten planning components of national immunization programmes.
An examination of how the planning steps were followed in specific planning components further revealed better alignment with the four planning steps of the planning components that come under direct control of immunization programme managers; over 60% of the countries followed all four steps in these planning component, which included cold chain, human resources, vaccination implementation, and vaccine supply and stock management planning components. Comparatively, alignment of the cMYPs with the planning steps of the guidelines was relatively lower among planning components that are shared with other technical units; these components include surveillance and laboratory analysis, which is a core function of epidemiology and public health laboratories units of the MOH; immunization promotion, which is often shared with health promotion units of the MOH; government funding for immunization, which is a function of the national budgeting office; and waste management, which may depend on the sanitation department. It is not clear whether this result was because immunization programme managers are less committed to, or not as experienced with, planning components that are shared with other technical units. However, planning experience has been shown to influence the quality of planning .
This study reveals some gaps between the recommended ideals of the cMYP guidelines and the application of the recommendations in actual practice. While the main limitation of this analysis was a lack of information on the possible explanatory factors for the observed gaps, ensuring the continued use and consistent application of recommendations and procedures of the guidelines by national immunization programmes calls for further studies to gain a better understanding of the factors contributing to these gaps in the diverse economic, political, and social contexts of the implementing countries. This understanding could contribute to the development of better guidelines, more effective introduction strategies, and ultimately more respect for their recommendations among users for greater public health impact.
Grading of Recommendations Assessment, Development and Evaluation
World Health Organization
United Nations Children’s Fund
Comprehensive Multiyear Plan
Global Immunization Vision and Strategy
gross national income
adverse events following immunization
WHO African region
WHO region of the Americas
WHO Eastern Mediterranean region
WHO European region
WHO South East Asia region
WHO Western Pacific region
Thomson R, Lavender M, Madhok R. Fortnightly review: how to ensure that guidelines are effective. BMJ. 1995;311:237–42.
Woolf SH. Practice guidelines: a new reality in medicine. I Recent developments. Arch Intern Med. 1990;150:1811–8.
Shroff MR, Jones SJ, Frongillo EA, Howlett M. Policy instruments used by states seeking to improve school food environments. Am J Public Health. 2012;102:222–9.
Walt G, Shiffman J, Schneider H, Murray SF, Brugha R, Gilson L. ‘Doing’ health policy analysis: methodological and conceptual reflections and challenges. Health Policy Plan. 2008;23:308–17.
Lohr KN, Eleazer K, Mauskopf J. Health policy issues and applications for evidence-based medicine and clinical practice guidelines. Health Policy. 1998;46:1–19.
Eddy DM. Clinical decision making: from theory to practice. Designing a practice policy. Standards, guidelines, and options. JAMA. 1990;263:3077, 3081, 3084.
Hamid H, Abanilla K, Bauta B, Huang K-Y. Evaluating the WHO assessment instrument for mental health systems by comparing mental health policies in four countries. Bull World Health Organ. 2008;86:467–73.
Anderson ML, Brownson RC, Fullilove MT, Teutsch SM, Novick LF, Fielding J, et al. Evidence-based public health policy and practice: promises and limits. Am J Prev Med. 2005;28:226–30.
Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: an evolving concept. Am J Prev Med. 2004;27:417–21.
Ogilvie D, Craig P, Griffin S, Macintyre S, Wareham NJ. A translational framework for public health research. BMC Public Health. 2009;9:116. doi:10.1186/1471-2458/9/116.
Brownson RC, Royer C, Ewing R, McBride TD. Researchers and policymakers: travellers in parallel universes. Am J Prev Med. 2006;30:164–72.
Briss P. Evidence-based: US road and public-health side of the street. Lancet. 2005;365:828–30.
Victoria CG, Habicht J-P, Bryce J. Evidence-based public health: moving beyond randomized trials. Am J Public Health. 2004;94:400–5.
Greenlund KJ, Giles WH. The prevention research centres program: translating research into public health practice and impact. Am J Prev Med. 2012;43:S91–2.
Murray CJ, Lopez AD. Evidence-based health policy—lessons from the global burden of disease study. Science. 1996;274:740–3.
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–6.
Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system BMC. Health Serv Res. 2005;5:25. doi:10.1186/1472-6963-5-25.
WHO. Global framework for immunization monitoring and surveillance. Geneva: World Health Organization; 2007. http://www.who.int/immunization/documents/WHO_IVB_07.06/en/. Accessed 12 March 2013.
German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN, et al. Updated guidelines for evaluating public health surveillance systems: recommendations from the guidelines working group. MMWR Recomm Rep. 2001;50:1–35. http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm. Accessed 10 September 2012.
WHO. Guidelines for intensified tuberculosis case-finding and isoniazid preventive therapy for people living with HIV in resource-constrained settings. Geneva: World Health Organization; 2011. http://www.who.int/tb/publications/2011/en/. Accessed 12 March 2013.
WHO: WHO-UNICEF. Guidelines for developing a comprehensive multi-year plan (cMYP. Geneva: World Health Organization; 2005. http://www.who.int/immunization/programmes_systems/financing/tools/cmyp/en/. Accessed 10 September 2012.
UNICEF. Cholera toolkit. New York: United Nations Children’s Fund; 2013. http://www.unicef.org/cholera_toolkit/. Accessed 14 June 2014.
Nabyonga Orem J, Bataringaya Wavamunno J, Bakeera SK, Criel B. Do guidelines influence the implementation of health programmes?—Uganda’s experience. Implement Sci. 2012;7:98.
Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci. 2008;3:1.
Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–9.
Kamara L, Lydon P, Bilous J, Vandelaer J, Eggers R, Gacic-Dobo M, et al. Global immunization vision and strategy (GIVS): a mid-term analysis of progress in 50 countries. Health Policy Plan. 2013;28:11–9.
Bilous J, Eggers R, Gasse F, Jarrett S, Lydon P, Magan A, et al. A new global immunisation vision and strategy. Lancet. 2006;367:1464–6.
Camhis M. Planning theory and philosophy. London: Tavistock Publications; 1978.
Frazier TM. The questionable role of statistics in comprehensive health planning. Am J Public Health Nations Health. 1970;60:1701–5.
Forester J. Bounded rationality and the politics of muddling through. Public Adm Rev. 1984;44:23–31.
Kørnøv L, Thissen WAH. Rationality in decision- and policy-making: implications for strategic environmental assessment. Impact Assess Proj Apprais. 2000;18:191–200.
The World Bank. Regional workshop and policy forum on sustainable immunization financing. http://go.worldbank.org/YYILTPSS40. Accessed 14 June 2014
Brenzel L, Politi C. Historical analysis of the comprehensive Multi-year plans in GAVI-Eligible countries. Bill & Melinda Gates Foundation and the World Health Organization. http://apps.who.int/immunization_financing/analysis/en/index.html. 2012. Accessed 05 April 2013
GAVI. Countries eligible for support. http://www.gavialliance.org/Support/Apply/Countries-eligible-for-support/. Accessed 05 April 2013
WHO: WHO. cMYP database. http://www.who.int/immunization/programmes_systems/financing/countries/en. Accessed 12 March 2013
Harting J, Rutten GM, Rutten ST, Kremers SP. A qualitative application of the diffusion of innovations theory to examine determinants of guideline adherence among physical therapists. Phys Ther. 2009;89:221–32.
Rothblatt DN. Rational planning reexamined. J Am Inst Plann. 1971;37:26–37.
Yano EM, Green LW, Glanz K, Ayanian JZ, Mittman BS, Chollette V, et al. Implementation and spread of interventions into the multilevel context of routine practice and policy: implications for the cancer care continuum. J Natl Cancer Inst Monogr. 2012;2012:86–99.
McNamara P. Provider-specific report cards: a tool for health sector accountability in developing countries. Health Policy Plan. 2006;21:101–9.
Lessler J, Metcalf CJ, Grais RF, Luquero FJ, Cummings DA, Grenfell BT. Measuring the performance of vaccination programmes using cross-sectional surveys: a likelihood framework and retrospective analysis. PLoS Med. 2011;8, e1001110.
Rogers E. Diffusion of innovations. 5th ed. New York: Free Press; 2003.
The authors acknowledge the contributions of WHO departments that provided free access to the cMYP database and relevant reports. The views expressed in this study are those of the authors and do not reflect the views of the WHO or the University of Lausanne.
The authors declare that they have no competing interests.
PM contributed to the conception and design of the study, data analysis, and interpretation and led the drafting of the manuscript. PZ, CP, and FP contributed to the conception and design of the study, data interpretation, and participated in drafting of the manuscript. All authors read, reviewed, and approved the final manuscript.
PM is a doctoral student at the Institute of Preventive and Social Medicine, Faculty of Medicine and Biology, University of Lausanne, 10 Route de la Corniche, 1010 Lausanne, Switzerland. PZ is a medical officer of the Department of Essential Medicines and Health Products, World Health Organization Avenue Appia 20 1211 Geneva 27. Switzerland. CP is also a medical officer at the Department of Immunization, Vaccines and Biological, World Health Organization Avenue Appia 20 1211 Geneva 27. Switzerland. FP is a professor/director of the Institute of Preventive and Social Medicine, Faculty of Medicine and Biology, University of Lausanne, 10 Route de la Corniche, 1010 Lausanne, Switzerland.
About this article
Cite this article
Mala, P., Zuber, P., Politi, C. et al. Ideal and reality: do countries adopt and follow recommended procedures in comprehensive multiyear planning guidelines for national immunization programmes?. Implementation Sci 10, 48 (2015). https://doi.org/10.1186/s13012-015-0239-8