A review of methods and tools to assess the implementation of government policies to create healthy food environments for preventing obesity and diet-related non-communicable diseases
Implementation Science volume 11, Article number: 15 (2015)
Policies to create healthy food environments are recognized as critical components of efforts to prevent obesity and diet-related non-communicable diseases. There has not been a systematic review of existing methods and tools used to assess the implementation of these government policies. The purpose of this study was to review methods and tools used for assessing the implementation of government policies to create healthy food environments.
The study conducted a systematic literature search. Multiple databases as well as the grey literature were searched. All study designs and review papers on assessing the implementation of government policies to create healthy food environments were included. A quality assessment of the methods and tools identified from relevant studies was carried out using the following four criteria: comprehensiveness, relevance, generalizability and feasibility. This quality assessment was completed by two independent reviewers.
The review identified 52 studies across different policy areas, levels and settings. Self-administered questionnaires and policy checklists were most commonly applied to assess the extent of policy implementation, whereas semi-structured interviews were most commonly used to evaluate the implementation process. Measures varied widely, with the existence of policy implementation the aspect most commonly assessed. The most frequently identified barriers and facilitators for policy implementation were infrastructure support, resources and stakeholder engagement. The assessment of policy implementation on food environments was usually undertaken in combination with other policy areas, particularly nutrition education and physical activity. Three tools/methods were rated ‘high’ quality and 13 tools/methods received ‘medium’ quality ratings.
Harmonization of the available high-quality methods and tools is needed to ensure that assessment of government policy implementation can be compared across different countries and settings and over time. This will contribute to efforts to increase government accountability for their actions to improve the healthiness of food environments.
Unhealthy food environments, particularly the greater availability of and access to heavily marketed ultra-processed food products , play a significant role in creating unhealthy diets [2, 3] which are one of the major risk factors of obesity and diet-related non-communicable diseases (NCDs) .
Food environments have been defined as the collective physical, economic, policy and sociocultural surroundings, opportunities and conditions that influence people’s food and beverage choices and nutritional status . Food environments are complex and are composed of multiple aspects, including food composition, food labelling, food marketing, food retail, food provision, food prices and food in trade and investment agreements [5, 6]. It is well recognized that efforts to improve the healthiness of food environments will need multi-level, multi-actor engagement .
Globally, there has been limited implementation of government policies to create healthy food environments . Those policies that have been implemented include nutrition information panels, front-of-pack labelling and regulations on the use of nutrition and health claims on foods, provision of healthy foods and nutrition standards in public institutions and other specific settings, economic tools to address food affordability, restricting unhealthy food advertising to children, improving nutritional quality of the whole food supply, incentives and rules to create a healthy retail and food service environment, and zoning laws and policies to place limits on the density or location of quick serve restaurants or other outlets selling mainly unhealthy foods in communities .
In the context of the limited implementation of government policies, there have been recent calls to increase accountability for government action to increase the healthiness of food environments. The assessment and evaluation of policy implementation is increasingly being recognized as a key mechanism for enhancing government accountability [10–13].
High-quality methods and tools are needed to conduct this assessment and evaluation. However, there has not been a systematic review of the quality of existing methods and tools used to assess the implementation of government policies related to food environments.
The objective of this study was to review and assess the quality of existing methods and tools used to assess the government implementation of food environment policies. This will help to inform the choice and harmonization of methods and tools for assessing the implementation of government policies and the implementation process to create healthy food environments for preventing obesity and diet-related NCDs . The harmonization of methods and tools for assessment of policy implementation is considered valuable to compare the extent of policy implementation and barriers/facilitators to policy implementation across countries.
We conducted a systematic search of published and grey literature to review methods and tools used to assess governments’ implementation of policies and actions to create healthy food environments for preventing obesity and diet-related NCDs. The grey literature in this review refers to non-academic publications, including publically available documents such as government reports, newsletters, fact sheets, working papers, technical reports, conference proceedings and policy documents. Recognizing the broad extent of existing literature on assessment and evaluation of policy impacts and outcomes, we focused on assessing the quality of the methods used for assessing the extent of policy implementation and the policy implementation process, including barriers and facilitators to policy implementation.
We first performed a search of peer-reviewed literature using the following electronic databases: MEDLINE (1950 to March 2015), Scopus (1960 to March 2015), Cochrane Library (1898 to March 2015) and Web of Science (1964 to March 2015). Then, reference lists of included articles were searched for additional relevant studies. Websites of international health, food and nutrition organizations (i.e. World Health Organization (WHO) including WHO regions, Food and Agriculture Organization of the United Nations, International Food Policy Research Institute, Organisation for Economic Co-operation and Development, Institute of Medicine (USA)) were hand-searched in order to identify additional publications. Websites pertaining to government organizations related to health, food and agriculture in countries mentioned on the aforementioned websites (i.e. UK, Canada, Australia, New Zealand, Norway, Finland, Scotland, South Africa, Ghana, Thailand and the USA) were also consulted for additional documents.
A search strategy was developed for MEDLINE and revised appropriately for the other databases. The key search terms were based on definitions of different aspects of food environments, developed by the International Network for Food and Obesity/NCDs Research, Monitoring and Action Support (INFORMAS) . These key aspects of food environments include food composition, food labelling, food promotion, food prices, food provision, food retail, food production and food trade and investment. These search keywords were used in combination with other groups of keywords which covered the following: ‘monitoring’ and/or ‘evaluation’ or ‘assessment’, ‘government policy’ and/or ‘government action’, and ‘obesity’ and/or ‘NCDs’. Searches through Medical Subject Headings (MeSH) for MEDLINE were conducted to identify other synonyms for the original keywords to be included in the search strategy.
The following search strategy was developed for MEDLINE: (“Policy”[Mesh]) AND (Public OR Government) AND (environment* OR (“Nutritive Value”[Mesh] OR food composition*) OR (“Food Labeling”[Mesh] OR “Food Labeling”[Mesh]) OR (“Marketing”[Mesh] OR food promotion OR food marketing) OR (food tax* OR beverage tax* OR food subsid* OR food pricing) OR (food retail* OR food availability OR zoning* OR outlet density OR outlet proximity) OR (food provision OR food service) OR (food trade* OR food investment OR food production)) AND (“Evaluation Studies as Topic”[Mesh] OR Monitor* OR benchmark*) AND (obes* OR non-communicable disease* OR noncommunicable disease* OR diabetes OR cancer* OR cardiovascular disease* OR coronary heart disease*).
Potentially relevant papers and documents which met the following criteria were selected by screening the titles and abstracts. The criteria for inclusion were that the study had to (1) assess the existence and/or level of implementation of policies and actions, or the implementation process of policies and actions; (2) cover policy aimed at improving the healthiness of food environments for preventing diet-related NCDs, including their risk factors, such as obesity; (3) cover policy developed by governmental bodies and officials; (4) be written in English and published up until March 2015; and (5) specify the tools used. The full texts of relevant articles for which the relevance could not be determined from the abstract alone were also examined. Studies which only focused on government policies and actions directed at the treatment or management of obesity and diet-related NCDs were excluded.
Quality assessment of methods and tools
The quality and feasibility of methods and tools included in this review were assessed. There are many different sets of criteria for assessing the quality and feasibility of research methods [15–20], but due to the nature of the tools and methods identified in this review (including both quantitative and qualitative methods and highly specific subject matter), no relevant tools were found that could provide a relevant overall assessment of the quality of study tools and methods. This study thus selected the criteria based on a review of the public health and political science literature to determine the assessment criteria most commonly used to assess the quality and feasibility of methods and tools [15–20]. This was supplemented by the authors’ judgement on the applicability of assessment criteria for this study that includes both quantitative and qualitative studies. The following four criteria were considered most relevant to critically assess the quality of the methods and tools used for measuring policy implementation in this context: comprehensiveness, relevance, generalizability and feasibility.
All tools and methods were assessed against these criteria, and the results were combined to form an overall quality rating for each tool/method (refer to Additional file 1 for more details of criteria and standards for quality assessment of the methods used). This quality assessment was completed by two independent reviewers in a two-step process. The first reviewer assessed the quality of all studies, and then, the quality of a 10 % random sample of the reviewed studies was assessed independently by the second reviewer. The 10 % of the study sample size is a common practice for random sampling in many research areas, including literature reviews [21–32]. The two reviewers were in consensus on the quality of all papers in the 10 % sample.
The extensive search of four electronic databases yielded 16,952 articles. After screening for duplicates, titles and abstracts, and assessment of full texts, there were 34 articles that met the study criteria. In addition, seven published reports from the grey literature and 11 papers identified from the references of already included studies were also included. In total, 52 articles were included in the review (Fig. 1).
Of the identified 52 relevant articles, 24 articles focused on assessing the extent of implementation of food environment policies and actions, 14 articles aimed to evaluate the policy implementation process or barriers/facilitators to policy implementation and 14 articles included both. Forty three of the 52 relevant articles were conducted in high-income countries, two were conducted in low- or middle-income countries and seven were carried out across world regions or at a global level (Table 1).
Assessing the extent of implementation of food environment policies
The literature search yielded 24 relevant studies which specifically focused on assessing the extent of the policy implementation by governments and 14 studies which examined the assessment of extent of the implementation together with the evaluation of the implementation process. Most studies (n = 30) were single-country studies, which were conducted in high-income countries, while some (n = 8) were multi-country studies, conducted across world regions or at a global level. Both quantitative methods (e.g. self-administered questionnaires) and qualitative methods (e.g. semi-structured interviews, focus group interviews and document review), or a combination of those, were used to assess the policy implementation by governments; however, quantitative methods were more frequently applied. Online supplementary information (Additional file 2) summarizes the identified studies, including implementation measures, key features of the methods and tools used to assess the food environment policy implementation and the overall quality rating of each tool. More detailed results from the quality assessment are provided in Table 2.
Policy areas, levels and settings
A small number of studies (n = 9) specifically measured the implementation of food environment policies and actions [5, 33–40]. Most of the studies assessed these policies as part of a range of policies to prevent obesity and NCDs. Many studies (68 %) centred on the implementation of policies addressing food environments in combination with either food and nutrition education or physical activity policy or both [4, 41–65].
The studies encompassed eight common policy domains related to food environments  namely food composition (n = 11), food labelling (n = 11), food promotion (n = 17), food prices (n = 10), food provision (n = 29), food retail (n = 12), food production (n = 6) and food trade and investment (n = 3). Several studies examined multiple domains of food environment policy in one study rather than a single domain only. The implementation of food provision policies appeared to be the most frequently examined (75 %) [5, 8, 33, 35, 37–52, 54, 55, 57–60, 62, 64, 65], followed by food promotion [5, 8, 34, 38–42, 44, 46, 48, 52, 56, 62, 64–66] and food retail [5, 39–42, 44, 56, 57, 62, 64–66].
Several studies (n = 20) focused on food environment policies at national or federal level [4, 34–37, 45, 47, 49–52, 54–56, 58–61, 66, 67], followed by state or provincial level [33, 43, 44, 46, 48, 53, 63] and local government level [38, 57]. Some studies (n = 9) assessed the policies across different levels, from national or state to local levels [5, 8, 39–42, 62, 64, 65].
Implementation of policies in school settings was the most commonly assessed (n = 15) [33, 35, 37, 43, 47–52, 54, 55, 58–60] while some studies focused on policies in various settings such as workplaces, schools, hospitals, childcare centres and communities [5, 8, 38–42, 44, 53, 62, 64, 65]. Some studies did not specify a particular setting.
Aspects measured by the study
The identified studies assessed various measures of policy implementation. Thirty studies investigated the existence of policy implementation [4, 5, 8, 33, 34, 36–40, 42, 44–46, 48–56, 58, 60–65]. Some studies (n = 15) also investigated the level or degree of policy implementation, but different methods were used to classify the different levels of policy implementation [5, 8, 35, 36, 39–41, 43, 47, 48, 57, 59, 62, 63, 66]. For example, the INFORMAS Healthy Food Environment Policy Index (Food-EPI) categorized the degree of implementation of food environment policies compared to international best practice into five levels (from 1 = less than 20 % implementation to 5 = 80–100 % implementation) [5, 39, 40]. Canada’s Report Card on Healthy Food Environments and Nutrition for Children graded the level of implementation of food environment policies and actions from A through F: grade ‘A’ where the policies and actions were successfully implemented so as to affect a large majority of children and youth and ‘F’ where the policies and actions were implemented so as to affect very few children and youth . The School Wellness Assessment Tool grouped the level of policy implementation into ‘fully in place’, ‘partially in place’, ‘under development’ and ‘not in place’ [47, 54]. Other policy implementation measures examined included implementation coverage (low, medium, high) of the policy or policies in targeted settings .
Methods and tools used to assess policy implementation
The methods used to assess policy implementation varied across and within studies. Of all the studies, 16 used quantitative, ten used qualitative and ten used mixed methods to assess policy implementation. Two studies reported indicators used only. Most of the methods used were self-administered questionnaires which were specifically designed for multi-domain food environment policies and actions combined with other NCD-related policies. The questionnaires required either a written response, typically with specific response options (e.g. yes/no and rating scale) or a verbal response, typically through telephone communications.
Several studies used one method only, such as self-administered questionnaires among stakeholders [8, 33, 37, 38, 44, 46–50, 52, 56–58, 66, 67], reviews of secondary data and public documents [4, 61] or semi-structured interviews with key stakeholders [43, 45, 59]. The studies which applied mixed methods conducted a quantitative questionnaire survey in combination with document review, interviews or focus groups, or a combination of these [5, 35, 39–41, 54, 55, 60, 62, 63]. Some studies combined different types of qualitative methods, such as document review, stakeholders’ interviews, expert consultation and school observations [34, 36, 42, 51, 53].
In total, 17 quantitative tools and 15 qualitative tools were used for assessing the policy implementation through the use of indicators, items or indexes. Key elements of most of the tools include uses of policy indicators or indexes and numerical scoring system especially numerical rating scale and yes/no formats and involvement of government officials in the studies.
Only three tools used received a ‘high’ quality. They were the INFORMAS Food-EPI [5, 39, 40], the WHO Global Nutrition Policy Review questionnaire tool  and thematic matrix for guiding the interviews for an evaluation of the Norwegian Action Plan on Nutrition . Eleven studies were rated as ‘medium’ quality [4, 34, 36, 41, 44, 46, 47, 54, 56, 60, 67], and 18 studies were rated as ‘low’ quality [33, 35, 37, 38, 43, 45, 48–53, 55, 57–59, 61, 63]. Quality ratings could not be completed for four studies due to absence or insufficiency of data on some individual criteria [62, 64–66].
Evaluating the implementation process of food environment policies
The literature search yielded 14 relevant studies which specifically focused on evaluating the policy implementation process and 14 studies which examined the evaluation of the implementation process, together with the assessment of the extent of the implementation. Out of all the identified studies, 27 single studies were conducted in high-income countries and one multi-country study was performed at a global level with WHO Member States. Nineteen studies applied qualitative methods (e.g. semi-structured interviews, focus groups and document review), four used quantitative methods (e.g. self-administered questionnaires) and five used mixed approaches. Online supplementary information (Additional file 3) summarizes the studies including policy implementation measures, key features of methods and tools used to assess barriers and facilitators to policy implementation, and the overall quality rating of each tool. More detailed results from the quality assessment are provided in Table 2.
Policy areas, levels and settings
Almost all studies (N = 22) examined the implementation process of multi-domain policies which addressed the promotion of healthy food environments together with other areas related to obesity and NCD prevention. Most often, food environment policies were assessed together with either nutrition education or physical activity policy, or both [42, 43, 45, 47, 49, 52, 53, 55, 59, 60, 62, 68–77].
The studies encompassed seven common domains related to food environments namely food composition (N = 5), food labelling (N = 6), food promotion (N = 7), food prices (N = 5), food provision (N = 22), food retail (N = 6) and food production (N = 4). Several studies examined multiple domains of food environment policy in one study rather than a single domain only. Food provision (79 %) was the most common food environment domain examined [33, 35, 42, 43, 45, 47, 49, 52, 55, 59, 60, 62, 68, 70, 72–74, 76–80], followed by food promotion [42, 52, 62, 71, 79–81].
Fourteen studies focused specifically at national or federal-level policy [35, 36, 45, 47, 49, 52, 55, 59, 60, 68, 71, 74, 76, 79], seven focused at state level [33, 53, 73, 77, 78, 80, 81] and five focused at subnational level, i.e. provincial , county , district  and local [69, 75] levels. Two assessed the policies at both national and local/subnational levels [42, 62].
School was the most targeted setting to assess the policy implementation process (61 %) [33, 35, 43, 47, 52, 55, 59, 60, 68, 70–74, 76, 78, 81] while some studies (18 %) focused on policies in various settings in a single study such as workplaces, schools, hospitals, childcare centres and communities [42, 53, 62, 69, 80]. One study examined specifically at community level . Some studies did not specify a particular setting [36, 75, 77, 79, 81].
Aspects measured by the study
The identified studies mainly investigated the factors (barriers and facilitators) impacting policy implementation. Out of 28 studies, 24 focused on the investigation of barriers and facilitators of policy implementation while four focussed on specific issues or problems, such as infrastructure and resource support, stakeholder engagement, and monitoring and evaluation mechanism for implementation [42, 53, 71, 77]. Key issues which were frequently observed as barriers or facilitators of policy implementation were support for infrastructure and/or resources such as financial and human resources [33, 35, 36, 42, 47, 49, 53, 55, 59, 60, 68–72, 74, 75, 77], stakeholders’ engagement in the policy implementation process and partnerships [35, 36, 43, 47, 59, 60, 68–71, 74, 77, 80], monitoring and evaluation mechanisms for implementation and enforcement issues (such as stakeholders’ resistance and negotiation with private sectors) [36, 49, 55, 59, 60, 68, 70, 71, 76, 77, 80, 81], coordination mechanisms and leadership and implementation governance [35, 36, 42, 47, 49, 70, 75, 76, 79, 80], role of implementers [35, 72, 75, 76, 79], and policy communications among stakeholders [42, 70, 72]. Other influential factors identified include organizational capacity [35, 55], governance  and leadership [60, 70].
Methods and tools used to evaluate the implementation process
Of all the studies, 19 studies were conducted using qualitative methods, while four studies used quantitative and five used mixed methods. Semi-structured interviews were most commonly used with a list of open-ended questions to facilitate and guide the interview. Most of the tools were originally developed for use in particular countries.
In several cases, one or more types of qualitative methods were used in one single study. In-depth interviews were most commonly used as primary sources of data [33, 42, 43, 45, 53, 59, 68–70, 72–75, 78–81]. Either policy implementers or both policy implementers and other relevant stakeholders were often recruited as informants for in-depth and focus group interviews. Other qualitative methods used include document review, field observation and expert review [35, 36, 53, 62, 71, 78, 79]. Some studies used either a quantitative survey [47, 49, 52, 77] or mixed methods [35, 55, 60, 62, 73] for evaluation.
Twenty-one qualitative tools and eight quantitative tools were reported for evaluating the policy implementation process. Among the qualitative tools used were interview guides, which varied from highly to loosely structured. In some cases, the tools were adapted from existing tools. For example, McDonnell et al. (2006) used standard recommended focus group protocols developed by Krueger and Casey . In several cases, the studies developed their own tools such as a thematic matrix , interview and focus group guides [35, 68, 72, 74, 75] and lists of open-ended questions or issues to be explored [33, 43, 53, 55, 59, 60, 62, 69, 70, 73, 78–81]. Among the tools used, seven qualitative tools were presented data in a form of narrative report while three quantitative tools were based on numerical scores with different forms of data presentation, i.e. yes/no [47, 49] and scales from 0 to 5 .
Only one tool for assessing the policy implementation process was rated ‘high’ quality, i.e. the thematic matrix for guiding the interviews for an evaluation of the Norwegian Action Plan on Nutrition . Five studies were rated ‘medium’ quality [36, 47, 60, 79, 80], and 19 studies were rated ‘low’ quality [33, 35, 43, 45, 49, 52, 53, 55, 59, 68–70, 72–76, 78, 81]. Three studies provided insufficient information for the assessment [62, 71, 77].
This review identified 52 relevant studies across different policy areas, levels and settings, including 49 tools/methods used for assessing the implementation of government policies to create healthy food environments. The quality of these tools/methods varied widely, with only three tools/methods rated as high quality according to the detailed assessment criteria.
There were some broad similarities in the assessed aspects measured by the study and the methods and tools used. It is clear that policy implementation by governments has been measured in varying levels of detail, such as the existence or absence of policy implementation, level/degree of policy implementation and implementation coverage. Studies evaluating policy implementation processes mainly sought information about barriers and facilitators of policy implementation, particularly infrastructure support and resources, stakeholder engagement, leadership, and available monitoring and evaluation systems, which were the most commonly identified factors which impacted the policy implementation process.
There are no common standard methods and tools used to measure the policy implementation or to assess the policy implementation process. This may be due to the differing contexts and the needs or interests of assessors using these methods. The three tools that were rated as high quality (i.e. the INFORMAS Food-EPI, WHO Global Nutrition Policy Review questionnaire tool, and thematic matrix for guiding the interviews for an evaluation of the Norwegian Action Plan on Nutrition) could provide starting points for researchers and policymakers to identify appropriate methods for use in national and local assessment and evaluation of food environment policy implementation. However, there may be scope to include aspects of other tools as part of assessment methods, depending on context-specific requirements and the particular focus required. For example, the Report Card on Healthy Food Environments and Nutrition for Children in Canada included, combined or adapted indicators of several tools used for measuring progress in creating healthy food environments for obesity prevention to fit its purpose and scope and Canadian context .
Consideration should be given to harmonization of the use of methods and tools in this area. While it will always be important to apply tools and methods that are appropriate to the specific context in which they are to be implemented, the use of similar tools in different contexts will allow comparisons across countries and settings and over time. This will also facilitate effective benchmarking of performance which can help contribute to increasing accountability of governments for their actions to improve the healthiness of food environments.
The global impetus to assess policies for changing food environments is relatively new, and the development of appropriate tools for assessing implementation progress in this area is relatively under-developed. In contrast, in other public health policy areas such as tobacco, alcohol and breast milk, tools are relatively more advanced and have been used for assessing changes over time in a range of countries [83–88]. Examples include approaches to measuring breastfeeding policy implementation including the implementation of the international code of marketing of breast milk substitutes by WHO , International Baby Food Action Network (IBFAN)  and UNICEF  and tracking the progress of the implementation of policies and actions in alcohol and tobacco control by WHO [83, 87, 88]. These approaches share commonalities in terms of types of methods used for assessing policy implementation and provide useful means for the development of healthy food environments. Ninety-two countries, for example, have implemented the World Breastfeeding Trends Initiative (WBTi) tool, developed by IBFAN Asia, to track and monitor status and benchmark the progress of implementation of the Global Strategy for Infant and Young Child Feeding . This includes assessment of the strengths and weaknesses of their related policies and programmes. The assessment is conducted every 3–5 years, and the findings and recommendations are actively fed back to policymakers in each country.
The main strength of this study is that it is a comprehensive review based on a thorough and systematic search of the literature for policy assessment and evaluation. To our knowledge, it is the first time such a review has been conducted. The study rated the quality of each tool, and the methods used to conduct the quality assessment could be applied elsewhere. However, this study has several limitations. Firstly, the search was restricted to English-language publications. This may have resulted in the exclusion of important non-English publications. Moreover, studies assessing policy implementation were predominantly from high-income countries rather than low- or middle-income countries. This may be due to literature search being limited to peer-reviewed studies or grey publications published in English only. It may have missed some relevant documents published in languages other than English, especially documents from countries where English is not the main language. Furthermore, the studies identified were conducted in different contexts with different focuses, so they may be difficult to compare. The degree to which an approach used in one context is applicable to other contexts is uncertain. However, our findings are consistent with one recent paper identifying that there is little monitoring for accountability globally in this area .
Although there is a growing concern about the impact of unhealthy food environments on the prevalence and severity of obesity and diet-related NCDs globally and nationally, and some governments have implemented policies to improve the healthiness of food environments, a relatively small proportion of the implementation of these policies and actions is being assessed and evaluated. This review investigated methods and tools used to assess and evaluate the implementation of government policies to create healthy food environments for preventing obesity and diet-related NCDs. It provides a shortlist of high-quality tools and methods for assessing the implementation of such policies. Harmonization of the use of these high-quality methods and tools is needed to ensure that assessment of government policy implementation can be compared across different countries and settings and over time. The findings from the review are timely in that they provide insights for informing policy implementation and strengthening accountability mechanisms in the context of the increasing prevalence of obesity and diet-related NCDs in low-, middle- and high-income countries.
INFORMAS Healthy Food Environment Policy Index
International Baby Food Action Network
International Network for Food and Obesity/NCDs Research, Monitoring and Action Support
World Breastfeeding Trends Initiative
World Health Organization
Swinburn B, Sacks G, Hall K, McPherson K, Finegood D, Moodie M, et al. The global obesity pandemic: shaped by global drivers and local environments. Lancet. 2011;378:804–14.
Ledikwe J, Blanck H, Kettel Khan L, Serdula M, Seymour J, Tohill B, et al. Dietary energy density is associated with energy intake and weight status in US adults. Am J Clin Nutr. 2006;83:1362–8.
Perez-Escamilla R, Obbagy J, Altman J, Essery E, McGrane M, Wong Y, et al. Dietary energy density and body weight in adults and children: a systematic review. J Acad Nutr Diet. 2012;112:671–84.
World Health Organization. Global status report on noncommunicable diseases 2010. Geneva: World Health Organization; 2011.
Swinburn B, Vandevijvere S, Kraak V, Sacks G, Snowdon W, Hawkes C, et al. Monitoring and benchmarking government policies and actions to improve the healthiness of food environments: a proposed government healthy food environment policy index. Obes Rev. 2013;14 Suppl 1:24–37. doi:10.1111/obr.12073.
Story M, Kaphingst K, Robinson-O’Brien R, Glanz K. Creating healthy food and eating environments: policy and environmental approaches. Annu Rev Public Health. 2008;29:253–72.
World Health Organization. Global action plan for the prevention and control of noncommunicable diseases 2013–2020. Geneva: World Health Organization; 2013.
World Health Organization. Global nutrition policy review: what does it take to scale up nutrition action? Geneva: World Health Organization; 2013.
World Cancer Research Fund International. NOURISHING framework. London: World Cancer Research Fund International; 2014. http://www.wcrf.org/int/policy/nourishing-framework. Accessed 6 Nov 2014.
Beaglehole R, Bonita R, Ezzati M, Alleyne G, Dain K, Kishore S, et al. NCD countdown 2025: accountability for the 25 × 25 NCD mortality reduction target. Lancet. 2014;384(9938):105–7. http://dx.doi.org/10.1016/S0140-6736(14)61091-6.
Kraak V, Swinburn B, Lawrence M, Harrison P. An accountability framework to promote healthy food environments. Public Health Nutr. 2014;17(11):1–17. doi:10.1017/S1368980014000093.
Beaglehole R, Bonita R, Horton R. Independent global accountability for NCDs. Lancet. 2013;381(9867):602–5. http://dx.doi.org/10.1016/S0140-6736(13)60101-4.
Swinburn B, Kraak V, Rutter H, Vandevijvere S, Lobstein T, Sacks G, et al. Strengthening of accountability systems to create healthy food environments and reduce global obesity. Lancet. 2015;385:2534–45. doi:10.1016/s0140-6736(14)61747-5.
Swinburn B, Sacks G, Vandevijvere S, Kumanyika S, Lobstein T, Neal B, et al. INFORMAS (International Network for Food and Obesity/non-communicable diseases Research, Monitoring and Action Support): overview and key principles. Obes Rev. 2013;14(S1):1–12.
Bonita R, Beaglehole R, Kjellstrom T. Basic epidemiology. 2nd ed. Geneva: World Health Organization; 2007.
Bowling A. Research methods in health: investigating health and health services. 4th ed. Maidenhead: McGraw-Hill Education; 2014.
Hagger-Johnson G. Introduction to research methods and data analysis in the health sciences. Hoboken: Taylor and Francis; 2014.
Hammond M, Wellington J. Research methods: the key concepts. Hoboken: Taylor and Francis; 2012.
Lawrence M, Worsley A. Public health nutrition: from principles to practice. Crows Nest: Allen & Unwin; 2007.
Lovegrove JA, Hodson L, Sharma S, Lanham-New SA. Nutrition research methodologies. 1st ed. Hoboken: Wiley; 2015.
Hill R. What sample size is “enough” in Internet survey research? An electronic journal for the 21st century. 1998;6(3–4).
Isaac S, Michael WB. Handbook in research and evaluation. San Diego, CA: Educational and Industrial Testing Services; 1995.
Treece EW, Treece JW. Elements of research in nursing. 3rd ed. St. Louis, MO: Mosby; 1982.
Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health. 2008;31(2):180–91. doi:10.1002/nur.20247.
Connelly LM. Pilot studies. Medsurg Nurs. 2008;17(6):411–2.
Evans N, Lasen M, Tsey K. Chapter 2. Research method. A systematic review of rural development research: characteristics, design quality and engagement with sustainability. Cham: Springer; 2015. p. 12.
Organisation for Economic Co-operation and Development. Chapter 3. Assessment of learning outcomes and teaching quality in Kazakhstan. Reviews of national policies for education: secondary education in Kazakhstan. Paris: Paris Organisation for Economic Co-operation and Development (OECD); 2014.
Fink A. Conducting research literature reviews: from the internet to paper. 4th ed. Thousand Oaks, California: Sage; 2014.
Social Care Institute for Excellence. Guidelines for preparing a research review. Social Care Institute for Excellence. 2016. http://www.scie.org.uk/opportunities/commissions/. Accessed 7 January 2016.
MacDermott L, Stead M, Hastings G. Does food promotion influence children’s diet? A review of evidence. In: Cameron N, Norgan NG, Ellison GTH, editors. Childhood obesity: contemporary issues. Boca Raton, FL: Taylor & Francis; 2005.
Willis BH, Quigley M. The assessment of the quality of reporting of meta-analyses in diagnostic research: a systematic review. BMC Med Res Methodol. 2011;11(1):1–11. doi:10.1186/1471-2288-11-163.
Harper SL, Edge VL, Schuster-Wallace CJ, Ar-Rushdi M, McEwen SA. Improving aboriginal health data capture: evidence from a health registry evaluation. Epidemiol Infect. 2011;139(11):1774–83. doi:10.1017/s095026881000275x.
Ardzejewska K, Tadros R, Baxter D. A descriptive study on the barriers and facilitators to implementation of the NSW (Australia) Healthy School Canteen Strategy. Health Educ J. 2013;72(2):136–45. doi:10.1177/0017896912437288.
Hawkes C, Lobstein T, Polmark C. Regulating the commercial promotion of food to children: a survey of actions worldwide. Int J Pediatr Obes. 2011;6(2):83–94. doi:10.3109/17477166.2010.486836.
Holthe A, Larsen T, Samdal O. Implementation of national guidelines for healthy school meals: the relationship between process and outcome. Scan J Educ Res. 2011;55(4):357–78. doi:10.1080/00313831.2011.587321.
Rodriguez-Fernandez R, Siopa M, Simpson SJ, Amiya RM, Breda J, Cappuccio FP. Current salt reduction policies across gradients of inequality-adjusted human development in the WHO European region: minding the gaps. Public Health Nutr. 2014;17(8):1894–904. doi:10.1017/s136898001300195x.
Seo D. Comparison of school food policies and food preparation practices before and after the local wellness policy among Indiana high schools. Am J Health Ed. 2009;40(3):165–73.
Silberfarb LO, Savre S, Geber G. An approach to assessing multicity implementation of healthful food access policy, systems, and environmental changes. Prev Chronic Dis. 2014;11:130233. doi:10.5888/pcd11.130233.
Vandevijvere S, Swinburn B. Towards global benchmarking of food environments and policies to reduce obesity and diet-related non-communicable diseases: design and methods for nation-wide surveys. BMJ Open. 2014;4(5), e005339. doi:10.1136/bmjopen-2014-005339.
Vandevijvere S, Swinburn B. Pilot test of the Healthy Food Environment Policy Index (Food-EPI) to increase government actions for creating healthy food environments. BMJ Open. 2015;5(1), e006194. doi:10.1136/bmjopen-2014-006194.
Olstad DL, Raine KD, Nykiforuk CIJ. Development of a report card on healthy food environments and nutrition for children in Canada. Prev Med. 2014;69:287–95. http://dx.doi.org/10.1016/j.ypmed.2014.10.023.
World Health Organization/Europe. Evaluation of the Norwegian nutrition policy with a focus on the Action Plan on Nutrition 2007–2011. Copenhagen: The Regional Office for Europe of the World Health Organization; 2013.
Mâsse LC, Naiman D, Naylor PJ. From policy to practice: implementation of physical activity and food policies in schools. Int J Behav Nutr Phys Act. 2013;10:71. doi:10.1186/1479-5868-10-71.
Martin J, Peeters A, Honisett S, Mavoa H, Swinburn B, de Silva-Sanigorski A. Benchmarking government action for obesity prevention-an innovative advocacy strategy. Obes Res Clin Pract. 2013;8(4):e388–98. doi:10.1016/j.orcp.2013.07.001.
Barnidge EK, Radvanyi C, Duggan K, Motton F, Wiggs I, Baker EA, et al. Understanding and addressing barriers to implementation of environmental and policy interventions to support physical activity and healthy eating in rural communities. J Rural Health. 2013;29(1):97–105. doi:10.1111/j.1748-0361.2012.00431.x.
Anderson L, Foster S, Flynn R, Fitterman M. Assessing public policies and assets that affect obesity risk while building new public health partnerships, New Hampshire, 2011. Prev Chronic Dis. 2013;10(8):120349. doi:10.5888/pcd10.120349.
Schwartz MB, Henderson KE, Falbe J, Novak SA, Wharton CM, Long MW, et al. Strength and comprehensiveness of district school wellness policies predict policy implementation at the school level. J Sch Health. 2012;82(6):262–7. doi:10.1111/j.1746-1561.2012.00696.x.
Phillips M, Goodell M, Raczynski J, Philyaw Perez A. Creating and using index scores in the analysis of school policy implementation and impact. J Sch Health. 2012;82(6):253–61. doi:10.1111/j.1746-1561.2012.00695.x.
Budd EL, Schwarz C, Yount BW, Haire-Joshu D. Factors influencing the implementation of school wellness policies in the United States, 2009. Prev Chronic Dis. 2012;9:110296.
Beam M, Ehrlich G, Donze Black J, Block A, Leviton LC. Evaluation of the healthy schools program: part I. Interim progress. Prev Chronic Dis. 2012;9(3):110106.
Pitt Barnes S, Robin L, O’Toole TP, Dawkins N, Kettel Khan L, Leviton LC. Results of evaluability assessments of local wellness policies in 6 US school districts. J Sch Health. 2011;81(8):502–11. doi:10.1111/j.1746-1561.2011.00620.x.
Gaines AB, Lonis-Shumate SR, Gropper SS. Evaluation of Alabama public school wellness policies and state school mandate implementation. J Sch Health. 2011;81(5):281–7. doi:10.1111/j.1746-1561.2011.00588.x.
Haire-Joshu D, Elliott M, Schermbeck R, Taricone E, Green S, Brownson RC. Surveillance of obesity-related policies in multiple environments: the Missouri Obesity, Nutrition, and Activity Policy Database, 2007–2009. Prev Chronic Dis. 2010;7(4):A80.
Belansky ES, Cutforth N, Delong E, Litt J, Gilbert L, Scarbro S, et al. Early effects of the federally mandated local wellness policy on school nutrition environments appear modest in Colorado’s rural, low-income elementary schools. J Am Diet Assoc. 2010;110(11):1712–7. http://dx.doi.org/10.1016/j.jada.2010.08.004.
Longley CH, Sneed J. Effects of federal legislation on wellness policy formation in school districts in the United States. J Am Diet Assoc. 2009;109(1):95–101. http://dx.doi.org/10.1016/j.jada.2008.10.011.
Leowski J, Krishnan A. Capacity to control noncommunicable diseases in the countries of South-East Asia. Health Pol. 2009;92(1):43–8. doi:10.1016/j.healthpol.2009.02.003.
Yeatman H. Action or inaction? Food and nutrition in Australian local governments. Public Health Nutr. 2008;12(9):1399–407. doi:10.1017/S1368980008004114.
Probart C, McDonnell E, Weirich JE, Schilling L, Fekete V. Statewide assessment of local wellness policies in Pennsylvania public school districts. J Am Diet Assoc. 2008;108(9):1497–502. http://dx.doi.org/10.1016/j.jada.2008.06.429.
Action for Healthy Kids. Progress or promises? What’s working for and against healthy schools. Skokie: Action for Healthy Kids; 2008.
Molaison E, Carr D, Federico H. Investigation of school professionals’ and parents’ attitudes toward school wellness implementation in elementary schools. University: National Food Service Management Institute and University of Mississippi; 2007
Yee S, Williams-Piehota P, Sorensen A, Roussel A, Hersey J, Hamre R. The nutrition and physical activity program to prevent obesity and other chronic diseases: monitoring progress in funded states. Prev Chronic Dis. 2006;3(1):A23.
Lang T, Dowler E, Hunter D. Review of the Scottish Diet Action Plan: progress and impacts 1996–2005. Edinburgh: Health Scotland; 2006.
Lower T, Durham G, Bow D, Larson A. Implementation of the Australian core public health functions in rural Western Australia. Aust N Z J Public Health. 2004;28(5):418–25.
Institute of Medicine. Evaluating obesity prevention efforts: a plan for measuring progress. Washington: Institute of Medicine of the National Academies Press; 2013.
World Health Organization. Global strategy on diet, physical activity and health: a framework to monitor and evaluate implementation. Geneva: World Health Organization; 2008.
Tagtow A, Roberts S. Cultivating resilience: a food system blueprint that advances health of Iowans, farms & communities. Elkhart: Iowa Food Systems Council; 2011.
Fox A, Balarajan Y, Cheng C, Reich M. Measuring political commitment and opportunities to advance food and nutrition security: piloting a rapid assessment tool. Health Policy Plan. 2014;30(5):1–13. doi:10.1093/heapol/czu035.
Sanchez V, Hale R, Andrews M, Cruz Y, Bettencourt V, Wexler P, et al. School wellness policy implementation: insights and recommendations from two rural school districts. Health Promot Pract. 2014;15(3):340–8. doi:10.1177/1524839912450878.
Middleton G, Henderson H, Evans D. Implementing a community-based obesity prevention programme: experiences of stakeholders in the north east of England. Health Prom Int. 2014;29(2):201–11. doi:10.1093/heapro/das072.
Fagen MC, Asada Y, Welch S, Dombrowski R, Gilmet K, Welter C, et al. Policy, systems, and environmentally oriented school-based obesity prevention: opportunities and challenges. J Prev Intervention Community. 2014;42(2):95–111. doi:10.1080/10852352.2014.881175.
Taylor J, McKenna M, Butler G. Monitoring and evaluating school nutrition and physical activity policies. Can J Public Health. 2010;101:S24–7.
MacLellan D, Holland A, Taylor J, McKenna M, Hernandez K. Implementing school nutrition policy: student and parent perspectives. Can J Diet Pract Res. 2010;71(4):172–7. doi:10.3148/71.4.2010.172.
Harris CV, Bradlyn AS, Tompkins NO, Purkey MB, Kennedy KA, Kelley GA. Evaluating the West Virginia Healthy Lifestyles Act: methods and procedures. J Phys Act Health. 2010;7 Suppl 1:S31–9.
Agron P, Berends V, Ellis K, Gonzalez M. School wellness policies: perceptions, barriers, and needs among school leaders and wellness advocates. J Sch Health. 2010;80(11):527–35. doi:10.1111/j.1746-1561.2010.00538.x.
Allender S, Gleeson E, Crammond B, Sacks G, Lawrence M, Peeters A, et al. Moving beyond ‘rates, roads and rubbish’: how do local governments make choices about healthy public policy to prevent obesity? Australia and New Zealand Health Policy. 2009;6:20. doi:10.1186/1743-8462-6-20.
McDonnell E, Probart C, Weirich J. School foodservice directors’ perceptions and concerns about local wellness policy development, implementation, and enforcement. J Child Nutr Manag. 2006;30(1). http://docs.schoolnutrition.org/newsroom/jcnm/06spring/mcdonnell/index.asp.
Dunět D, Butterfoss F, Hamre R, Kuester S. Using the State Plan Index to evaluate the quality of state plans to prevent obesity and other chronic diseases. Prev Chronic Dis. 2005;2:1–10.
Abery E, Drummond C. Implementation of mandatory nutritional guidelines in South Australian primary school canteens: a qualitative study. Improv Sch. 2014;17(1):41–53.
Crammond B, Van C, Allender S, Peeters A, Lawrence M, Sacks G, et al. The possibility of regulating for obesity prevention—understanding regulation in the Commonwealth Government. Obes Rev. 2013;14(3):213–21. doi:10.1111/obr.12004.
Shill J, Mavoa H, Allender S, Lawrence M, Sacks G, Peeters A, et al. Government regulation to promote healthy food environments—a view from inside state governments. Obes Rev. 2012;13(2):162–73. doi:10.1111/j.1467-789X.2011.00937.x.
Chung A, Shill J, Swinburn B, Mavoa H, Lawrence M, Loff B, et al. An analysis of potential barriers and enablers to regulating the television marketing of unhealthy foods to children at the state government level in Australia. BMC Public Health. 2012;12:1123. doi:10.1186/1471-2458-12-1123.
Krueger R, Casey M. Focus groups: a practical guide for applied research. Thousand Oaks: Sage Publications; 2000.
Rekve D, editor. Implementation of the global strategy to reduce the harmful use of alcohol. Committee on National Alcohol Policy and Action 8th Meeting; 2011 March 1st-2nd; Luxembourg-Kirchberg, rue Alcide De Gasperi: World Health Organization; 2011.
The International Baby Food Action Network Asia. World Breastfeeding Trends Initiative (WBTi). Dehli: the International Baby Food Action Network Asia. 2008.
World Health Organization. Infant and young child feeding: a tool for assessing national practices and policies and programmes. Geneva: World Health Organization; 2003.
World Health Organization. Country implementation of the international code of marketing of breast-milk substitutes: status report 2011. Geneva: World Health Organisation; 2013.
World Health Organization. WHO report on the global tobacco epidemic, 2013: enforcing bans on tobacco advertising, promotion and sponsorship. Luxembourg: World Health Organization; 2013.
Cox A, Lutz B, Webb D, Sahal-Estime M, Small R, Trivedi V. Development planning and tobacco control: integrating the who framework convention on tobacco control into UN and national development planning instruments. New York: United Nations Development Programme; 2014.
The United Nations Children’s Fund. Infant and young child feeding programming status. New York: United Nations Children’s Fund; 2012.
SP, GS and ML are researchers within a National Health and Medical Research Council (NHMRC) Centre for Research Excellence in Obesity Policy and Food Systems (APP1041020). GS is the recipient of an Australian Research Council Discovery Early Career Researcher Award (project number DE160100307). ML is a researcher within an Australian Research Council Linkage Project, ‘Modelling policy interventions to protect Australia’s food security in the face of environmental sustainability challenges’ (LP120100168). SV is funded through a research fellowship from the National Heart Foundation of New Zealand.
SP designed the review and undertook the data extraction. ML was the second reviewer to assess independently the quality of all the studies. SP, ML, SV, GS, AW and VT contributed to the drafting of the manuscript and have read and approved the final manuscript.
About this article
Cite this article
Phulkerd, S., Lawrence, M., Vandevijvere, S. et al. A review of methods and tools to assess the implementation of government policies to create healthy food environments for preventing obesity and diet-related non-communicable diseases. Implementation Sci 11, 15 (2015). https://doi.org/10.1186/s13012-016-0379-5