Skip to main content
Download PDF
Download ePub

Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies

Implementation Science volume 15, Article number: 37 (2020) Cite this article

Abstract

Background

Nutrition and physical activity policies have the potential to influence lifestyle patterns and reduce the burden of non-communicable diseases. In the world of health-related guidelines, GRADE (Grading of Recommendations, Assessment, Development and Evaluation) is the most widely used approach for assessing the certainty of evidence and determining the strength of recommendations. Thus, it is relevant to explore its usefulness also in the process of nutrition and physical activity policymaking and evaluation.

The purpose of this scoping review was (i) to generate an exemplary overview of documents using the GRADE approach in the process of nutrition and physical activity policymaking and evaluation, (ii) to find out how the GRADE approach has been applied, and (iii) to explore which facilitators of and barriers to the use of GRADE have been described on the basis of the identified documents. The overarching aim of this work is to work towards improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior.

Methods

A scoping review was conducted according to current reporting standards. MEDLINE via Ovid, the Cochrane Library, and Web of Science were systematically searched up until 4 July 2019. Documents describing a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as “guidelines,” or systematic reviews used to inform policymaking were included.

Results

Thirty-six documents were included. Overall, 313 GRADE certainty of evidence ratings were identified in systematic reviews and guidelines; the strength of recommendations/policies was assessed in four documents, and six documents mentioned facilitators or barriers for the use of GRADE. The major reported barrier was the initial low starting level of a body of evidence from non-randomized studies when assessing the certainty of evidence.

Conclusion

This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions (policymaking), as well as in the development of guidelines intended to guide policymaking. Several areas for future research were identified to explore the use of GRADE in health policymaking and evaluation.

Peer Review reports

Background

According to the most recent report by the Global Burden of Disease study group, non-communicable diseases (NCDs), such as cardiovascular disease, cancer, and type 2 diabetes, accounted for 64.8% of total deaths worldwide in 2017 [1]. Major risk factors for the development of NCDs are obesity, dyslipidemia, insulin resistance, and hypertension [1,2,3]. Suboptimal dietary behavior and physical inactivity, including increased sedentary behavior, are related to these risk factors. Therefore, adherence to an optimal diet and physical activity recommendations can contribute to the prevention of obesity and several NCDs [4].

On a public health level, policies are important instruments that can be used to change people’s behavior [5]. While the definition of what constitutes a policy remains a topic of debate, for the purpose of this review, we use the term “policy” both for (a) interventions implemented at a population level, as well as for (b) recommendations on a specific course of action to be implemented by a government or other public body (see Table 1).

Table 1 Overview of the definitions “policy” and examples for policymaking in the present scoping review
Full size table

Policies have the potential to influence lifestyle patterns within a population [5]. Certain good-practice characteristics of successful policies on optimal diets and physical activity have been identified in recent research, such as the use of the theory of change, analyses of target populations and target behavior, content development and management, multidimensionality, and appropriate consideration of practitioners and settings [9]. For example, interventions such as mass media campaigns, school programs, or changes of the environment through communities showed to be effective in increasing physical activity levels among a population, especially when they operate on multiple levels and contain multiple components [10]. However, to be maximally effective, they require regular evaluation and improvement. The Policy Evaluation Network (PEN), a consortium funded by the European Union, was founded to help improve policymaking in the areas of dietary behavior, physical activity, and sedentary behavior by evaluating existing policies [6, 11]. This scoping review is part of the PEN-project, with a special focus on policies that explicitly target, or are likely to impact upon, population health, in particular through NCDs.

To increase the acceptance and impact of polices, it might be helpful if they adhered to principles widely accepted, for instance, similar to health guidelines, i.e., they should be transparent, structured, and evidence-informed [12, 13]. Within the PEN-project, it is explored if these guideline standards, which also include the assessment of the certainty of evidence, should be considered when developing and evaluating a policy [14].

The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach and the GRADE Evidence to Decision (EtD) frameworks are important methods for the development and evaluation of recommendations for practice and can therefore also help to make evidence-informed decisions on a population level [15,16,17]. GRADE is a well-recognized tool to facilitate structured decision-making and is potentially the most widely used approach for assessing the certainty of evidence and determining the strength of a recommendation [18]. It is standard practice to use the GRADE approach to form trustworthy evidence-informed recommendations, especially in the development of guidelines in the broader field of medicine. For this reason, it seems sensible to explore its potential in the process of developing and evaluating policies concerned with nutrition and physical activity [16].

The primary aim of this scoping review is to investigate how and to what degree the GRADE approach has contributed to policy development in the context of nutrition and physical activity. To this end, the following steps were taken: (i) an overview of documents using the GRADE approach in the context of nutrition and physical activity policymaking and evaluation was compiled, (ii) methods of applying the GRADE approach (e.g., by rating the certainty of evidence or the strength of a recommendation) were recorded, and (iii) facilitators of and barriers to the use of GRADE described in the identified documents were identified and explored.

Methods

This scoping review was conducted according to the methodology of the Joanna Briggs Institute’s (JBI) Reviewers’ Manual 2015 and is reported following the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) extension for scoping reviews, the PRISMA-ScR Statement (Additional file 1) [19, 20]. To complement our scoping review, a survey was conducted by contacting European health policymaking institutions to further investigate the use of GRADE in nutrition and physical activity policymaking and evaluation and to potentially identify additional policy documents.

Search strategy

First, policy evaluations in the form of systematic reviews and other relevant documents published in peer-reviewed journals were identified through searches in the databases MEDLINE, Web of Science, and the Cochrane Library. The search strategy was developed with the support of an information specialist and adapted for each database accordingly. Databases were searched up until 4 of July 2019, and the search was not restricted to any language. The MEDLINE (Ovid) search strategy is presented in Additional file 2. Moreover, forward citation tracking and reference checking from included documents were performed.

Secondly, an internet-based search through the Google™ search engine and through Google Scholar™ was performed to identify gray literature and different relevant institutions and organizations, such as the Food and Agriculture Organization for the United Nations (FAO), World Health Organization (WHO), World Cancer Research Fund International (WCRF), the National Institute for Health and Care Excellence (NICE), and the Campbell Collaboration. Following this, the types of documents relevant for policymaking and which address dietary behavior, physical activity, and/or sedentary behavior, such as guidelines, reports, or systematic reviews, were searched on the websites of relevant organizations. Documents were found both through the Google search directly and also retrieved from the websites of those organizations and institutions identified in the search.

Thirdly, an extensive overview of existing policies including policy evaluations from the NOURISHING-database by the WCRF was considered [21]. Lastly, science platforms and databases such as ProQuest, Dimensions, BASE, Metager, Oaister, and the Political Science Database (Berkeley) were searched.

Inclusion criteria

Because of the lack of a standard definition of the term “policy” across various areas of health policy and policy decision-making, the current search also aimed to identify documents that might not have used the term “policies” or “policy evaluations.” We focused on documents that described a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as “guidelines,” or systematic reviews used to inform policymaking [22]. Furthermore, we established criteria for interventions and/or recommendations (i.e., policies) to be eligible. Documents had to meet the following criteria to be included in this scoping review:

  1. (i)

    At least one intervention or recommendation of a policy document targeted the PEN-related outcomes: dietary behavior, physical activity, or sedentary behavior.

  2. (ii)

    Recommendations targeted a population rather than individuals. This means that at least one intervention or recommendation had to fit in one of the following categories: school program (only if mandatory to school children, e.g., more physical exercise lessons or change in school meal offerings), mass media (e.g., commercials, poster campaigns), infrastructure (e.g., standing desks), financial incentive (e.g., taxation on sugar or fat), change of offerings (e.g., portion/package sizes), nutrition labeling, and population education (in the sense of recommendations, e.g., guidelines). These categories were established by reviewing a set of pre-identified documents.

  3. (iii)

    Specific measures or action plans (potential policies) that were or could be implemented by a government or an inter-governmental organization such as the WHO (e.g., labeling and school programs).

  4. (iv)

    In addition to meeting criteria i–iii, GRADE was used to assess the certainty of evidence or the strength of recommendations or reasons for not using GRADE were explicitly noted.

Exclusion criteria

There were three exclusion criteria:

  1. (i)

    Since the GRADE approach is primarily applicable in the context of evidence syntheses within systematic reviews, health technology assessments (with or without meta-analyses) or guidelines, surveys, and primary studies (e.g., randomized controlled trials (RCTs) or non-randomized studies (NRSs)) were excluded.

  2. (ii)

    Documents concerning the methodological aspects of policy implementation were excluded.

  3. (iii)

    Documents about interventions that specifically target individuals instead of a population were also excluded.

Selection process of sources of evidence

First, title and abstract screening was performed by one reviewer (JZ). Only clearly irrelevant references were excluded at this stage. Second, for all potentially relevant references, full-text publications were obtained and checked for final inclusion by two reviewers (JZ, LS) independently. Uncertainties were resolved through discussion with a third author (JJM).

Data extraction

For included documents, one reviewer (JZ) extracted the following characteristics: name of first author, year of publication, document type, study aim, outcome of interventions (physical activity, dietary behavior, sedentary behavior, multi-component), design of primary studies included in the document, policy/intervention(s), target group/study setting, use of GRADE, GRADE ratings for relevant outcomes (including down- and upgrading factors), facilitators, and barriers to apply GRADE. The data extraction form was piloted and proved to be applicable to all types of retrieved documents. Data extraction was verified by a second reviewer (LS).

Data categorization and presentation

The documents were categorized according to policy evaluation, evidence of effectiveness for policymaking, or a recommendation to guide policymaking (see Table 1). Furthermore, we categorized the documents according to the applied interventions (type of policy) in the studies (e.g., school program and mass media), according to the setting of the studies (e.g., workplace and community) and according to PEN-relevant outcomes (dietary behavior, physical activity, and sedentary behavior). All these characteristics were collected for each document and were displayed in Table 2. In further tables, we displayed the GRADE ratings, facilitators, and barriers for the use of GRADE (Table 3), as well as the strengths of recommendations (Additional file 4).

Table 2 Characteristics of included studies
Full size table
Table 3 Studies that applied the GRADE approach for rating PEN-relevant outcomes
Full size table

Survey

To complement the scoping review, a survey was conducted by contacting Ministries of Health or other responsible institutions that play a role in health policymaking in all 53 European countries, as listed by the WHO (Additional file 3) [59]. Contact information was searched through the FAO websites, the WHO websites, and respective websites of national governments. An e-mail was sent in English with a brief description of the project and the question whether the GRADE approach had been used for nutrition and physical activity policymaking in these countries.

Results

The database search retrieved 6164 documents (MEDLINE via Ovid, 4782; Cochrane Library, 309; Web of Science, 1073). Another 19 documents were retrieved through further reference checking and from the gray literature. After removal of duplicates (n = 511), title and abstract screening of 5672 documents was performed, from which 5443 documents were excluded. The remaining 229 documents were reviewed in full-text screening. The eligibility criteria were met by 36 documents which were included in this scoping review (Fig. 1) [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58]. The science databases and platforms did not yield any additional documents relevant for this scoping review.

Fig. 1
figure 1

Flow diagram showing study selection process

Full size image

An overview of the use of GRADE in policymaking and policy evaluation

The characteristics of the included documents are displayed in Table 2. All documents were published in English, and they were either systematic reviews with meta-analyses (n = 10; 28%), systematic review protocols (n = 8; 22%), systematic reviews without meta-analyses (n = 9; 25 %), guidelines (n = 6; 17%), an umbrella review (n = 1; 3%), or papers on guideline making (n = 2; 5%). From these findings, 17 documents (47%) were Cochrane reviews (including five Cochrane review protocols). Four documents were international guidelines by the WHO and one guideline by NICE (physical activity and the environment [45]). Furthermore, two sets of national-level guideline recommendations to guide policymaking were found: the Canadian 24-Hour Movement Guidelines for Children and Youth [52, 53]. Documents were published between 2011 and 2019, and half of those (n = 18) were published within the last 3 years. Settings of the studies included communities, schools, or workplaces. The majority of recommendations/interventions addressed the general population through various forms of mass media. Furthermore, the included documents examined various combinations of interventions addressing dietary behavior, physical activity, and sedentary behavior, and also combinations of these with other health-related topics. Most of the studies investigated interventions related to dietary behavior alone (39%), followed by physical activity (17%). There was only one study that assessed interventions aiming to diminish sedentary behavior alone (3%), and all other documents (41%) were based on interventions that targeted various outcomes, focusing on multicomponent interventions that addressed general health and aspects such as alcohol or tobacco consumption.

Many of the documents included did not review implemented policies and focused rather on interventions that have the potential to drive policy in the future. Seven of the 36 documents include studies in which the effects of implemented nutrition and physical activity policies were examined, including nutrition labeling [27, 49], SSB-taxes [35], the WHO Health Promoting Schools framework [39], taxation of fat content in food [40], sodium restriction [43], and taxation of unprocessed sugar or sugar-added foods [48]. Five of these are based on policies issued by governments (see Table 2). Twenty-two of the included documents applied the GRADE approach for grading the certainty of evidence for PEN-related outcomes (for instance, increase of physical activity, change of body-mass index (BMI), and sugar or salt intake) (see Table 3). The remaining 14 documents were either systematic review protocols (n = 8, [26, 33, 35, 36, 40, 44, 48, 50]) where GRADE will be used for grading the certainty of evidence, one recommendation to guide policymaking which used GRADE-ADOLOPMENT (GRADE Evidence to Decision frameworks for adoption, adaptation, and de novo development of trustworthy recommendations) (n = 1, [47]), documents in which GRADE was used on another development stage of the document (i.e., recommendations where GRADE ratings of included systematic reviews were evaluated, n = 2, [52, 53]), or documents in which the barriers against the use of GRADE were stated (n = 3, [30, 32, 42]) (see Table 4).

Table 4 Facilitators and barriers mentioned by authors
Full size table

Table 3 shows the ratings of the certainty of evidence for all PEN-related outcomes for each study (max. seven ratings per identified document are presented in the table), and the reasons for down- and upgrading. Overall, there were 313 individual outcome ratings (107 GRADE ratings were done for the NICE guideline alone [45]): 34% of very low, 38% of low, 21% of moderate, and 7% of high certainty of evidence. The main reasons for downgrading were high risk of bias, imprecision (e.g., for wide confidence interval, small sample size, and incomplete outcome data), or there was an initial low starting level of a body of evidence from non-randomized study design type (Table 3).

The WHO guidelines were the only documents which, in addition to rating the certainty of evidence for each outcome, also provided the “strength of recommendations” for each recommendation included in the guideline (Additional file 4).

Facilitators and barriers for the use of GRADE from identified documents

In two documents, an adaption of GRADE was used for primary outcomes [24, 30]. In three documents, the authors mentioned the GRADE approach or presented specific barriers that prevented them from applying GRADE [30, 32, 42], and in one document, the GRADE-ADOLOPMENT approach was used [47] (see Table 4).

Barriers for the use of GRADE for the assessment of the certainty of evidence were mentioned only in six documents. Reasons cited by authors for their decision not to use GRADE included, among others, incomplete, insufficient, or inaccessible data [24, 32, 42]. In another document, it was mentioned that difficulties in the application of the GRADE approach were seen due to the research topic, where RCTs are difficult to conduct, or because of subjective outcome measures such as quality of life [42]. Dyson et al. [30] mentioned some general challenges with the GRADE approach, such as the initial low starting level of a body of evidence from cohort studies and the difficulties of applying GRADE for individualized interventions. Instead, Dyson et al. adopted a grading system very similar to the GRADE system, in which recommendations on the certainty of evidence are provided on a scale from 1 to 4 (strong to very low strength) [30]. The motivation for using this adapted approach is not discussed. In the NICE guideline, a modified version of GRADE was used where NRSs started as “high quality evidence” when the study design was the most feasible one for the examined intervention [45]. Only one document mentioned facilitators for the use of GRADE; however, this was for another GRADE methodology, namely the GRADE-ADOLOPMENT, which was used by Okely et al. for the development of new guidelines [47]. There was no mention of facilitators or barriers for assessing the strengths of recommendations in any of the included documents. Facilitators and barriers are presented in Table 4.

Complementary survey among European health policymakers

No documents were retrieved through the survey sent to ministries and authorities in 53 European countries. Staff members from eleven countries replied to the request, stating that their national policies were not developed according to the GRADE approach (Additional file 3). Representatives from those countries, namely Austria, Lithuania, Luxemburg, Monaco, the Netherlands, Northern Ireland, and Poland, replied that they do not use GRADE in any way for policymaking. The German Nutrition Society used a newly developed grading system called NutriGrade for their protein guideline [60]. As a response to the survey, Bosnia and Herzegovina showed explicit interest in using GRADE for future policymaking. Iceland and Norway stated that they do not use GRADE for policymaking but in some cases for national clinical guidelines. Monaco replied that they refer to the WHO or French national recommendations. Furthermore, the Nordic Nutrition Recommendations use WHO guidelines, and Norway and Iceland reported that their policies are either based on the Nordic Nutrition Recommendations or they make policies which are based on WHO guidelines directly. No reasons were given whatsoever why GRADE has not been used directly.

Discussion

Summary of findings

To the best of our knowledge, this scoping review is the first to examine if and to what extent the GRADE approach has been used in nutrition and physical activity policymaking and policy evaluation in the areas of dietary behavior, physical activity, and sedentary behavior. There were no findings of actual policies or formulation processes from governmental bodies; however, five systematic reviews included evidence from interventions that were based on policies issued by a government or governmental organization (see Table 2). This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions, as well as for guidelines and recommendations intended to guide policymaking, where the evidence for a recommendation or an intervention has to be assessed. These results may contribute to improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior.

Almost half of the included documents that used the GRADE approach to assess the certainty of evidence are Cochrane Reviews (n = 17). The reason for this is that Cochrane adopted the GRADE approach for the evaluation of the certainty of evidence in systematic reviews [61]. The strength of recommendations has only been determined in WHO guidelines, and no barriers to its use were mentioned in these cases. Already in 2003, the WHO selected the GRADE approach as the method of choice to be used in their guideline development [62]. This explains why the guidelines identified in the context of this scoping review are mostly WHO guidelines.

Although guideline and recommendation development are a major part of policymaking, a number of perceived concerns may prevent institutions from adopting available methods. With respect to applying GRADE to policies affecting dietary behavior, physical activity, and sedentary behavior, concerns that are frequently raised include the assessment of risk of bias and the initial low starting level of a body of evidence obtained from NRSs. According to the GRADE approach, the certainty of evidence is initially determined by study design: a body of evidence from RCTs starts with a high certainty, whereas a body of evidence from NRSs, such as prospective observational studies, starts with low certainty due to confounding and selection bias issues associated with NRSs [63, 64]. However, with the advent of tools that use the concept of a target trial as reference point, such as the ROBINS-I (risk of bias in non-randomized studies of interventions), initial certainty of the evidence can also be high for bodies of evidence from NRSs [63, 64]. Nevertheless, as stated above, it is likely that NRSs would be downrated to low certainty because of confounding and selection bias [63].

There has been a long debate regarding what constitutes best evidence in nutrition research and whether it emerges from RCTs in which the effects of a dietary change on disease, intermediate or surrogate markers, or recognized risk markers are evaluated [65]. However, most dietary intervention RCTs are of short duration and do not target patient-relevant outcomes such as morbidity or mortality. RCTs, if well-designed and conducted, provide robust answers to the research questions they address and are considered the ideal methodology for causal inference. NRSs, on the other hand, provide less-robust information regarding causality but are usually considered more applicable for nutrition research. Additionally, all common dietary assessment methods used in nutrition research, such as food consumption records, 24 h recalls, dietary records, dietary history, and food frequency questionnaires suffer from inherent limitations, for example, because the measures are subjective or the results are difficult to reproduce [66].

Apart from the WHO guidelines and the NICE guideline, there were no publications closer to the legislative policymaking process (e.g., policy briefs, policy documents, proposals to inform laws or regulations) that used GRADE. The legislative process underlying policymaking is complex and time-consuming. In addition, there are specific challenges associated with translating empirical findings from the lab into actionable policies in government [66,67,68,69]. The absence of documents could be due to various factors: for example, key documents not being publicly available (e.g., internal position papers by political parties), documents being informed by GRADE not explicitly using the term, or the search strategy being specifically tailored to identify those types of policy documents. It could also be due to GRADE not being employed to inform policymakers’ decisions, for example, due to issues in uptake of evidence in the policymaking process [67,68,69], inadequate knowledge-translation strategies [70,71,72], or an unawareness about GRADE. Future research needs to address this issue.

Strengths and limitations

This scoping review has several strengths. Firstly, the inclusion criteria and the search strategy planned for this scoping review were broad, including an extensive search of gray literature such as databases of the WHO or the WCRF in addition to more traditional sources. This access to a wide variety of documents in the research field yields a better overview than previously available. Secondly, we performed a rigorous approach to screening and data extraction, which was performed by two reviewers independently. Thirdly, the survey among national policymakers of the European region brought relevant input and insights into this study by giving an idea about the current use of GRADE within policymaking institutions.

Limitations of this scoping review in our view are as follows. Firstly, developing a suitable search strategy was challenging. Though the search was systematic and as broad as possible, it proved to be difficult to grasp all the facets of the topic. Secondly, only documents were included in which GRADE was applied or reasons were provided for not using GRADE. A different approach with inclusion of documents that could have applied GRADE but did not do so and a research design enabling an in-depth investigation of policymakers’ views (e.g., a mixed methods approach) might have provided further insights into this topic. These approaches would have allowed for a thorough investigation of the extent of evidence use without structured consideration of certainty of evidence and strength of recommendations in policymaking and evaluation. However, this was beyond the scope of this project. Thirdly, a standard definition of the terms “policy” and “health policy” was difficult to determine since various definitions can be found in the literature. Only a few of the included documents (n = 7) actually examined real policies. In these cases, the authors did not distinguish between the types of included studies, that is, if an intervention was carried out in an experimental/artificial setting or based on a law/regulation. Lastly, a document was included even when only one of the interventions in the review was considered a policy intervention although the majority of the remaining interventions were not. In this case, ratings for the certainty of evidence with GRADE are likely to be higher than for NRSs in real-life settings as it is much easier to conduct RCTs in experimental settings. 

Findings from other studies

This is the first scoping review on the use of GRADE in nutrition and physical activity policy evaluation, but other publications have also addressed the complexity of interventions in the public health field, which might prevent researchers and policymakers from applying this approach. Public health and health policy interventions are often thought of as events in systems [73], whereby complex intervention (e.g., characterized by several interacting active components intended to influence multiple behaviors [74]) are implemented in an interactive system (e.g., characterized by changes of the system due to emergent properties, adaptivity, or feedback mechanisms [73, 75]). However, there is specific guidance on how the GRADE approach can be applied in these cases [76, 77].

A study conducted by Rehfuess and Akl in 2013 examined the experience with applying the GRADE approach to public health interventions [77]. In their survey performed in 2013, NICE, which is the national institute that provides guidance and advice to improve health and social care in the UK, stated that the most important reason for not using GRADE when investigating (complex) public health interventions was the fact that most of the evidence originates from NRSs [77]. However, since 2014, NICE included the GRADE approach in their manual for guideline development, also for public health guidelines [78]. Since bodies of NRSs start with low certainty of evidence, it seemed that this would hinder the application of GRADE in such cases [30]. This issue was also brought up in the findings of this scoping review as mentioned above.

Recently, Blake and colleagues showed that from 32 food-based dietary guidelines published since 2010, none of the guidelines used the GRADE approach to evaluate the certainty of evidence or graded the strength of recommendations [79]. However, the authors also acknowledged that methods may not be directly applicable to the types of studies included in food-based dietary guidelines. Of note, GRADE has established working groups focusing on an approach for public health, modeling studies, or environmental exposure studies.

Implications that follow for the broader research field

It has been pointed out that the use of the GRADE approach in particular in relation to bias assessment is challenging (e.g., users of GRADE may inappropriately double count the risk of confounding and selection bias so the certainty of evidence is downgraded too much) [60, 80]. Further, the usefulness of GRADE to assess the evidence of effectiveness in domains of health policy other than dietary behavior, physical activity, and sedentary behavior needs to be explored. In general, knowledge translation and exchange strategies (i.e., methods of bringing scientific findings to policymakers) have provided mixed results regarding the use or uptake of scientific evidence in the policymaking process [70,71,72]. One key issue has been the perceived limited relevance, generalizability, and applicability of research findings to a specific setting [81]. Future research should focus on assessing the views of policymakers on the usefulness of GRADE to inform their decisions.

Moreover, beyond the evidence of effectiveness in a given context, public health and health policy decision-makers have to consider numerous additional aspects [82,83,84,85], for example, stakeholder interests, ethical concerns, equity/equality considerations, financial implications, or societal and cultural values and norms. These have to be taken into account along with scientific evidence of effectiveness which is often not the most important consideration [68]. In addition, there are individual, group, or systemic factors such as own experiences of policymakers, the political system, institutional mechanisms (structures, processes, or regulations), or contextual, societal factors [69, 86]. The GRADE EtD frameworks might be a suitable approach to address these issues. They might help policymakers to identify, prioritize, and address issues beyond benefit-harm considerations in the healthcare decision-making process in a structured and transparent way such as resource use and cost-effectiveness, equity, acceptability, and feasibility [15, 17]. However, we did not identify any documents which used the EtD frameworks.

Conclusion

This scoping review was the first attempt to shed some light on how the GRADE approach has been used in nutrition and physical activity policymaking and evaluation. In conclusion, we show that the GRADE approach is able to support the processes of health policymaking (e.g., guidelines) and policy evaluation (e.g., systematic reviews evaluating policy interventions) by facilitating a structured and transparent use of evidence, and that it has already been used in the areas of dietary behavior, physical activity, and sedentary behavior. The aforementioned barriers regarding the use of the GRADE approach have been recognized by the GRADE working group as well, since new tools such as ROBINS-I have been endorsed for the assessment of the certainty of evidence, especially to help decision-makers and policymakers in the area of public health. GRADE is being used as a standard method during guideline development in various institutions, which is an important part in the process of informing policymaking. Nevertheless, our survey shows that GRADE is not being used regularly for actual policymaking. We believe the evaluation of policies should be consistent with standardized methods in order to have the best possible impact on improving health and quality of life, and the GRADE approach could play an important role in this.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Abbreviations

BMI:

Body mass index

EtD:

Evidence to Decision

FAO:

Food and Agriculture Organization for the United Nations

GRADE:

Grading of Recommendations Assessment, Development and Evaluation

GRADE-ADOLOPMENT:

GRADE Evidence to Decision frameworks for adoption, adaptation, and de novo development of trustworthy recommendations

JBI:

Joanna Briggs Institute

LDL:

Low-density lipoprotein

NCDs:

Non-communicable diseases

NICE:

National Institute for Health and Care Excellence

NRSs:

Non-randomized studies

PEN:

Policy Evaluation Network

PRISMA:

Preferred Reporting Items for Systematic reviews and Meta-Analyses

RCTs:

Randomized controlled trials

ROBINS-I:

Risk of bias in non-randomized studies of interventions

SSB:

Sugar-sweetened beverages

WCRF:

World Cancer Research Fund International

WHO:

World Health Organization

References

  1. Stanaway JD, Afshin A, Gakidou E, Lim SS, Abate D, Abate KH, et al. Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1923–94.

    Article  Google Scholar 

  2. Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet (London, England). 2016;388(10053):1659–724.

  3. Ezzati M, Riboli E. Behavioral and dietary risk factors for noncommunicable diseases. N Engl J Med. 2013;369(10):954–64.

    Article  CAS  PubMed  Google Scholar 

  4. Afshin A, Sur PJ, Fay KA, Cornaby L, Ferrara G, Salama JS, et al. Health effects of dietary risks in 195 countries, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2019;393(10184):1958–72.

    Article  Google Scholar 

  5. Woods C, Mutrie N. Putting Physical Activity on the Policy Agenda; 2012. p. 92–104.

    Google Scholar 

  6. Lakerveld J, Woods C, Hebestreit A, Brenner H, Flechtner-Mors M, Harrington JM, et al. Advancing the evidence base for public policies impacting on dietary behaviour, physical activity and sedentary behaviour in Europe: The Policy Evaluation Network promoting a multidisciplinary approach. Food Policy. 2020;101873.

  7. World Health Organization; Health Policy [Available from: http://www.who.int/topics/health_policy/en/]. Accessed 9 Dec 2019.

  8. de Leeuw E, Clavier C, Breton E. Health policy--why research it and how: health political science. Health Res Policy Syst. 2014;12:55.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Brug J, van der Ploeg HP, Loyen A, Ahrens W, Allais O, Andersen LF, et al. Determinants of diet and physical activity (DEDIPAC): a summary of findings. Int J Behav Nutr Phys Act. 2017;14(1):150.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Puggina A, Aleksovska K, Buck C, Burns C, Cardon G, Carlin A, et al. Policy determinants of physical activity across the life course: a ‘DEDIPAC’ umbrella systematic literature review. Eur J Pub Health. 2018;28(1):105–18.

    Article  Google Scholar 

  11. PEN. Effectiveness of existing policies for lifestyle interventions - Policy Evaluation Network (PEN) 2017 [Available from: https://www.jpi-pen.eu/]. Accessed 9 Dec 2019.

    Google Scholar 

  12. WHO Handbook of guideline development. 2014. [Available from: https://apps.who.int/medicinedocs/en/m/abstract/Js22083en/]. Accessed 9 Dec 2019.

  13. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P, et al. Guidelines International Network: Toward International Standards for Clinical Practice Guidelines. Ann Intern Med. 2012;156(7):525–31.

    Article  PubMed  Google Scholar 

  14. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.

    Article  PubMed  Google Scholar 

  15. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ (Clinical research ed). 2016;353:i2016–i.

    Google Scholar 

  16. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical research ed). 2008;336(7650):924–6.

    Article  Google Scholar 

  17. Moberg J, Oxman AD, Rosenbaum S, Schunemann HJ, Guyatt G, Flottorp S, et al. The GRADE Evidence to Decision (EtD) framework for health system and public health decisions. Health Res Policy Syst. 2018;16(1):45.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Goldet G, Howick J. Understanding GRADE: an introduction. J Evid Based Med. 2013;6(1):50–4.

    Article  PubMed  Google Scholar 

  19. The Joanna Briggs Institute. Joanna Briggs Institute Reviewers’ Manual: 2015 edition / Supplement, 2015.

    Google Scholar 

  20. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

    Article  PubMed  Google Scholar 

  21. The NOURISHING-database; World Cancer Research Fund International (WCRF) [Available from: https://www.wcrf.org/int/policy/nourishing-database.

  22. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6.

    Article  PubMed  Google Scholar 

  23. Al-Khudairy L, Loveman E, Colquitt JL, Mead E, Johnson RE, Fraser H, et al. Diet, physical activity and behavioural interventions for the treatment of overweight or obese adolescents aged 12 to 17 years. Cochrane Database Syst Rev. 2017;6:Cd012691.

    PubMed  Google Scholar 

  24. Baker PRA, Francis DP, Soares J, Weightman AL, Foster C. Community wide interventions for increasing physical activity. Cochrane Database Syst Rev. 2015;1.

  25. Balogun OO, O’Sullivan EJ, McFadden A, Ota E, Gavine A, Garner CD, et al. Interventions for promoting the initiation of breastfeeding. Cochrane Database Syst Rev. 2016;11.

  26. Carducci B, Oh C, Keats EC, Gaffey MF, Roth DE, Zulfiqar AB. Protocol: Impact of the food environment on diet-related health outcomes in school-age children and adolescents in low- and middle-income countries: a systematic review. Campbell Collaboration. 2018.

  27. Crockett RA, King SE, Marteau TM, Prevost AT, Bignardi G, Roberts NW, et al. Nutritional labelling for healthier food or non-alcoholic drink purchasing and consumption. Cochrane Database Syst Rev. 2018;2.

  28. Cushing CC, Brannon EE, Suorsa KI, Wilson DK. Systematic review and meta-analysis of health promotion interventions for children and adolescents using an ecological framework. J Pediatr Psychol. 2014;39(8):949–62.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Dobbins M, Husson H, DeCorby K, LaRocca RL. School-based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6 to 18. Cochrane Database Syst Rev. 2013;2.

  30. Dyson PA, Twenefour D, Breen C, Duncan A, Elvin E, Goff L, et al. Diabetes UK evidence-based nutrition guidelines for the prevention and management of diabetes. Diabet Med. 2018;35(5):541–7.

    Article  CAS  PubMed  Google Scholar 

  31. Elvsaas IKO, Giske L, Fure B, Juvet LK. Multicomponent lifestyle interventions for treating overweight and obesity in children and adolescents: a systematic review and meta-analyses. J Obes. 2017;2017:5021902.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Erickson J, Sadeghirad B, Lytvyn L, Slavin J, Johnston BC. The scientific basis of guideline recommendations on sugar intake: a systematic review. Ann Intern Med. 2017;166(4):257–67.

    Article  PubMed  Google Scholar 

  33. Flatz A, Pfeifer N, Radtke T, Kriemler S, Klerings I, Wolfenden L, et al. Interventions implemented through sporting organisations for promoting healthy behaviour or improving health outcomes. Cochrane Database Syst Rev. 2016;5.

  34. Freak-Poli RL, Cumpston M, Peeters A, Clemes SA. Workplace pedometer interventions for increasing physical activity. Cochrane Database Syst Rev. 2013;4:CD009209.

    Google Scholar 

  35. Heise TL, Katikireddi SV, Pega F, Gartlehner G, Fenton C, Griebler U, et al. Taxation of sugar-sweetened beverages for reducing their consumption and preventing obesity or other adverse health outcomes. Cochrane Database Syst Rev. 2016;8.

  36. Heise TL, Romppel M, Molnar S, Buchberger B, Berg A, Gartlehner G, et al. Community gardening, community farming and other local community-based gardening interventions to prevent overweight and obesity in high-income and middle-income countries: protocol for a systematic review. BMJ Open. 2017;7(6):e016237.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Hodder RK, Stacey FG, Wyse RJ, O’Brien KM, Clinton-McHarg T, Tzelepis F, et al. Interventions for increasing fruit and vegetable consumption in children aged five years and under. Cochrane Database Syst Rev. 2017;9.

  38. Hollands GJ, Shemilt I, Marteau TM, Jebb SA, Lewis HB, Wei Y, et al. Portion, package or tableware size for changing selection and consumption of food, alcohol and tobacco. Cochrane Database Syst Rev. 2015;2017(3) (no pagination).

  39. Langford R, Bonell CP, Jones HE, Pouliou T, Murphy SM, Waters E, et al. The WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement. Cochrane Database Syst Rev. 2014;4.

  40. Lhachimi SK, Pega F, Heise TL, Fenton C, Gartlehner G, Griebler U, et al. Taxation of the fat content of foods for reducing their consumption and preventing obesity or other adverse health outcomes. Cochrane Database Syst Rev. 2016;10.

  41. Martin A, Fitzsimons C, Jepson R, Saunders DH, van der Ploeg HP, Teixeira PJ, et al. Interventions with potential to reduce sedentary time in adults: systematic review and meta-analysis. Br J Sports Med. 2015;49(16):1056–63.

    Article  PubMed  Google Scholar 

  42. Matwiejczyk L, Mehta K, Scott J, Tonkin E, Coveney J. Characteristics of effective interventions promoting healthy eating for pre-schoolers in childcare settings: an umbrella review. Nutrients. 2018;10(3):01.

    Article  Google Scholar 

  43. McLaren L, Sumar N, Barberio AM, Trieu K, Lorenzetti DL, Tarasuk V, et al. Population-level interventions in government jurisdictions for dietary sodium reduction. Cochrane Database Syst Rev. 2016;9:CD010166.

    PubMed  Google Scholar 

  44. Mosdol A, Lidal IB, Straumann GH, Vist GE. Targeted mass media interventions promoting healthy behaviours to reduce risk of non-communicable diseases in adult, ethnic minorities. Cochrane Database Syst Rev. 2017;2:CD011683.

    PubMed  Google Scholar 

  45. NICE guideline. Physical activity and the environment March 2018 [Available from: https://www.nice.org.uk/guidance/ng90]. Accessed 25 Nov 2019.

  46. Oakman J, Neupane S, Proper KI, Kinsman N, Nygard CH. Workplace interventions to improve work ability: a systematic review and meta-analysis of their effectiveness. Scand J Work Environ Health. 2018;44(2):134–46.

    PubMed  Google Scholar 

  47. Okely AD, Ghersi D, Hesketh KD, Santos R, Loughran SP, Cliff DP, et al. A collaborative approach to adopting/adapting guidelines - The Australian 24-Hour Movement Guidelines for the early years (Birth to 5 years): an integration of physical activity, sedentary behavior, and sleep. BMC Public Health. 2017;17(Suppl 5):869.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Pfinder M, Katikireddi SV, Pega F, Gartlehner G, Fenton C, Griebler U, et al. Taxation of unprocessed sugar or sugar-added foods for reducing their consumption and preventing obesity or other adverse health outcomes. Cochrane Database Syst Rev. 2016;8.

  49. von Philipsborn P, Stratil JM, Burns J, Busert LK, Pfadenhauer LM, Polus S, et al. Environmental interventions to reduce the consumption of sugar-sweetened beverages and their effects on health. Cochrane Database Syst Rev. 2019;6(6):Cd012292.

    Google Scholar 

  50. Salam RA, Das JK, Hoodbhoy Z, Nathani KR, Z.A. B. Protocol: Effects of lifestyle modification interventions to prevent and manage child and adolescent obesity: a systematic review. Campbell Collaboration. 2018.

  51. Shrestha N, Kukkonen-Harjula KT, Verbeek JH, Ijaz S, Hermans V, Pedisic Z. Workplace interventions for reducing sitting at work. Cochrane Database Syst Rev. 2018;6:Cd010912.

    PubMed  Google Scholar 

  52. Tremblay MS, Leblanc AG, Carson V, Choquette L, Connor Gorber S, Dillman C, et al. Canadian Physical Activity Guidelines for the Early Years (aged 0-4 years), Applied physiology, nutrition, and metabolism = Physiologie appliquee, nutrition et metabolisme. 2012;37(2):345–69.

  53. Tremblay MS, Carson V, Chaput JP, Connor Gorber S, Dinh T, Duggan M, et al. Canadian 24-Hour Movement Guidelines for Children and Youth: An Integration of Physical Activity, Sedentary Behaviour, and Sleep. Appl Physiol Nutr Metab. 2016;41(6 Suppl 3):S311–27.

    Article  PubMed  Google Scholar 

  54. Verweij LM, Coffeng J, van Mechelen W, Proper KI. Meta-analyses of workplace physical activity and dietary behaviour interventions on weight outcomes. Obes Rev. 2011;12(6):406–29.

    Article  CAS  PubMed  Google Scholar 

  55. WHO Guidelines Approved by the Guidelines Review Committee. Guideline: assessing and managing children at primary health-care facilities to prevent overweight and obesity in the context of the double burden of malnutrition: updates for the Integrated Management of Childhood Illness (IMCI). Geneva: World Health Organization Copyright (c) World Health Organization; 2017.

    Google Scholar 

  56. Guideline. Potassium Intake for Adults and Children. Geneva: World Health Organization; 2012. Recommendations and remarks. [Available from: https://www.ncbi.nlm.nih.gov/books/NBK132453/?report=classic]. Accessed 25 Nov 2019.

    Google Scholar 

  57. Guideline. Sodium Intake for Adults and Children. Geneva: World Health Organization; 2012. [Available from: https://www.ncbi.nlm.nih.gov/books/NBK133309/]. Accessed 25 Nov 2019.

    Google Scholar 

  58. Guideline. Sugars Intake for Adults and Children. Geneva: World Health Organization; 2015. [Available from: https://www.ncbi.nlm.nih.gov/books/NBK285537/]. Accessed 25 Nov 2019.

    Google Scholar 

  59. World Health Organization. Countries of Europe [Available from: http://www.euro.who.int/en/countries].

  60. Schwingshackl L, Knuppel S, Schwedhelm C, Hoffmann G, Missbach B, Stelmach-Mardas M, et al. Perspective: NutriGrade: a scoring system to assess and judge the meta-evidence of randomized controlled trials and cohort studies in nutrition research. Adv Nutr (Bethesda, Md). 2016;7(6):994–1004.

    Article  Google Scholar 

  61. JPT H, S G, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011.

  62. Schünemann HJ, Fretheim A, Oxman AD. Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations. Health Res Policy Syst. 2006;4:21.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Schunemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2018.

  64. Sterne JA, Hernan MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ (Clinical research ed). 2016;355:i4919.

    Google Scholar 

  65. Maki KC, Slavin JL, Rains TM, Kris-Etherton PM. Limitations of observational evidence: implications for evidence-based dietary recommendations. Adv Nutr (Bethesda, Md). 2014;5(1):7–15.

    Article  CAS  Google Scholar 

  66. Shim J-S, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. Epidemiol Health. 2014;36:e2014009–e.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Saltelli A, Giampietro M. What is wrong with evidence based policy, and how can it be improved? Futures. 2017;91:62–71.

    Article  Google Scholar 

  68. Moat KA, Lavis JN, Abelson J. How contexts and issues influence the use of policy-relevant research syntheses: a critical interpretive synthesis. Milbank Q. 2013;91(3):604–48.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Liverani M, Hawkins B, Parkhurst JO, et al. PLoS One. 2013;8(10):e77404–e.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M. The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012;12:751.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Busert LK, Mutsch M, Kien C, Flatz A, Griebler U, Wildner M, et al. Facilitating evidence uptake: development and user testing of a systematic review summary format to inform public health decision-making in German-speaking countries. Health Res Policy Syst. 2018;16(1):59.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Petkovic J, Welch V, Jacob MH, Yoganathan M, Ayala AP, Cunningham H, et al. The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: a systematic review. Implement Sci. 2016;11(1):162.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43(3-4):267–76.

    Article  PubMed  Google Scholar 

  74. Lewin S, Hendry M, Chandler J, Oxman AD, Michie S, Shepperd S, et al. Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). BMC Med Res Methodol. 2017;17(1):76.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Petticrew M, Knai C, Thomas J, Rehfuess EA, Noyes J, Gerhardus A, et al. Implications of a complexity perspective for systematic reviews and guideline development in health decision making. BMJ Glob Health. 2019;4(Suppl 1):e000899.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Montgomery P, Movsisyan A, Grant SP, Macdonald G, Rehfuess EA. Considerations of complexity in rating certainty of evidence in systematic reviews: a primer on using the GRADE approach in global health. BMJ Glob Health. 2019;4(Suppl 1):e000848.

    Article  PubMed  PubMed Central  Google Scholar 

  77. Rehfuess EA, Akl EA. Current experience with applying the GRADE approach to public health interventions: an empirical study. BMC Public Health. 2013;13:9.

    Article  PubMed  PubMed Central  Google Scholar 

  78. National Institute for Health and Care Excellence (NICE). Developing NICE guidelines: the manual [Available from: https://www.nice.org.uk/process/pmg20/chapter/introduction-and-overview]. Accessed 9 Dec 2019.

  79. Blake P, Durão S, Naude CE, Bero L. An analysis of methods used to synthesize evidence and grade recommendations in food-based dietary guidelines. Nutr Rev. 2018;76(4):290–300.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Schwingshackl L, Knuppel S, Schwedhelm C, Hoffmann G, Missbach B, Stelmach-Mardas M, et al. Reply to JJ Meerpohl et al. Adv Nutr (Bethesda, Md). 2017;8(5):790–1.

    Article  Google Scholar 

  81. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology. Eval Health Prof. 2006;29(1):126–53.

    Article  PubMed  Google Scholar 

  82. Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, et al. From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decisionmaking. Cost Effect Resource Alloc. 2012;10(1):9.

    Article  Google Scholar 

  83. Gu Y, Lancsar E, Ghijben P, Butler JR, Donaldson C. Attributes and weights in health care priority setting: a systematic review of what counts and to what extent. Soc Sci Med. 2015;146:41–52.

    Article  PubMed  Google Scholar 

  84. Stratil J, Baltussen R, Scheel I, Nacken A, Rehfuess E. Development of the WHO-INTEGRATE evidence-to-decision framework: an overview of systematic reviews of decision criteria for health decision-making. Cost Effectiveness and Resource Allocation. forthcoming.

  85. Youngkong S, Kapiriri L, Baltussen R. Setting priorities for health interventions in developing countries: a review of empirical studies. Tropical Med Int Health. 2009;14(8):930–9.

    Article  Google Scholar 

  86. Elliott H, Popay J. How are policy makers using evidence? Models of research utilisation and local NHS policy making. J Epidemiol Community Health. 2000;54(6):461–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Eva Rehfuess and Peter von Philipsborn from the LMU Munich for their valuable contributions to the manuscript. Furthermore, we thank the information specialist Edith Motschall from the University of Freiburg for her help and expertise in the development of the search strategy.

Funding

The PEN project is funded by the Joint Programming Initiative “A Healthy Diet for a Healthy Life” (JPI HDHL), a research and innovation initiative of EU member states and associated countries. The funding agencies supporting this work are (in alphabetical order of participating countries) France: Institut National de la Recherche Agronomique (INRA); Germany: Federal Ministry of Education and Research (BMBF); Ireland: Health Research Board (HRB); Italy: Ministry of Education, University and Research (MIUR); The Netherlands: The Netherlands Organisation for Health Research and Development (ZonMw); New Zealand: The University of Auckland, School of Population Health; Norway: The Research Council of Norway (RCN); and Poland: The National Centre for Research and Development (NCBR).

Author information

Author notes

  1. Jasmin Zähringer and Lukas Schwingshackl contributed equally to this work.

Authors and Affiliations

  1. Institute for Evidence in Medicine, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany

    Jasmin Zähringer, Lukas Schwingshackl, Daniela Küllenberg de Gaudry & Joerg J. Meerpohl

  2. Institute for Medical Information Processing, Biometry and Epidemiology (IBE), Pettenkofer School of Public Health, LMU Munich, Munich, Germany

    Ani Movsisyan & Jan M. Stratil

  3. Department of Statistical Sciences, University of Bologna, Bologna, Italy

    Sara Capacci

  4. Division of Sports- and Rehabilitation Medicine, Medical Center, Ulm University Hospital, Ulm, Germany

    Jürgen M. Steinacker

  5. Department Prevention and Evaluation, Leibniz-Institute for Prevention Research and Epidemiology – BIPS, Bremen, Germany

    Sarah Forberger & Wolfgang Ahrens

  6. McMaster GRADE Centre and Department of Health Research Methods, Evidence, and Impact, McMaster University Health Sciences Centre, Hamilton, Ontario, Canada

    Holger J. Schünemann

  7. Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany

    Joerg J. Meerpohl

Authors
  1. Jasmin Zähringer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lukas Schwingshackl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ani Movsisyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan M. Stratil
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sara Capacci
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jürgen M. Steinacker
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sarah Forberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wolfgang Ahrens
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Daniela Küllenberg de Gaudry
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Holger J. Schünemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Joerg J. Meerpohl
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

On behalf of the PEN consortium

Contributions

JZ, LS, AM, JMSt, SC, JMS, SF, WA, DKG, HJS, and JJM contributed to the conception and design of the paper. JZ, LS, and JJM were involved in the acquisition and analysis of the data. JZ, LS, and JJM interpreted the results. JZ, LS, and JJM drafted this manuscript. JZ, LS, AM, JMSt, SC, JMS, SF, WA, DKG, HJS, and JJM provided critical revisions and all authors have read and approved the final manuscript. This work is part of the PEN-project (https://www.jpi-pen.eu/), a consortium funded by the European Union, founded to help improve policymaking in the areas of dietary behavior, physical activity, and sedentary behavior by evaluating existing policies. Contact: jpi-pen@leibniz-bips.de.

Corresponding author

Correspondence to Joerg J. Meerpohl.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

JZ: No conflict of interest

LS: Member of GRADE Working Group

AM: Member of GRADE Working Group

JMSt: Developer of the WHO-INTEGRATE EtD framework

CS: No conflict of interest

JMS: No conflict of interest

SF: No conflict of interest

WA: No conflict of interest

DKG: Member of GRADE Working Group

HJS: Co-chair of the GRADE working group; Director, McMaster GRADE Centre

JJM: Member of GRADE Working Group; Director, Freiburg GRADE Center

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR).

Additional file 2.

Search in MEDLINE via Ovid, 4 July 2019.

Additional file 3.

Survey among European health policymakers.

Additional file 4.

WHO – Strengths and recommendations according to GRADE.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zähringer, J., Schwingshackl, L., Movsisyan, A. et al. Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies. Implementation Sci 15, 37 (2020). https://doi.org/10.1186/s13012-020-00984-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-020-00984-2

Keywords

Implementation Science

ISSN: 1748-5908