Skip to main content

Table 3 Summary of the results

From: The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels

Section in the EtD table

Compiled feedback

Order of columns and rows

The order was not a relevant issue for most panels

Criteria in EtD

One methodologist expressed concerns regarding the overlap between the sections “acceptability” and “feasibility” with “values and preferences” and “resource considerations”.

Another methodologist considered the section “priority” redundant.

According to another methodologist, the criteria of the table were not easy to apply to recommendations addressing multiple comparisons.

Judgment column

Two methodologists stated that allowing modifications to the answer options might be needed to accommodate different contexts and scenarios.

Research evidence

Although for the majority of panels there were no issues with this column, one methodologist stated that the difference between the purposes of the columns “available evidence” and “additional consideration” was not clear.

Additional considerations

One methodologist suggested using this column to summarize the available evidence as a general narrative statement.

Background

One methodologist considered this section redundant (with respect to information that is present in the main text of the guideline).

PICO

Three methodologists expressed concerns regarding how the question is presented, specifically, they suggested making the PICO structure more explicit (and using the exact terms that P-I-C-O stand for).

Perspective

One methodologist considered this section particularly relevant, as being explicit about the perspective may help to make transparent the decisions made to formulate recommendations.

Overall certainty of the Evidence

In general, this section was well evaluated by methodologists, with no major suggestion for improvement. Two methodologists made minor wording suggestions.

Another methodologist suggested expanding the content to include more information about each particular outcome.

Values and preferences (“Uncertainty about how much people value the main outcomes”)

This section posed significant difficulties in several panels. Four methodologists suggested differentiating between “variability” and “uncertainty” of patients’ values and preferences.

Another methodologist suggested making explicit the source of patients’ values and preferences. The order of presenting and discussing values and preferences was also questioned and subsequently changed in the EtD framework.

Balance of benefits and harms

There were major difficulties in 6 guideline panels. According to methodologists, panelists had problems answering consistently the questions about the size of the effect.

Additionally, two methodologists considered the questions of this section redundant.

Resource use

Three guideline panels (all without health economists) struggled answering the question about the relationship between incremental cost and benefits. These panels proposed “is the treatment cost-effective?” as a better alternative. The only guideline panel with health economists considered the questions of this section too superficial.

Equity

Three guideline panels struggled with this question. A more clear definition of health equity and more guidance on how to answer the question were considered necessary.

Two methodologists suggested adding the option “no effect on health equity” to the answers options.

Acceptability

Two methodologists expressed problems when trying to identify the relevant stakeholders. More guidance was considered necessary.

Feasibility

None

Panel decisions

One methodologist considered the wording of this section confusing when the recommendation under discussion involved two active treatments.

Another methodologist suggested de-emphasizing the option “no recommendation”

Justification/remarks

Two panels struggled to decide what to include in the remarks. More guidance was considered necessary.

Subgroup considerations

One methodologist considered that subgroups should be more explicit in the table.

Implementation considerations

According to methodologists, this section was used in different ways across guideline panels. More guidance in what to include was considered necessary.

Monitoring and evaluation

None

Research priorities

In general, there was agreement regarding the importance of this section.