Systematic review step | Possible “shortcuts” | Potential impact on the validity of the results | Relevant AMSTAR question and potential impact of shortcut on AMSTAR score |
---|---|---|---|
Preparation of a protocol | • Omit protocol | Unknown | Q1. Loss of one point if a protocol is not prepared and/or not mentioned in report |
Question formulation | • Limit the number of questions and sub-questions • Limit the scope of the question/s | None expected | |
Selecting relevant studies | • One reviewer screens titles and abstracts • One reviewer screens full text | Unknown, though one reviewer could miss up to 9 % of eligible randomized controlled trials [42] | Q2. Loss of one point if only one reviewer does screening and/or only one reviewer does data extraction |
Data extraction | • One reviewer extracts data | Can increase the number of errors but the impact on results is not known [43–45] | |
• One reviewer extracts data with checking by a second reviewer | Unknown | ||
• Data extraction limited to key characteristics, results, conflicts of interest | Unknown | ||
Literature search | • Limit number of databases searched • Limit or omit hand searching of references lists and relevant journals • Eliminate consultation with experts to find additional studies | Limiting the number of databases searched can increase efficiency without compromising validity [46–51], especially if combined with some hand searching and contact with experts [52–57] | Q3. Loss of one point if less than two databases searched and/or no supplementary strategies |
Inclusion criteria | |||
Gray literature | • Limit or omit gray literature | Could introduce publication bias but the evidence is mixed [16, 58–60] | Q4. Loss of one point if gray literature omitted |
Language | • English only | ||
Dates | • Narrow time frame, e.g., last 5 or 10 years | None expected | |
Study types | • Restrict study types to systematic reviews (and economic evaluations) • Restrict study types to randomized controlled trials or controlled clinical trials (and economic evaluations) | ||
Quality assessment | • Limit or omit quality assessment | Not recommended. Several authors suggest that, where resources are limited, priority should be given to quality assessment rather than extensive searching [51, 73] | Q7 and Q8. Loss of two points if not assessed, documented and used in formulation of conclusions |
• Omit “a priori” specification • Done by one reviewer | Unknown | ||
Data synthesis | • Narrative synthesis only (no meta-analysis) | Unknown – meta-analysis can increase power and precision but also has potential to mislead if not applied appropriately and done correctly [19] | Q9. None if explained that meta-analysis not possible due to heterogeneity. If not, loss of one point |
Assessment of publication bias | • Omit | Unknown | Q10. Loss of one point if omitted |
Assessment of conflict of interest | • Omitted for individual studies and/or for systematic review | Unknown | Q11. Loss of one point if omitted |
Report | • Information included limited | Unknown but can impact on AMSTAR score if insufficient detail of methods provided to enable a quality assessment. Sufficient detail of methods will help the reviewer to assess the validity of the results [17, 18, 23] | Q1–11. Potential large loss of points if key AMSTAR questions not covered |
External peer review | • Omit or limit | Unknown |