Table 5 Practical guidance on making an overall CERQual assessment for a review finding

• Each overall CERQual assessment should ideally be made through discussion among the review authors. This process may also involve consulting with an expert group for a synthesis

• Using the CERQual Evidence Profile, look across the assessments you have made for each CERQual component. Note particularly any components for which you have serious concerns

• Decide whether you will ‘rate down’ (i.e., lower the level of confidence in the review finding) at all for the concerns identified and, if so, whether you will rate down by one or two levels. When making this overall assessment, consider the following:

 - Typically, the overall assessment of confidence should be rated down by at least one level for each component for which you have identified serious concerns

 - Where concerns in relation to a component are minor or moderate, it may not be necessary to rate down. However, if there are a number of such concerns, it may be appropriate to rate down by one level to represent two or more of these concerns

• When making a judgement on whether to ‘rate down’, also consider the following:

 - To some extent, the importance of concerns regarding a CERQual component needs to be judged in relation to the review finding. For instance, where a finding represents ‘mid-level’ theory regarding a phenomenon, it may be important that this is backed by considerable data and that the fit between the data from the primary studies and the review finding is clear. Concerns regarding adequacy of data and coherence may therefore be particularly critical in making an overall CERQual assessment for this finding

 - The data contributing to a review finding may come from studies that are assessed to have different levels of concern in relation to a CERQual component. This variation can be captured in three ways: (1) make a judgment that captures the highest level of concern for the component; (2) make a judgment that captures the lowest level of concern for the component; or (3) make a judgment that captures the “middle ground” for the component.

• For example, a synthesis of parents’ views and experiences of communication for childhood vaccination includes a finding that parents felt that the information that they received about vaccination was unbalanced or one sided. This finding was based on three ethnographic studies, two assessed as having no or very minor concerns and one assessed as having moderate concerns regarding methodological limitations; and two focus group studies, both assessed as having moderate concerns regarding methodological limitations. The synthesis authors decide to give an umbrella assessment of ‘moderate concerns’ for the methodological limitations component

 - The initial assessment of coherence may prompt changes in the way that a review finding is conceptualised and described if, for example, it becomes clear that the finding would make more sense as two separate findings. Where this occurs, the assessments for all of the CERQual components may then need to be revisited for the finding/s

• Recommended standard phrases for describing the assessment for each CERQual component and the overall assessment are provided in Additional file 4

• You should strive to be consistent across review findings in a synthesis in assessing the extent of concerns regarding each CERQual component. Consistency across syntheses is harder to achieve and it is more important to be explicit and transparent regarding judgements so that the rationale for these are clear to users