Table 1 CERQual assessments of methodological limitations in the context of a review finding – Examples

Example 1. No or very minor concerns

A qualitative evidence synthesis examined mistreatment of women during childbirth in medical facilities [11]*. One review finding dealt with women’s preferences: “Women preferred female to male practitioners.” Nine studies contributed to this review finding. All of these studies were assessed as having methodological limitations concerning reflexivity (the individuals collecting and analysing the data were also providing healthcare during childbirth). This body of evidence supporting the review finding was assessed as having no or minor concerns regarding methodological limitations because the dual role of researcher and healthcare provider was not seen to affect this stated preference.

Example 2. Minor concerns

A qualitative evidence synthesis explored parents’ and informal caregivers’ views and experiences regarding communication about childhood vaccinations [8]*. One finding was that “parents liked to receive information about vaccination before the baby was born for reasons such as fatigue and time limitations for reading about vaccination after delivery.” Five studies contributed data to this finding. None of the studies used methods such as triangulation or respondent validation to check the credibility of their findings. The authors concluded that there were “minor concerns regarding methodological limitations due to a lack of discussion by primary authors regarding credibility of the data.”

Example 3. Moderate concerns

Another review finding from the qualitative synthesis examining mistreatment of women during childbirth [11] was considered to be of a sensitive nature since it discussed the women’s bodies and directly criticized specific types of caregivers: “Some women complained of lack of understanding and rough treatment from caregivers, specifically during vaginal and abdominal exams.” Twenty studies contributed data to this review finding. Five studies were assessed as having methodological limitations related to how the data was collected (it is not clear that the authors obtained informed consent) and related to researcher reflexivity (the individuals collecting the data were also providing healthcare during childbirth). An additional fifteen studies were assessed as having methodological limitations only related to the reflexivity of the researcher (the researchers’ role was either unclear, or they were also healthcare providers in maternity wards). The body of evidence contributing to the review finding was assessed as having moderate concerns regarding methodological limitations due to concerns regarding reflexivity –the researchers’ dual role as health providers and caregivers during childbirth was seen as potentiallyhaving an effect on what the women would report afterwards regarding their experiences.

Example 4. Serious concerns

Another finding from the synthesis on communication about childhood vaccinations was that “some parents vaccinated their children because of perceived pressure from the health services” [8]. Seven studies contributed data to this finding. Three of these studies did not describe data collection methods in detail, lacked discussion of researcher reflexivity, and described inappropriate analysis methods (counting). Four studies did not present sufficient data to support the findings, and did not report on how the data was collected or analysed. The authors concluded that there were “serious concerns regarding methodological limitations due to data collection and analysis methods and a lack of researcher reflexivity.”