Skip to content

Advertisement

  • Research
  • Open Access
  • Open Peer Review

Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist

Implementation Science201813:135

https://doi.org/10.1186/s13012-018-0828-4

  • Received: 26 July 2018
  • Accepted: 18 October 2018
  • Published:
Open Peer Review reports

Abstract

Background

Health practice guidelines (HPGs) are important tools for the translation of evidence into practice. Reporting Items for Practice Guidelines in HealThcare (RIGHT) checklist provides guidance on reporting health practice guidelines (HPGs). We assessed the reporting completeness and quality of a set of national (Croatian) and relevant transnational (European) HPGs.

Methods

The national sample included all HPGs published in the official journal of the Croatian Medical Association in 2014–2016. We searched PubMed to identify relevant European guidelines (n = 24). Two independent reviewers assessed the adherence with the items on the RIGHT checklist. Kappa score was used to measure the level of agreement. Frequentist and Bayes statistics Bayes factor (BF10) was used to evaluate the differences between the national and transnational HPGs.

Results

Overall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0–15.0) RIGHT reporting items, and European counterparts reported a median of 16.0 (95% CI 14.0–17.2) out of the total of 35 checklist items (Mann Whitney U test, P = 0.048; BF10 = 1.543). European HPGs were better than Croatian HPGs in reporting stakeholder involvement and values and preferences (BF10 = 80.63), as well as describing the implications of costs and resources (BF10 = 55.15). Croatian HPGs better reported HPGs specified aims (BF10 = 16.90), primary intended users (BF10 = 8.70), and sources of funding (BF10 = 122.90). Most insufficiently reported items for both HPG sets were defining the guideline questions and clear outcomes, quality assurance, management of funding and conflicts of interest, and guideline limitations.

Conclusions

Important methodological details are missing from most published HPGs at national and transnational levels. To ensure better quality and adequate use of HPGs, reporting guidelines should be endorsed and used by developers and users alike.

Keywords

  • Health practice guidelines
  • Reporting guidelines
  • RIGHT checklist
  • Evidence translation

Background

Health practice guidelines (HPGs) are “statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” [1]. Guidelines that can be trusted are not only based on a systematic review of best available evidence, but they should be developed by a multidisciplinary panel in a transparent, stakeholder inclusive, documented, and accessible process [1].

HPGs have become a standard in clinical practice and their number has been on a continual increase. PubMed search for MeSH terms “Practice Guideline [Publication Type]” or “Practice Guidelines as Topic” returned 126,573 publications on May 2, 2018, with 23,976 publications related to Europe. Despite this high production rate of and interest in HPGs, there is a great variation in their development, quality, implementation, and use, as demonstrated by the recent overview of the framework for HPGs development at the national level in Europe [2]. In many countries, the development of HPGs is tasked to different specialty/professional societies [2], which thus have a central role in translation of evidence to local practice, most often in the local language. The local guidelines are usually published in national journals or on the websites of professional societies, which serve as dissemination outlets for the HPGs.

Quality of reporting has been recognized as a key measure for successful translation of evidence to practice, reducing overall waste in research and helping in eliminating non-replicable studies [3]. Whereas a number of guidelines for reporting research results have been developed for different types of research [4], the quality of reporting of HPGs is not well known, primarily because there have been no tools specially developed for HPGs. Experience from other fields shows that reporting guidelines, such as Consolidated Standards of Reporting Trials (CONSORT) for randomized controlled trials and STARD for diagnostic studies, are effective in improving completeness of reporting in their respective fields [57]. Recently, two reporting checklists have become available for HPGs: Appraisal of Guidelines, Research and Evaluation (AGREE) Reporting Checklist [8] and Reporting Items for Practice Guidelines in HealThcare (RIGHT) [9]. Whereas the AGREE checklist was developed on the concept of the AGREE tools for the methodological rigor and transparency of HPGs [10], the RIGHT checklist follows standard reporting guideline approach established by other reporting statements, such as CONSORT and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [11, 12]. The RIGHT checklist was developed to assist guideline developers in reporting, journal editors and peer reviewers in decision making, and health care practitioners in understanding and implementing guidelines [9]. So far, RIGHT checklist has only been used to evaluate Chinese guidelines for sepsis and for acupuncture [13, 14].

The aim of our study was to assess and compare the quality of reporting in nationally and transnationally produced (European) HPGs by using the RIGHT reporting checklist. We assessed HPGs developed by professional societies in Croatia, a non-English speaking country, and compared them to similar European HPGs produced by transnational professional societies. We included only European HPGs as comparators on the basis of similarity of populations they address, health problems that are explored, and also because Croatian guidelines often rely on, adapt, and cite their European counterparts.

Methods

Study design

We used a cross-sectional design to assess a cohort of published Croatian and relevant European guidelines.

Cohort of Croatian HPGs

In Croatia, most of the guidelines are developed independently by specialty societies gathered around the Croatian Medical Association, except a few nationally developed guidelines, such as ISKRA HPGs concerning the use of antibiotics [15]. HPGs are most often published in Liječnički vjesnik, the official journal of the Croatian Medical Association [16]. For this study, we included all HPGs published in Liječnički vjesnik from 2014 until March 2017: 26 guidelines (last guideline published in 2016).

Search strategy for relevant European HPGs

To identify matching European guidelines, we reviewed the bibliography of included documents and searched PubMed and Google. We used the guideline topic as a search term combined with “Europe” and “guideline or recommendation.” Guideline topic included the clinical condition and intervention mentioned in the Croatian HPGs (e.g., type 2 diabetes pharmacotherapy, primary breast cancer diagnosis). Two authors (RT and MV) independently identified guidelines for inclusion, and the final decision was made by consensus.

Some of the Croatian HPGs covered more than one topic, i.e., clinical condition and/or intervention. In those cases, we searched for all topics covered by the guideline and included all relevant European guidelines.

The search for European guidelines was performed in July 2017. We excluded guidelines that were adapted for non-European populations, global or international guidelines, and those unavailable in English or Croatian.

Data extraction

We used the RIGHT checklist to evaluate the published HPGs. The RIGHT checklist [17] contains 22 requirements organized into 7 sections with a total of 35 items: basic information (6 items), background (8 items), evidence (5 items), recommendations (7 items), review and quality assurance (2 items), funding and declaration and management of interest (4 items), and other information (3 items). Two authors (RT and MV) independently assessed the adherence of HPGs with the RIGHT checklist. In case of conflicting opinions, the third author (AM) was consulted and agreement was reached by consensus. Kappa statistic was used to measure the level of agreement.

Data analysis

Data are presented as the number of RIGHT checklist items reported in each guideline, as well as number of guidelines that reported individual RIGHT checklist items and their proportion, with 95% confidence interval. Due to the non-normality of distributions, we performed Mann-Whitney U test to compare the two groups of HPGs scores overall. For each RIGHT item separately, we calculated the Bayes factor in the contingency table and frequentist chi-square test to evaluate the differences between the two groups. Bayes factor (BF10) was calculated assuming a default prior distribution. P value < 0.05 and BF10 ≥ 3 were considered significant [18]. In cases of conflicting frequentist and Bayesian results of significance, we considered the results of Bayes statistics for final interpretation. Calculations were done using MedCalc Statistical Software version 16.4.3 (MedCalc Software bvba, Ostend, Belgium; https://www.medcalc.org; 2016), and JASP software, version 0.8.6 (https://jasp-stats.org/).

Results

We reviewed all published Croatian HPGs (n = 26) and 24 matching European HPGs. Eleven Croatian HPGs did not have European counterparts, and 3 of them had multiple European counterparts. Because of these discrepancies, we did not do a matched analysis. Kappa coefficient for individual 35 items of RIGHT checklist ranged from 0.76 to 1.0. All of the European guidelines were produced by transnational specialty societies, and most of Croatian guidelines were produced by national specialty societies (23 out of 26; 3 were produced by individual authors, independently from specialty societies; 1 of those declared to be supported by professional societies). All of the guidelines were produced or updated in 2014 or later. The full list of guidelines and individual results of the adherence to the RIGHT checklist are available in the Additional file 1.

Overall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0–15.0) RIGHT reporting items. Their European counterparts reported a median of 16.0 (95% CI 14.0–17.2) out of the total of 35 checklist items. The difference was not significant (P = 0.048, Mann-Whitney U test, BF10 = 1.543). The lowest total number of addressed RIGHT items was 11 (31%) (Croatian HPGs on melanoma and cystine urolithiasis), and the highest was 22 (63%, European resuscitation guideline).

The results of adherence to individual RIGHT checklist items are presented in Tables 1, 2, and 3. For nine checklist items, both groups of HPGs had a satisfactory level of reporting (≥ 80% of HPGs in each group adhered to the checklist item): item 1a—clear guideline titles, described as mentioning the focus of the guideline; item 1c—defining it as guidelines or recommendations in the title; item 4—providing an address of a corresponding author; item 9a—lists of authors and their affiliations; item 5description of basic epidemiology of health problems; item 7a—description of primary populations addressed by the HPG; item 13a—clear and actionable recommendations; item 13b—separate recommendations for important subgroups; and item 13c—indicated strengths of recommendations and certainty of evidence.
Table 1

Adherence of health practice guidelines (HPGs) to RIGHT checklist items addressing the domains of basic information and background (total number and percentage with 95% confidence intervals (CI) of addressed items)

RIGHT checklist item

Croatian HPGs (n = 26)

European HPGs (n = 24)

Pa

BF10b

Domain 1: Basic information

Title/subtitle

Total number (%, 95% CI)

 

 1a—Identify the report as a guideline, that is, with “guideline(s)” or “recommendation(s)” in the title.

26 (100.0, 87.3–100.0)

22 (91.7, 74.2–97.6)

0.130

0.34

 1b—Describe the year of publication of the guideline.

1 (3.8, 2.0–21.6)

3 (12.5, 4.0–31.0)

0.260

0.33

 1c—Describe the focus of the guideline, such as screening, diagnosis, treatment, management, prevention or others.

26 (100.0, 87.3–100.0)

21 (87.5, 69.0–95.7)

0.063

0.73

Executive summary

 2—Provide a summary of the recommendations contained in the guideline.

10 (38.4, 20.9–59.3)

17 (70.8, 50.8–85.1)

0.022

4.44

Abbreviations and acronyms

 3—Define new or key terms, and provide a list of abbreviations and acronyms if applicable.

4 (15.3, 5.0–35.7)

4 (16.7, 6.7–35.9)

0.902

0.26

Corresponding developer

 4—Identify at least one corresponding developer or author who can be contacted about the guideline.

26 (100.0, 87.3–100.0)

23 (95.8, 79.8–99.3)

0.293

0.16

Domain 2: Background

Brief description of the health problem(s)

 5—Describe the basic epidemiology of the problem, such as the prevalence/incidence, morbidity, mortality, and burden (including financial) resulting from the problem.

23 (88.4, 68.7–96.9)

22 (91.7, 74.2–97.6)

0.706

0.22

Aim(s) of the guideline and specific objectives

 6—Describe the aim(s) of the guideline and specific objectives, such as improvements in health indicators (e.g., mortality and disease prevalence), quality of life, or cost savings.

20 (76.9, 55.9–90.2)

9 (37.5, 21.2–57.3)

0.005

16.90

Target population(s)

 7a—Describe the primary population(s) that is addressed by the recommendation(s) in the guideline.

22 (84.6, 64.3–94.9)

22 (91.7, 74.2–97.6)

0.443

0.29

 7b—Describe any subgroups that are given special consideration in the guideline.

20 (76.9, 55.9–90.2)

15 (62.5, 42.7–78.8)

0.266

0.57

End-users and settings

 8a—Describe the intended primary users of the guideline (such as primary care providers, clinical specialists, public health practitioners, program managers, and policy-makers) and other potential users of the guideline.

12 (46.2, 27.1–66.3)

3 (12.5, 4.0–31.0)

0.009

8.70

 8b—Describe the setting(s) for which the guideline is intended, such as primary care, low- and middle-income countries, or in-patient facilities.

8 (30.7, 15.1–51.9)

4 (16.7, 6.7–35.9)

0.243

0.55

Guideline development groups

 9a—Describe how all contributors to the guideline development were selected and their roles and responsibilities (e.g., steering group, guideline panel, external reviewer, systematic review team, and methodologists).

4 (15.3, 0.5 to 35.7)

3 (12.5, 4.0–31.0)

0.769

0.25

 9b—List all individuals involved in developing the guideline, including their title, role(s) and institutional affiliation(s).

26 (100.0, 87.3–100.0)

23 (95.8, 79.8–99.3)

0.293

0.16

aFrequentist chi-square test

bBayesian 2 × 2 contingency table

Table 2

Adherence of health practice guidelines (HPGs) to RIGHT checklist items addressing the domains of evidence and recommendations (total number and percentage with 95% confidence intervals (CI) of addressed items)

RIGHT checklist item

Croatian HPGs (n = 26)

European HPGs (n = 24)

Pa

BF10b

Domain 3: Evidence

Healthcare questions

Total number (%, 95% CI)

 

 10a—State the key questions that were the basis for the recommendations in PICO (population, intervention, comparator, and outcome) or other format as appropriate.

1 (3.8, 0.2–21.6)

0 (0.0, 0.0–13.8)

0.332

0.15

 10b—Indicate how the outcomes were selected and sorted.

1 (3.8, 0.2–21.6)

0 (0.0, 0.0–13.8)

0.332

0.14

Systematic reviews

 11a—Indicate whether the guideline is based on new systematic reviews done specifically for this guideline or whether existing systematic reviews were used.

3 (11.5, 3.0–31.3)

3 (12.5, 4.0–31.0)

0.917

0.23

 11b—If the guideline developers used existing systematic reviews, reference these and describe how those reviews were identified and assessed (provide the search strategies and the selection criteria, and describe how the risk of bias was evaluated) and whether they were updated.

0 (0.0, 0.0–16.2)

3 (12.5, 4.0–31.0)

0.063

0.73

Assessment of the certainty of the body of evidence

 12—Describe the approach used to assess the certainty of the body of evidence.

19 (73.1, 51.9–87.7)

18 (75.0, 55.1–88.0)

0.877

0.30

Domain 4: Recommendations

Recommendations

 13a—Provide clear, precise, and actionable recommendations.

26 (100.0, 87.3–100.0)

24 (100.0, 86.2–100.0)

0.777

NA

 13b—Present separate recommendations for important subgroups if the evidence suggests that there are important differences in factors influencing recommendations, particularly the balance of benefits and harms across subgroups.

25 (96.2, 78.4–99.8)

22 (91.7, 74.2–97.6)

0.504

0.21

 13c—Indicate the strength of recommendations and the certainty of the supporting evidence.

21 (80.7, 60.2–92.7)

21 (87.5, 69.0–95.7)

0.517

0.31

Rationale/explanation for recommendations

 14a—Describe whether values and preferences of the target population(s) were considered in the formulation of each recommendation. If yes, describe the approaches and methods used to elicit or identify these values and preferences. If values and preferences were not considered, provide an explanation.

2 (7.7, 1.3–26.6)

12 (50.0, 31.4–68.6)

< 0.001

80.63

 14b—Describe whether cost and resource implications were considered in the formulation of recommendations. If yes, describe the specific approaches and methods used (such as cost effectiveness analysis) and summarize the results. If resource issues were not considered, provide an explanation.

0 (0.0, 0.0–16.2)

8 (33.3, 17.9–53.3)

< 0.001

55.15

 14c—Describe other factors taken into consideration when formulating the recommendations, such as equity, feasibility and acceptability.

2 (7.7, 1.3–26.6)

6 (25.0, 12.0–44.9)

0.095

0.93

Evidence to decision processes

 15—Describe the processes and approaches used by the guideline development group to make decisions, particularly the formulation of recommendations (such as how consensus was defined and achieved and whether voting was used).

9 (34.6, 17.9–55.6)

4 (16.7, 6.7–35.9)

0.148

0.81

NA not applicable

aFrequentist chi square test

bBayesian 2 × 2 contingency table

Table 3

Adherence of health practice guidelines (HPGs) to RIGHT checklist items addressing the domains of review and quality assurance, funding, declaration and management of interest, and other information (total number and percentage with 95% confidence intervals (CI) of addressed items)

RIGHT checklist item

Croatian HPGs (n = 26)

European HPGs (n = 24)

Pa

BF10b

Domain 5: Review and quality assurance

External review

Total number (%, 95% CI)

 

 16—Indicate whether the draft guideline underwent independent review and, if so, how this was executed and the comments considered and addressed.

1 (3.8, 0.2–21.6)

10 (41.7, 24.5–61.2)

< 0.001

55.35

Quality assurance

 17—Indicate whether the guideline was subjected to a quality assurance process. If yes, describe the process.

1 (3.8, 0.2–21.6)

3 (12.5, 4.0–31.0)

0.260

0.33

Domain 6: Funding, declaration and management of interest

Funding source(s) and role(s) of the funder

 18a—Describe the specific sources of funding for all stages of guideline development.

18 (69.2, 48.1–84.9)

5 (20.8, 9.2–40.5)

< 0.001

122.90

 18b—Describe the role of funder(s) in the different stages of guideline development and in the dissemination and implementation of the recommendations.

2 (7.7, 1.3–26.6)

2 (8.3, 2.3–25.9)

0.933

0.19

Declaration and management of interest

 19a—Describe what types of conflicts (financial and non-financial) were relevant to guideline development.

2 (7.7, 1.3–26.6)

22 (91.7, 74.2–97.6)

< 0.001

1.02 × 108

 19b—Describe how conflicts of interest were evaluated and managed and how users of the guideline can access the declarations.

1 (3.8, 0.2–21.6)

1 (4.2, 0.7–20.2)

0.954

0.15

Domain 7: Other information

Access

 20—Describe where the guideline, its appendices, and other related documents can be accessed.

1 (3.8, 0.2–21.6)

3 (12.5, 4.0–31.0)

0.260

0.33

Suggestions for further research

 21—Describe the gaps in the evidence and/or provide suggestions for future research.

12 (46.2, 27.1–66.3)

23 (95.8, 79.8–99.3)

< 0.001

733.7

Limitations of the guideline

 22—Describe any limitations in the guideline development process (such as the development groups were not multidisciplinary or patients’ values and preferences were not sought), and indicate how these limitations might have affected the validity of the recommendations.

1 (3.8, 0.2–21.6)

1 (4.2, 0.7–20.2)

0.954

0.15

aFrequentist chi square test

bBayesian 2 × 2 contingency table

For 12 out of 35 RIGHT items, both sets of HPGs had poor reporting (≤ 20% of HPGs in each group adhered to the checklist item): item 9a—description of roles and responsibilities of authors and contributors; item 10a—a clear and structured healthcare question in PICO or other appropriate format; item 10b—clearly defined outcomes; items 11a and 11b—process of selection of systematic reviews; item 17—description of quality assurance process; item 18b—reporting on the roles of funding bodies in the development; item 19b—management of conflicts of interest; item 22—description of limitations of the guidelines, such as risk of bias, conflicts of interest or lack of patient involvement; and item 20—availability of guidelines and additional documents.

European guidelines were significantly better in providing a HPG summary (item 2; BF10 = 4.44), reporting stakeholder involvement and values and preferences (item 14a; BF10 = 80.63), as well as describing the implications of costs and resources (item 14b; BF10 = 55.15, Table 2). They also reported better than Croatian HPGs on existing conflicts of interest (item 16; BF10 = 55.35), external reviews (item 19a; BF10 = 1.02 × 108), and on sufficiency of evidence for recommendations or provided suggestions for future research (item 21; BF10 = 733.70).

Croatian HPGs were significantly better in reporting specified aims of the guideline (item 6, BF10 = 16.90), their primary intended users (item 8a, BF10 = 8.70), and sources of funding (item 18a, BF10 = 122.90).

For item 12 (description of the approach used to assess the certainty of the body of evidence), the Croatian and European HPGs sets had similar adherence (73.1%, 95% CI 51.9–87.7 for Croatian, and 75.0%, 95% CI 55.1–88.0 for European HPGs; BF10 = 0.30). We also extracted the information on what sort of approach was used in the HPGs. Grading of Recommendations, Assessment, Development and Evaluation (GRADE) handbook [19] and Oxford Centre for Evidence Based Medicine levels of evidence [20] were predominant approaches in Croatian HPGs (53.8% of HPGs), but were less frequent in the European HPGs set (16.7% of HPGs). European HPGs mostly mentioned adapted Infectious Diseases Society of America–United States Public Health Service Grading System [21] (45.8% of HPGs). Other grading systems, such as American College of Chest Physicians (ACCP) modified GRADE system [22] or Scottish Intercollegiate Guidelines Network (SIGN) guide [23], were mentioned in 7.7% of Croatian and 12.50% of European HPGs. Some HPGs were unclear about the grading systems, mentioning the levels of evidence but not referencing any grading systems (23.1% of Croatian and 16.7% of European HPGs). Finally, 15.4% of Croatian and 8.3% of European HPGs did not mention or imply the use of any grading systems for levels of evidence.

Discussion

Our study showed that both national and transnational HPGs were poorly reported in journals, with adequate presentation for fewer than 50% of necessary reporting items. This is, to the best of our knowledge, the first evaluation of HPGs in Europe using the RIGHT reporting checklist, developed for guidance in reporting guidelines in healthcare, including clinical and public health areas. It is important to keep in mind that the RIGHT checklist is not a guide for development nor a tool for assessment of methodological quality of HPGs, but a tool for checking the clarity of presentation of the HPGs to a wide audience, including both developers and readers, so that HPGs could be adequately understood and implemented in practice [9].

Our finding of inadequate reporting of HPGs matches previous observations of poor methodological quality of HPGs produced by specialty societies [24, 25]. There was no difference between the quality of reporting of national HPGs, which are directly implemented in practice in individual health care systems, and those produced by transnational societies, which put emphasis on multidisciplinary collaboration and have standard operating procedures for guideline development [26].

The relatively small sample size is the most important limitation of our study. The sample size was determined by the cohort of available Croatian HPGs. We did not include HPGs published earlier than 2014 as they were not numerous and their presentation was of poor quality (our unpublished observation). The small sample size was also the reason why we based the interpretation of results on the results of Bayes and not frequentist statistics. The advantages of coherency, independency of the intention with which data are collected, and minimum bias toward null hypothesis made Bayesian inference valid for small sample sizes [27]. It is also important to keep in mind that we analyzed national HPGs in a single country, so the results may not be generalizable to the whole population of nationally produced guidelines. However, the similarity of findings for Croatian and matching transnational guidelines indicates that the level of reporting quality is similar across professional societies in Europe. Finally, it is important to keep in mind that the RIGHT checklist does not evaluate the validity of the actual recommendations in the HPGs, but only the quality of their publicly available presentation.

The analysis of compliance with individual reporting items demonstrated that there was an overall insufficient emphasis on providing clear aims and a summary of recommendations. Guideline developers failed to provide clear instructions on how the guidelines are intended to be used and by whom. In this way, basic principles of evidence-based medicine had not been addressed because there was little information on the users of HPGs, the purpose of HPGs, and acknowledging barriers and constraints that may limit their implementation [28].

Both national and transnational sets of HPGs adequately reported on strengths of recommendations and certainty of supporting evidence, but details on how the evidence was searched for and evaluated were unsatisfactory. Established grading systems, such as GRADE and Oxford EBM, were used in half or less than a half of the analyzed HPGs, and 22% of HPGs were unclear about grading evidence. This means that current HPGs do not satisfy the preference of clinicians for recommendations that have clearly presented processes behind them and provide evidence summaries [29, 30]. Clear and transparent reporting should also help address problems such as relying on lower levels of evidence than stated in HPGs [31], relying on low levels of evidence to form recommendations [32], and inadequate presentation of evidence grading [33].

In national HPGs, reporting on the inclusion of patient values and preferences in the making of recommendations was poor, and reporting cost and resource considerations was almost non-existent. This reflects generally unsatisfactory involvement of the patients in guideline development [34], as well as integration of evidence on patient preferences [35], even though the involvement of patients and other stakeholders is especially important for development and adaptation of HPGs oriented at lower- and middle-income populations, which often have limited resources and different patient experiences and priorities [36, 37].

We also found that reporting of conflicts of interest (COI) was worse in Croatian than in European HPGs, but that both groups of guidelines showed low adherence to providing declarations on the conflicts of interest and elaborating policies regarding their management. Available data show that the prevalence of conflicts of interest in guideline development is high [38], and that conflicts of interest can influence interpretation and recommendations of treatments [39], as well as definitions of diseases [40]. Croatia is a country that follows the practice of the European Federation of Pharmaceutical Industries and Associations (EFPIA) Code for disclosure of financial transfers [41], but does not have a national legislation that requires disclosure on a central platform, such as Denmark [42]. Even in systems with mandatory conflict of interest declaration, like Denmark, there is a discrepancy between disclosures in the national register and self-reporting in publications [42]. In Croatia, only 11% of physicians made their individual data available in 2016, when the first report from the Innovative Pharmaceutical Initiative, an association that gathers 33 pharmaceutical companies that cover 60% of Croatian pharmaceutical market, was published [43].

Conclusion

Our study provides evidence to recommend the use of reporting checklists to guideline developers both at national and transnational levels. A variety of tools have been developed to help translation of evidence to reliable and high-quality recommendations, including GRADE system for rating quality of evidence [44], AGREE tools for guideline methodology [10], checklists for development, implementation, evaluation, and quality [45, 46]. Without complete and transparent reporting, there is no way of knowing how good and how appropriate HPGs are for use in specific clinical circumstances. In times of overwhelming production of guidelines, which are supposed to help the end-user in finding and summarizing the best evidence [47], it is particularly important to achieve clarity and enable informed decisions. In future research, the usefulness of RIGHT reporting guideline should be tested in actual health practice settings and they should be compared with similar reporting guidelines, such as AGREE Reporting Checklist [8]. Endorsement of reporting guidelines for HPGs, such as RIGHT, by professional associations, health websites, and journals would contribute to the effort of transparency for developers and to the aim of adequate implementation for users of guidelines. Only then can we hope to achieve the goal of full transparency, comprehension, and availability of information that concerns a variety of stakeholders.

Abbreviations

AGREE: 

Appraisal of Guidelines for Research and Evaluation

CONSORT: 

Consolidated Standards of Reporting Trials

GRADE: 

Grading of Recommendations Assessment, Development and Evaluation

HPG: 

Health practice guideline

PRISMA: 

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RIGHT: 

Reporting Items for Practice Guidelines in HealThcare

SIGN: 

Scottish Intercollegiate Guidelines Network

STARD: 

Standards for Reporting of Diagnostic Accuracy Studies

Declarations

Acknowledgements

Not applicable.

Funding

This study was funded by the Croatian Science Foundation, grant No. IP-2014-09-7672 “Professionalism in Health Care.” The funder had no role in the design of this study, during its execution and data interpretation.

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

AM and RT designed the study. RT and MV extracted the data. IB and RT performed data analysis. RT drafted the manuscript. All authors made intellectual contributions to the manuscript and approved the final version.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

AM declares that she contributed to the development of the RIGHT guideline. RT, MV, and IB declare that they have no conflicts of interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia

References

  1. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. In: Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011. Available from: https://www.ncbi.nlm.nih.gov/books/NBK209539/. doi: 10.17226/13058.Google Scholar
  2. Legido-Quigley H, Panteli D, Brusamento S, Knai C, Saliba V, Turk E, et al. Clinical guidelines in the European Union: mapping the regulatory basis, development, quality control, implementation and evaluation across member states. Health policy. 2012;107(2–3):146–56. Epub 2012/09/04View ArticleGoogle Scholar
  3. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76. Epub 2014/01/15View ArticleGoogle Scholar
  4. Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR network. BMC Med. 2010;8:24. Epub 2010/04/28View ArticleGoogle Scholar
  5. To MJ, Jones J, Emara M, Jadad AR. Are reports of randomized controlled trials improving over time? A systematic review of 284 articles published in high-impact general and specialized medical journals. PLoS One. 2013;8(12):e84779. Epub 2014/01/07View ArticleGoogle Scholar
  6. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology. 2006;67(5):792–7. Epub 2006/09/13View ArticleGoogle Scholar
  7. Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285(15):1992–5. Epub 2001/04/20View ArticleGoogle Scholar
  8. Brouwers MC, Kerkvliet K, Spithoff K. The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ. 2016;352:i1152. Epub 2016/03/10View ArticleGoogle Scholar
  9. Chen Y, Yang K, Marusic A, Qaseem A, Meerpohl JJ, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med. 2017;166(2):128–32. Epub 2016/11/29View ArticleGoogle Scholar
  10. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421–4.View ArticleGoogle Scholar
  11. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. Epub 2010/03/25View ArticleGoogle Scholar
  12. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. Epub 2009/07/22View ArticleGoogle Scholar
  13. Chen RB, Chen YL, Gai GZ, Xie YM, Wang YY. Analysis of reporting specification about sepsis practice guideline based on RIGHT standard. Zhongguo Zhong yao za zhi. 2017;42(8):1514–7. Epub 2017/10/27PubMedGoogle Scholar
  14. Xiaoyang S, Lu G, Ru L. Assessment of reporting quality of guidelines on acupuncture and moxibustion by using RIGHT Statement. Chinese Medical Doctor Association Annual Conference on Evidence-based Medicine 2017.Google Scholar
  15. Intersectoral Coordination Mechanism for the Control of Antimicrobial Resistance (ISKRA). Croatian National Guidelines. Available from: http://www.iskra.bfm.hr/eng/Guidlines.aspx?id=61Google Scholar
  16. Liječnički vjesnik - glasilo Hrvatskoga Liječničkog Zbora. Available from: https://lijecnicki-vjesnik.hlz.hr/. Accessed 28 May 2018.
  17. RIGHT working group. RIGHT Checklist. Available from: http://www.right-statement.org/right-statement/checklist. Accessed 28 May 2018.Google Scholar
  18. Jeffreys H. Theory of probability. Oxford: Oxford University Press; 1961.Google Scholar
  19. Schünemann H BJ, Guyatt G, Oxman A, editors. GRADE handbook for grading quality of evidence and strength of recommendations. Updated October 2013. Available from: http://gdt.guidelinedevelopment.org/app/handbook/handbook.html Accessed 28 May 2018.
  20. Howick J. Levels of evidence. Oxford: Centre for Evidence Based Medicine; 2009. Available from: http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/ Google Scholar
  21. Dykewicz CA. Summary of the guidelines for preventing opportunistic infections among hematopoietic stem cell transplant recipients. Clin Infect Dis. 2001;33(2):139–44.View ArticleGoogle Scholar
  22. Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest. 2006;129(1):174–81. Epub 2006/01/21View ArticleGoogle Scholar
  23. SIGN guidelines—an introduction to SIGN methodology for the development of evidence-based clinical guidelines. SIGN Publication No. 39, SIGN Secretariat, Royal College of Physicians of Edinburgh. Edinburgh 1999.Google Scholar
  24. Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000;355(9198):103–6. Epub 2000/02/16View ArticleGoogle Scholar
  25. Tudor KI, Kozina PN, Marusic A. Methodological rigour and transparency of clinical practice guidelines developed by neurology professional societies in Croatia. PLoS One. 2013;8(7):e69877. Epub 2013/07/31View ArticleGoogle Scholar
  26. ESMO Guidelines Methodology. Available from: http://www.esmo.org/Guidelines/ESMO-Guidelines-Methodology. Accessed 28 May 2018.Google Scholar
  27. Wagenmakers EJ, Marsman M, Jamil T, Ly A, Verhagen J, Love J, et al. Bayesian inference for psychology. Part I: theoretical advantages and practical ramifications. Psychon Bull Rev. 2018;25(1):35–57. Epub 2017/08/06View ArticleGoogle Scholar
  28. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725.View ArticleGoogle Scholar
  29. Brandt L, Vandvik PO, Alonso-Coello P, Akl EA, Thornton J, Rigau D, et al. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. 2017;7(2):e011569.View ArticleGoogle Scholar
  30. Neumann I, Alonso-Coello P, Vandvik PO, Agoritsas T, Mas G, Akl EA, et al. Do clinicians want recommendations? A multicenter study comparing evidence summaries with and without GRADE recommendations. J Clin Epidemiol. 2018;99:33–40. Epub 2018/03/14View ArticleGoogle Scholar
  31. Lee D, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of america practice guidelines. Arch Intern Med. 2011;171(1):18–22.View ArticleGoogle Scholar
  32. Alexander PE, Brito JP, Neumann I, Gionfriddo MR, Bero L, Djulbegovic B, et al. World Health Organization strong recommendations based on low-quality evidence (study quality) are frequent and often inconsistent with GRADE guidance. J Clin Epidemiol. 2016;72:98–106. Epub 2015/01/27View ArticleGoogle Scholar
  33. Neumann I, Brignardello-Petersen R, Wiercioch W, Carrasco-Labra A, Cuello C, Akl E, et al. The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels. Implement Sci. 2016;11:93. Epub 2016/07/16View ArticleGoogle Scholar
  34. Armstrong MJ, Bloom JA. Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies. Res Involv Engagem. 2017;3:19. Epub 2017/10/25View ArticleGoogle Scholar
  35. Chong CA, Chen IJ, Naglie G, Krahn MD. How well do guidelines incorporate evidence on patient preferences? J Gen Intern Med. 2009;24(8):977–82. Epub 2009/04/24View ArticleGoogle Scholar
  36. Dizon JM, Machingaidze S, Grimmer K. To adopt, to adapt, or to contextualise? The big question in clinical practice guideline development. BMC Res Notes. 2016;9(1):442. Epub 2016/09/15View ArticleGoogle Scholar
  37. Gonzalez-Suarez C, Grimmer-Somers K, Dizon J, King E, Lorenzo S, Valdecanas C, et al. Contextualizing Western guidelines for stroke and low back pain to a developing country (Philippines): an innovative approach to putting evidence into practice efficiently. J Healthc Leadersh. 2012;4:141–56.View ArticleGoogle Scholar
  38. Norris SL, Holmer HK, Ogden LA, Burda BU. Conflict of interest in clinical practice guideline development: a systematic review. PLoS One. 2011;6(10):e25153. Epub 2011/11/01View ArticleGoogle Scholar
  39. Wang AT, McCoy CP, Murad MH, Montori VM. Association between industry affiliation and position on cardiovascular risk with rosiglitazone: cross sectional systematic review. BMJ. 2010;340:c1344. Epub 2010/03/20View ArticleGoogle Scholar
  40. Moynihan RN, Cooke GP, Doust JA, Bero L, Hill S, Glasziou PP. Expanding disease definitions in guidelines and expert panel ties to industry: a cross-sectional study of common conditions in the United States. PLoS Med. 2013;10(8):e1001500. Epub 2013/08/24View ArticleGoogle Scholar
  41. EFPIA. EFPIA code on disclosure of transfers of value from pharmaceutical companies to healthcare professionals and healthcare organizations. Consolidated version, June 2014. Available from: https://efpia.eu/media/25837/efpia-disclosure-code.pdf. Accessed 28 May 2018.
  42. Rasmussen K, Schroll J, Gotzsche PC, Lundh A. Under-reporting of conflicts of interest among trialists: a cross-sectional study. J R Soc Med. 2015;108(3):101–7. Epub 2014/11/13View ArticleGoogle Scholar
  43. IFI. Innovative pharmaceutical initiative: public announcement 2016. Available from: http://ifi.hr/en/public-announcement-2016/. Accessed 28 May 2018.Google Scholar
  44. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ. 2008;336(7652):1049–51.View ArticleGoogle Scholar
  45. Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123–42. Epub 2013/12/18View ArticleGoogle Scholar
  46. Grimmer K, Dizon JM, Milanese S, King E, Beaton K, Thorpe O, et al. Efficient clinical evaluation of guideline quality: development and testing of a new tool. BMC Med Res Methodol. 2014;14:63. Epub 2014/06/03View ArticleGoogle Scholar
  47. Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J, et al. Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6. Epub 2013/01/11View ArticleGoogle Scholar

Copyright

© The Author(s). 2018

Advertisement