A systematic review of the quality of reporting in published smoking cessation trials for pregnant women: an explanation for the evidence-practice gap?

Background To facilitate translation of evidence into clinical practice, it is critical that clear, specific, and detailed information about interventions is provided in publications to promote replication, appropriate aggregation in meta-analysis, and implementation. This study examined whether twenty elements of interventions deemed essential for such translational application were reported in sufficient detail in smoking cessation trials with pregnant women. Methods Searches of electronic databases using MeSH terms and keywords identified peer-reviewed English language studies published between 2001 and 2012. Eligible studies reported a smoking cessation intervention targeted at pregnant women and met Cochrane’s Effective Practice and Organization of Care group study design criteria. Each intervention arm of eligible studies was assessed against the developed twenty criteria. Results Thirty relevant studies reported the findings of 45 intervention arms. The mode of delivery of the intervention was reported in 100% of intervention arms. Other well-reported criteria included reporting of the provider who delivered the intervention (96%), sample characteristics (80%), and the intervention setting (80%). Criteria not reported adequately included care provided to women who relapse (96% not reported), details about training given to providers (77% not reported), and the method of quit advice advised (76% not reported). No studies reported 100% of relevant criteria. Conclusions Current standards of reporting of intervention content and implementation are suboptimal. The use of smoking cessation specific checklists for reporting of trials, standard reporting using behaviour change taxonomies, and the publication of protocols as supplements should be considered as ways of improving the specificity of reporting. Electronic supplementary material The online version of this article (doi:10.1186/s13012-014-0094-z) contains supplementary material, which is available to authorized users.


Background
The importance of detailed and accurate reporting in randomised controlled trials Effective translation of evidence into clinical practice should follow a logical pathway. First, evidence should be generated through the conduct of well-designed and methodologically rigorous randomised controlled trials, which are considered the gold standard in evaluating the comparative effectiveness of healthcare interventions [1]. Replication of effective interventions should then occur to provide verification of findings, allow for the extension and generalization of results, and ensure that adoption of findings will be effective, efficient, and improve care [2]. The findings of well-designed trials can then be combined in meta-analysis to increase the precision of estimations of intervention effect [3]. Clear, specific, and actionable clinical practice guidelines can then be generated for to guide care, and recommendations communicated and disseminated through training and accreditation processes [4]. This process of evidence generation necessitates that randomised controlled trials, on which other information is based, provide detailed, specific, and easily interpretable information.
Quality of reporting-what is the current state of the literature?
In the early 1990's, a number of reviews identified systemic failures to fully report the methodological characteristics of intervention studies [2,[5][6][7]. As a result, in 1996 the Consolidated Standards of Reporting Trials (CONSORT) guidelines were developed to ensure uniform reporting of the methodological aspects of randomised controlled trials, reduce bias in estimates of intervention effectiveness, and improve internal and external validity [8]. Use of CONSORT criteria has been shown to improve accuracy and quality of reporting [9][10][11][12], and is now a prerequisite for publication in many journals. A number of guidelines aimed at improving the quality of reporting of observational studies [4] and systematic reviews [7,13,14] have since also been published and adopted [15]. However, less attention has been paid to ensuring detailed and specific information is provided about the content of interventions and precisely how they are delivered [16]. The adequacy of description of intervention content and implementation is not currently captured by these tools. CONSORT guidelines ask researchers to report the "precise details of the treatments intended for each group, and when and where they were actually administered", however more specific details including timing of treatment, content of intervention materials, and the intervals between delivery of intervention components that are needed to guide clinical practice are not captured by CONSORT or other methodological quality rating tools including those produced by Cochrane [17].
Several recent studies have examined the extent of accurate and complete reporting of essential elements of interventions across different fields of healthcare [16][17][18][19][20]. In one analysis by Glasziou et al. [17], only 50% of intervention studies deemed important for clinical practice selected for abstraction in the journal Evidence Based Medicine were deemed replicable. This was despite studies examined being published in high-impact journals including The New England Journal of Medicine, The Lancet, and The British Medical Journal. Duff et al. [18] examined reporting of therapeutic details required for clinical application of the findings of oncology drug trials, finding that only 11% of trials reported all aspects deemed to be essential for replication and clinical translation. Currow et al. [19] examined reporting of data items deemed essential for interpretation of study generalizability in specialist palliative care research, finding that adequate data was reported only 28% of the time. Most recently, Schroter et al. [16] found that 33% of drug treatments and 73% of non-drug treatments published in the British Medical Journal over one year were considered non-replicable.
Despite known effective interventions, many pregnant women do not receive best-practice smoking cessation care Smoking is the most significant modifiable cause of adverse pregnancy outcomes. Babies born to mothers who smoke during pregnancy have 50% greater risk of perinatal death [21], 60% greater risk of preterm birth [21], and weigh on average 200 g less than babies born to nonsmoking mothers [22]. Smoking cessation during pregnancy reduces negative pregnancy outcomes, including miscarriage, foetal growth retardation, preterm delivery [23], and perinatal mortality [24]. Despite evidence for the effectiveness of smoking cessation interventions for pregnant women [25], the provision of best-evidence care continues to be suboptimal. A recent review [26] found that just over half of all healthcare providers are likely to ask pregnant women if they smoke and provide advice to quit, while less than one-half actually assist women to quit by providing specific intervention or referral. Failure to appropriately intervene with pregnant women who smoke could be the result of a number of factors, including healthcare providers not being aware of or agreeing with the evidence for effective interventions, and perceiving that the results of trials do not apply to their patient group or are too difficult or too expensive to implement [26]. Another explanation may be that poor reporting of data in clinical trials has inhibited the implementation of best-practice care in clinical practice.
What do clinicians need to know to translate evidence from smoking cessation clinical trials into clinical practice?
The information reported in randomised controlled trials should be sufficient to allow readers to assess to whom the trial result can reasonably be applied, and how to replicate the intervention. Several studies have developed checklists for assessment of study external validity and other information deemed important in reporting of randomised controlled trials [27][28][29][30]. These criteria typically include reporting of the setting of the study; recruitment method; participant information, treatment, or intervention delivered (duration, intensity, dose, delivery format, timing, compliance); outcome assessment; and follow-up.
Smoking cessation interventions typically incorporate substantial behavioural components that are difficult to both describe and reproduce. To provide appropriate clinical care, clinicians also need clear advice about: the precise behavioural strategies that should be used, what skill individuals providing behavioural support should have, how intense the intervention should be, whether to use nicotine replacement therapy, how to deal with women who do not want to quit, and those who relapse. No studies have examined the adequacy of reporting of this specific information for smoking cessation trials with pregnant women.

Aims
To examine whether twenty elements of interventions and their implementation deemed essential for replication, implementation, and aggregation of studies in meta-analysis are reported in sufficient detail in smoking cessation trials with pregnant women. Criteria were developed based on previously published checklists developed to assess the external validity and applicability of intervention trials, and critical assessment of the information necessary to effectively provide smoking cessation in line with clinical practice guidelines.

Literature search
Medline, The Cochrane Library, and PsycINFO databases were searched for relevant studies using MeSH terms and subject headings (see Table 1). Searches were limited to studies published between 2001 and 2012. The reference lists of previous reviews of relevant literature and the reference lists of retrieved articles were also searched to identify any additional relevant studies. The review protocol is available from the authors.

Inclusion and exclusion criteria
Studies were include if they: evaluated a smoking cessation intervention targeted at pregnant women; had smoking cessation or abstinence as an outcome measure; used a design endorsed by the Effective Practice and Organisation of Care (EPOC) group [31] (randomised controlled trials; controlled clinical trials; controlled before and after studies that provided adjustment for confounders; and interrupted time series studies were included); and were published in an English language peer-reviewed journal. Studies were excluded if: the intervention was targeted at a pregnant or postpartum woman's partner or family member; the intervention was targeted at changing healthcare provider behaviour only; or the intervention examined relapse prevention only without examining cessation or abstinence.

Study selection
The titles and abstracts of identified studies were assessed for relevance by one reviewer and rejected on initial screening if the reviewer could determine that the study did not meet inclusion criteria. The remaining studies were retained and assessed against eligibility criteria by two reviewers (JB and AH). A backward and forward reference list search was conducted using Web of Science for all included studies to determine whether additional details of any studies were described in other publications reporting the same study.

Development of checklist of essential elements
A concise checklist of elements deemed essential to the reporting of intervention content and implementation was developed based on previously published checklists developed to assess the external validity and applicability of intervention trials [32][33][34]; and critical assessment by the authors of the information required to effectively provide smoking cessation care from prenatal into the postnatal period based on recommendations contained with clinical practice guidelines. A standard data extraction checklist was developed, then pilot tested by two authors (JB and AH) by independently assessing five studies using the checklist, then meeting to refine and resolve any disagreements. The data extraction checklist is described in Table 2.

Data coding
Studies meeting eligibility criteria were coded by two individuals. Discrepancies were resolved by a third. Each study was coded as to whether the study reported the information (yes or no). As not all criteria were relevant to all studies (for example, some studies did not use nicotine replacement therapy), the proportion of all relevant elements reported for each study was then calculated. A score of 100% indicated complete reporting of all applicable elements.

Search results
The initial search yielded 319 citations, and five additional citations were identified through reference list searches (see Figure 1). After duplicates were removed, 258 unique articles were assessed against eligibility criteria of which, 30 studies  met criteria for inclusion. These studies described 45 intervention arms. One additional publication [65] reporting the same study methodology of an included study [49] was identified through backwards and forwards referencing searching. This manuscript was referred to when coding the primary manuscript, however only the primary publication reporting the cessation or abstinence outcome is cited in the results.

Provider
Is it clear who delivered the intervention? E.g., Nurse, Doctor, Midwife, Counsellor.

Delivery
Is the mode of delivery of the intervention components described? E.g., Face-to-face, telephone, mail, internet Is the method of delivery of the intervention described? E.g., is it part of usual care or a separate program

Schedule
Is the timing of the first and last contact described

Intensity and Duration
Is the total number of intended contacts for the intervention described? Must be reported for each mode of contact Is the total time period the intervention was delivered over clear? E.g., 12 week intervention Is the frequency of contact (the time period between each contact) described? E.g., daily, fortnightly Is the approximate duration of each contact described? E.g., 15 minute face-to-face session

Intervention fidelity
Is the number of participants who received the intervention as intended reported?

Intervention materials
Is the method of quit advice provided described? E.g., recommendation to quit immediately, set quit date Are the behavioural change approaches adequately described? Needs to refer to specific techniques used in enough detail for replication, or refer to detailed manual that is available to reader.
Are written materials (brochures, pamphlets) adequately described? Needs to be described in a level of detail that is replicable, OR materials publically available Was the type, strength, frequency and duration of use of NRT described?
Was the use of incentives clearly described? Needs to describe type (financial or other), amount, frequency, conditions under which they were provided Was social support adequately described? Needs to include type of support (support groups, quit buddy)

Clinical Pathway
Is the process of care for women unwilling to quit described? E.g., are they approached again? If so, in what timeframe and how?
Is the process of care for women who relapse described? What support is provided? How soon after relapse and how?

Skills and Qualifications
Are the skills and professional qualifications of the individual delivering the intervention described?
Training Is the process of training in the intervention adequately described? Duration/format/training manual relapse (96% not reported) and details about the specific training given to providers who delivered the intervention (77% not reported). Seventy-six percent of studies did not provide information about the specific type of quit advice given to participants (e.g., whether to set a quit date or stop immediately), and 70% did not describe the behavioural counselling techniques used in sufficient detail for replication. Only one-half of the intervention arms provided information about the timing of the first and last contacts between providers and participants, and only 45% provided information about the duration of each contact. Table 4 summarises the proportion of intervention elements reported for each intervention arm. No intervention arms reported 100% of all relevant criteria. One intervention arm reported 94% of relevant criteria. Thirtysix percent of intervention arms reported less than 50% of relevant criteria. Four intervention arms reported less than 30% of relevant criteria.

Discussion
This review assessed the adequacy of reporting of elements deemed essential for replication, implementation, aggregation in meta-analysis, and subsequent adoption of findings into clinical practice in smoking cessation trials with pregnant women. We specifically focused on reporting in intervention studies, the findings of which are typically used in generating recommendations for clinical practice guidelines.
Of the 45 intervention arms, none adequately reported 100% of relevant items. Only one intervention arm reported more than 80% of all items. Important information about the delivery of interventions including the type of quit advice provided, the behavioural intervention and written materials provided, the clinical pathway of care, and the training of interventionists were the most poorly described. Information about the intensity and duration of intervention contacts were generally only reported in about one-half of all intervention arms. These factors are critical for both replication of study findings and implementation in clinical practice. The intensity of an intervention carries cost, resource, and feasibility implications for use in routine clinical practice. When such critical data are either not reported or are vague and non-specific, little guidance is provided about the specific components of interventions that should be delivered as part of routine clinical care, or the policies and resources needed to translate best evidence practice into care. Healthcare providers and policy makers are instead left to interpret incomplete information, and draw their own assumptions to fill in the gaps where detail is missing.

Strategies to improve reporting in trials
A number of studies have made recommendations about ways to improve reporting in scientific studies. One suggested approach is the creation of checklists that dictate the specific elements of interventions that should be reported [32]. As suggested by Glasziou et al. [17], it may be necessary to tailor such checklists to different types of interventions. Several attempts have been made to do this. For example, in an attempt to improve standards in reporting of acupuncture trials, Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) [66] have been developed as an extension to the CONSORT criteria. The STRICTA guidelines have since been adopted by a number of journals publishing in acupuncture research. There has been a similar call for standards of reporting to be developed for drug administration in oncology [18] and surgery [67] in order to ensure consistency in the details reported. There are clear advantages to adopting a standard of reporting for smoking cessation trials. Smoking cessation interventions usually incorporate behavioural strategies, which by their nature are often multi-factorial and can be heavily dependent on the skills and precise behaviours of the person delivering the intervention. If we are to identify the most effective, successful, and appropriate smoking cessation strategies for pregnant women, we  need to understand and be able to replicate the specific components of these trials. Michie [68] and Abraham [69] have written extensively about this issue and suggested a taxonomy of behaviour change techniques may be useful in standardising the reporting of components of interventions. An examination of the use of such behaviour change techniques in published smoking cessation trials with pregnant women was recently conducted, with eleven techniques identified as effective [70]. However, the authors noted that the descriptions of interventions were not detailed enough to allow analysis of the effectiveness of individual techniques across studies, highlighting the critical need for improvement in reporting for advancing the evidence base. Strict length limitations imposed by peer-reviewed journals may also limit the reporting of intervention detail. Electronic methods of peer-review publication allow for supplements or appendices providing detailed information to be published online alongside traditional journal articles [17], and could be better used by authors to make available study protocols as a supplement to the peer reviewed paper, or linking the publication with the published trial registration [68]. While this recommendation is frequently made in many papers examining quality of reporting [71,72], it is not yet standard practice. Some journals are however starting to require additional information be provided. For example, the journal Addiction requires studies reporting behavioural interventions to include very detailed descriptions of the full study protocol or intervention manual to be published as supplementary material to the peer-reviewed manuscript.

Strengths and limitations of this review
The findings of this review should be considered in light of several limitations. First, the checklist of essential elements used may not capture all of the elements relevant for clinical translation. However, this checklist was developed based on previous studies in other areas [69,73], and could be used as a starting point to the development of a more comprehensive smoking cessation specific checklist. Second, it is possible that some eligible relevant articles were not identified by our search strategy and therefore not included in this review. Finally, we only coded the presence or absence of reporting of key information, not the quality or accuracy of the information presented.

Conclusion
In order to adequately translate successful smoking cessation interventions into clinical care for pregnant women, healthcare providers need clear guidance about which interventions are most effective. The current standard of reporting of intervention content and implementation is suboptimal, and prohibits replication, implementation, and generation of higher levels of evidence through aggregation in meta-analysis. This ultimately limits the translation of research findings into practice. Clear standards for reporting of the methodological aspects of trials have been adopted, but there is a demonstrated need for improvement in the reporting of essential aspects of interventions and their implementation. For trials relating to pregnant women, increased focus on accurate and specific reporting of the type of quit advice provided, the precise behavioural intervention and written materials provided to participants, the training of intervention providers, and the clinical pathway of care are all of critical importance.