An analysis of general medical and specialist journals that endorse CONSORT found that reporting was not enforced consistently

doi:10.1016/j.jclinepi.2005.01.004

Journal of Clinical Epidemiology

Volume 58, Issue 7, July 2005, Pages 662-667

https://doi.org/10.1016/j.jclinepi.2005.01.004 Get rights and content

Abstract

Background

We aimed to determine if specialist journals implement specific Consolidated Standards for Reporting Trials (CONSORT) recommendations to the same extent as general medical journals.

Methods

Analysis of random controlled trials (RCTs) in five general medical journals (n = 100) and 10 specialist journals (n = 100), all endorsing CONSORT. We evaluated the likelihood of reporting important methodologic criteria. Analyses controlled for the nested effect of journal within each journal type.

Results

General medical journals published, on average, more CONSORT items per RCT than specialist journals (7.9 [SD 1.8] vs. 6.5 [SD 2.2] out of 11 possible items, P = .02). When compared with specialist journals, RCTs in general medical journals published a participant flow diagram more frequently (83 vs. 42%, odds ratio [OR] 6.7, 95% confidence interval [CI] 3.4–12.9) and more likely to report the method of randomization (78 vs. 55%, OR 2.9, 95% CI 1.5–5.3) and allocation concealment (48 vs. 26%, OR 2.6, 95% CI 1.4–4.7); they were less likely to publish RCTs reporting adverse events (58 vs. 78%, OR 0.3, 95% CI 0.2–0.7). Both page length and impact factor were weakly associated with number of CONSORT items reported.

Conclusion

General medical and specialist journals that endorse CONSORT do not enforce reporting issues consistently, with specialty journals lagging behind general medical journals.

Introduction

At least 152 journals have now adopted the Consolidated Standards for Reporting Trials (CONSORT) [1], a checklist of 22 recommended items for reporting randomized controlled trials (RCTs) and a participant flow diagram, documenting the flow of participants through RCTs.

CONSORT comprises a checklist and flow diagram to help improve the quality of reports of RCTs. It aims to standardize the reporting of important methodological issues in trials. The checklist includes items, based on evidence, that should be addressed in the published report; the flow diagram provides readers with a clear picture of the progress of all participants in the trial, from the time they are randomized until the end of their involvement. The intent is to make the experimental process more transparent, whether flawed or not, so that users of the RCT results can more appropriately evaluate their validity [1].

Several studies have examined the impact of the CONSORT recommendations on the quality of reports of RCTs [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. Most of these investigations have been limited to high-impact general medical journals with limited information about specialty medical journals. One study identified a difference between general and specialty journals [5], with general medical journals reporting more items in the CONSORT checklist than specialty journals. However, that study examined RCTs reported between 1985 and 1997 (prior to the publication of the revised CONSORT statement in 2001) and included journals irrespective of their adoption of CONSORT. Whether RCTs published recently in general or specialty medical journals that endorse CONSORT continue to differ to a similar extent is unknown.

Our hypothesis was that specialty medical journals that endorse CONSORT have implemented the CONSORT checklist to a lesser extent than general medical journals. To test this hypothesis, we determined the extent to which RCTs published in 2003 in five general and 10 specialty journals endorsing CONSORT published a participant flow diagram and reported methods of randomization and restriction, allocation concealment and implementation, blinding status, sample-size calculations, use of intention-to-treat analysis, explanation of withdrawals and adverse events, and ethics review/informed consent.

Section snippets

Selection of journals and RCTs

In March 2003, we obtained the list of journals that endorse the revised CONSORT statement (from www.consort-statement.org). From this list, we chose the five leading general medical journals (Annals of Internal Medicine, Archives of Internal Medicine, BMJ, JAMA, and The Lancet) and selected the 10 leading specialty journals (AIDS, Archives of General Psychiatry, Diabetes, Gastroenterology, Gut, Hepatology, Journal of the American College of Cardiology, Journal of Clinical Oncology, Journal of

Results

We included 100 RCTs from general medical journals and 100 RCTs from specialist journals in our analysis. Fig. 1 describes the inclusion of articles for this study. Prior to inclusion fulfilment, we excluded 34 potentially relevant general medical articles due to publishing their methods previously (n = 28), being brief reports (n = 4) and being cost-related studies of RCTs (n = 2). Thirty-three potentially relevant specialist articles were excluded as their methods were published previously (n = 16),

Discussion

We found that in 2003, general medical and specialist journals that endorse CONSORT do not enforce reporting issues consistently. General medical journals continue to implement CONSORT to a greater extent than specialist journals, and that this relationship remained after taking into account journal characteristics. We found that page length and impact factor were weakly but significantly correlated with the extent of reporting of CONSORT items.

To our knowledge, our study is the first to

Acknowledgments

We wish to thank Prof. Doug Altman and Dr. David Moher for critical suggestions through the writing of this manuscript. Contributions: study concept and design: Mills, Gagnier, Montori; acquisition of data: Mills, Wu; analysis and interpretation of data: Mills, Wu, Gagnier, Heels-Ansdell, Montori; drafting of the manuscript: Mills, Wu, Gagnier, Heels-Ansdell, Montori; critical revision of the manuscript for important intellectual content: Mills, Montori; statistical expertise: Heels-Ansdell,

References (37)

D. Moher et al.
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials
Lancet
(2001)
P.J. Devereaux et al.
The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist
Control Clin Trials
(2002)
V.M. Montori et al.
In the dark: the reporting of blinding status in randomized controlled trials
J Clin Epidemiol
(2002)
E.J. Mills et al.
Randomized controlled trials in long-term care residents with dementia: a systematic review
J Am Med Dir Assoc
(2003)
H. MacPherson et al.
Standards for reporting interventions in controlled trials of acupuncture: the STRICTA recommendations
Complement Ther Med
(2001)
D.G. Altman et al.
Randomisation and baseline comparisons in clinical trials
Lancet
(1990)
M. Egger et al.
Value of flow diagrams in reports of randomized controlled trials
JAMA
(2001)
T.A. Faunce et al.
Of consents and CONSORTs: reporting ethics, law, and human rights in RCTs involving monitored overdose of healthy volunteers pre and post the “CONSORT” guidelines
J Toxicol Clin Toxicol
(2003)
M. Piggott et al.
Has CONSORT improved the reporting of randomized controlled trials in the palliative care literature? A systematic review
Palliat Med
(2004)
K. Huwiler-Muntener et al.
Quality of reporting of randomized trials as a measure of methodologic quality
JAMA
(2002)

D. Moher et al.

Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation

JAMA

(2001)

J. Nuovo et al.

Reporting number needed to treat and absolute risk reduction in randomized controlled trials

JAMA

(2002)

M. Bhandari et al.

Application of the Consolidated Standards of Reporting Trials (CONSORT) in the fracture care literature

J Bone Joint Surg Am

(2002)

M. Clarke et al.

Discussion sections in reports of controlled trials published in general medical journals

JAMA

(2002)

M. Clarke et al.

Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents?

JAMA

(1998)

K.F. Schulz et al.

Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials

JAMA

(1995)

S. Hollis et al.

What is meant by intention to treat analysis? Survey of published randomised controlled trials

BMJ

(1999)

V.M. Montori et al.

Intention-to-treat principle

CMAJ

(2001)

Cited by (66)

Methodology reporting improved over time in 176,469 randomized controlled trials
2023, Journal of Clinical Epidemiology
To describe randomized controlled trial (RCT) methodology reporting over time.
We used a deep learning-based sentence classification model based on the Consolidated Standards of Reporting Trials (CONSORT) statement, considered minimum requirements for reporting RCTs. We included 176,469 RCT reports published between 1966 and 2018. We analyzed the reporting trends over 5-year time periods, grouping trials from 1966 to 1990 in a single stratum. We also explored the effect of journal impact factor (JIF) and medical discipline.
Population, Intervention, Comparator, Outcome (PICO) items were commonly reported during each period, and reporting increased over time (e.g., interventions: 79.1% during 1966–1990 to 87.5% during 2010–2018). Reporting of some methods information has increased, although there is room for improvement (e.g., sequence generation: 10.8–41.8%). Some items are reported infrequently (e.g., allocation concealment: 5.1–19.3%). The number of items reported and JIF are weakly correlated (Pearson's r (162,702) = 0.16, P < 0.001). The differences in the proportion of items reported between disciplines are small (<10%).
Our analysis provides large-scale quantitative support for the hypothesis that RCT methodology reporting has improved over time. Extending these models to all CONSORT items could facilitate compliance checking during manuscript authoring and peer review, and support metaresearch.
Incomplete reporting of patient-reported outcomes in multiple sclerosis: A meta-epidemiological study of randomized controlled trials
2022, Multiple Sclerosis and Related Disorders
Citation Excerpt :
These contradictory findings may be due to inconsistent journal implementation of the CONSORT guideline which can affect the interpretability of RCTs. ( Mills et al., 2005) In contrast to our findings, studies have found evidence that journal endorsement of CONSORT may lead to increased completeness of reporting. ( Plint et al., 2006; Turner et al., 2012) Although endorsement of CONSORT is widespread with 585 journals endorsing it, there is a lack of endorsement for CONSORT extensions. (
Multiple Sclerosis significantly affects quality of life, which is often measured by patient-reported outcomes. The incorporation of patient-reported outcomes within clinical trials supplements the efficacy of outcomes in order to provide a patient's perspective for clinicians. Our objective was to evaluate current literature for completeness of reporting of PROs in randomized controlled trials (RCTs) for the management of MS.
We used MEDLINE, Embase, and Cochrane Central Register of Controlled Trials to search for RCT publications investigating the management of MS. After duplicate screening via Rayyan, RCTs fitting our inclusion criteria were abstracted employing the Consolidated Standards of Reporting Trials - Patient-Reported Outcome (CONSORT-PRO) adaptation and the Cochrane Collaboration Risk of Bias (RoB) 2.0 tool. Mean percent completion of an adaptation of CONSORT-PRO was calculated to address completeness of reporting. In addition, bivariate regression models were used to evaluate relationships between trial characteristics and completeness of reporting.
Our search returned 3,966 results and 92 RCTs were included for data abstraction and analysis. We found an overall completion of 48.68% (SD=19.03). Sixty-five (of 92; 70.65%) of the RCTs were evaluated as having ‘high’ RoB. There were significant associations between completeness of reporting and the following: mention of CONSORT within published RCTs (t=2.55, p=.013), length of PRO follow-up (t=2.9, p=.005; t=2.14, p=.035), and sample size (t=3.12, p=.002). No other significant associations were found.
Our study found incomplete adherence to the CONSORT-PRO adaptation among RCTs pertaining to MS. Of the most underreported items, the failure to report a hypothesis and define an approach to missing data threaten the validity of the evidence acquired from RCTs. Furthermore, PROs provide an opportunity to supplement trial outcomes with the patient's perspective. Thus, trialists of future RCTs may improve PRO reporting with increased adherence to the CONSORT-PRO adaptation.
The Conduct and Reporting of Child Health Research: An Analysis of Randomized Controlled Trials Published in 2012 and Evaluation of Change over 5 Years
2018, Journal of Pediatrics
Citation Excerpt :
Journals that endorse CONSORT publish RCTs that are more completely reported compared with those that do not.16 Nevertheless, even CONSORT- endorsing journals do not consistently enforce reporting issues,39 and support to guide peer reviewers in assessing reporting quality is often inadequate.40 As indicated by our data, registered child health RCTs are more likely to be at low RoB and to be more completely reported.
For child health randomized controlled trials (RCTs) published in 2012, we aimed to describe design and reporting characteristics and evaluate changes since 2007; assess the association between trial design and registration and risk of bias (RoB); and assess the association between RoB and effect size.
For 300 RCTs, we extracted design and reporting characteristics and assessed RoB. We assessed 5-year changes in design and reporting (based on 300 RCTs we had previously analyzed) using the Fisher exact test. We tested for associations between design and reporting characteristics and overall RoB and registration using the Fisher exact, Cochran-Armitage, Kruskal-Wallis, and Jonckheere-Terpstra tests. We pooled effect sizes and tested for differences by RoB using the χ² test for subgroups in meta-analysis.
The 2012 and 2007 RCTs differed with respect to many design and reporting characteristics. From 2007 to 2012, RoB did not change for random sequence generation and improved for allocation concealment (P < .001). Fewer 2012 RCTs were rated high overall RoB and more were rated unclear (P = .03). Only 7.3% of 2012 RCTs were rated low overall RoB. Trial registration doubled from 2007 to 2012 (23% to 46%) (P < .001) and was associated with lower RoB (P = .009). Effect size did not differ by RoB (P = .43)
Random sequence generation and allocation concealment were not often reported, and selective reporting was prevalent. Measures to increase trialists' awareness and application of existing reporting guidance, and the prospective registration of RCTs is needed to improve the trustworthiness of findings from this field.
Principles of Epidemiology and Public Health
2018, Principles and Practice of Pediatric Infectious Diseases
The use of tailored subheadings was successful in enhancing compliance with CONSORT in a dental journal
2017, Journal of Dentistry
Efforts to enhance the reporting of clinical trials have intensified in recent years with automated strategies and editorial involvement showing promise in improving compliance with accepted guidelines. This study aimed to evaluate the effectiveness of a concerted approach to adherence to CONSORT (CONsolidated Standards Of Reporting Trials) guidelines in a dental journal.
Following the publication of an exemplar clinical trial on the American Journal of Orthodontics and Dentofacial Orthopedics (AJO-DO) website and related changes to the author guidelines, trial submissions were required to follow a standard format incorporating subheadings mirroring the CONSORT guidelines. Compliance with CONSORT was assessed in initial submissions over a 30-month period. Reporting was compared to submissions of randomized controlled trials (RCTs) which did not include subheadings over the same period.
Seventy-one RCTs were submitted to the AJO-DO from January 2014 to June 2016, 49 with subheadings and 22 without. Most CONSORT items (e.g. random sequence generation, allocation concealment and blinding) were more frequently adequately reported when RCTs were submitted with inclusion of subheadings. Overall, reporting quality of the submitted RCTs was 15.2% higher with use of the subheadings format (95%CI: 10.5, 20.0; p < 0.001) with a mean overall score of 87.3%.
Enhanced compliance of submitted RCTs was found with use of a bespoke approach to trial presentation utilizing CONSORT item subheadings. The improvement in initial submissions is particularly encouraging as this arose without input either from peer reviewers or journal editors. This simple approach may have wider applicability.
The quality of the reported sample size calculations in randomized controlled trials indexed in PubMed
2017, European Journal of Internal Medicine
There are limited data on the quality of reporting of information essential for replication of the calculation as well as the accuracy of the sample size calculation. We examine the current quality of reporting of the sample size calculation in randomized controlled trials (RCTs) published in PubMed and to examine the variation in reporting across study design, study characteristics, and journal impact factor. We also reviewed the targeted sample size reported in trial registries.
We reviewed and analyzed all RCTs published in December 2014 with journals indexed in PubMed. The 2014 Impact Factors for the journals were used as proxies for their quality.
Of the 451 analyzed papers, 58.1% reported an a priori sample size calculation. Nearly all papers provided the level of significance (97.7%) and desired power (96.6%), and most of the papers reported the minimum clinically important effect size (73.3%). The median (inter-quartile range) of the percentage difference of the reported and calculated sample size calculation was 0.0% (IQR − 4.6%;3.0%). The accuracy of the reported sample size was better for studies published in journals that endorsed the CONSORT statement and journals with an impact factor. A total of 98 papers had provided targeted sample size on trial registries and about two–third of these papers (n = 62) reported sample size calculation, but only 25 (40.3%) had no discrepancy with the reported number in the trial registries.
The reporting of the sample size calculation in RCTs published in PubMed-indexed journals and trial registries were poor. The CONSORT statement should be more widely endorsed.

View all citing articles on Scopus

View full text

Review ArticleAn analysis of general medical and specialist journals that endorse CONSORT found that reporting was not enforced consistently

Abstract

Background

Methods

Results

Conclusion

Introduction

Section snippets

Selection of journals and RCTs

Results

Discussion

Acknowledgments

Lancet

Control Clin Trials

J Clin Epidemiol

J Am Med Dir Assoc

Complement Ther Med

Lancet

Value of flow diagrams in reports of randomized controlled trials

JAMA

Of consents and CONSORTs: reporting ethics, law, and human rights in RCTs involving monitored overdose of healthy volunteers pre and post the “CONSORT” guidelines

J Toxicol Clin Toxicol

Has CONSORT improved the reporting of randomized controlled trials in the palliative care literature? A systematic review

Palliat Med

Quality of reporting of randomized trials as a measure of methodologic quality

JAMA

Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation

JAMA

Reporting number needed to treat and absolute risk reduction in randomized controlled trials

JAMA

Application of the Consolidated Standards of Reporting Trials (CONSORT) in the fracture care literature

J Bone Joint Surg Am

Discussion sections in reports of controlled trials published in general medical journals

JAMA

Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents?

JAMA

Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials

JAMA

What is meant by intention to treat analysis? Survey of published randomised controlled trials

BMJ

Intention-to-treat principle

CMAJ

Review Article
An analysis of general medical and specialist journals that endorse CONSORT found that reporting was not enforced consistently