Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study
BMJ 2008; 336 doi: https://doi.org/10.1136/bmj.39482.526713.BE (Published 20 March 2008) Cite this as: BMJ 2008;336:655
- Cynthia Lokker, research associate1,
- K Ann McKibbon, associate professor1,
- R James McKinlay, data analyst1,
- Nancy L Wilczynski, research manager1,
- R Brian Haynes, professor1
- 1Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics, McMaster University Faculty of Health Sciences, Hamilton, ON, Canada L8N 3Z5
- Correspondence to: C Lokker lokkerc{at}mcmaster.ca
- Accepted 28 January 2008
Abstract
Objective To determine if citation counts at two years could be predicted for clinical articles that pass basic criteria for critical appraisal using data within three weeks of publication from external sources and an online article rating service.
Design Retrospective cohort study.
Setting Online rating service, Canada.
Participants 1274 articles from 105 journals published from January to June 2005, randomly divided into a 60:40 split to provide derivation and validation datasets.
Main outcome measures 20 article and journal features, including ratings of clinical relevance and newsworthiness, routinely collected by the McMaster online rating of evidence system, compared with citation counts at two years.
Results The derivation analysis showed that the regression equation accounted for 60% of the variation (R2=0.60, 95% confidence interval 0.538 to 0.629). This model applied to the validation dataset gave a similar prediction (R2=0.56, 0.476 to 0.596, shrinkage 0.04; shrinkage measures how well the derived equation matches data from the validation dataset). Cited articles in the top half and top third were predicted with 83% and 61% sensitivity and 72% and 82% specificity. Higher citations were predicted by indexing in numerous databases; number of authors; abstraction in synoptic journals; clinical relevance scores; number of cited references; and original, multicentred, and therapy articles from journals with a greater proportion of articles abstracted.
Conclusion Citation counts can be reliably predicted at two years using data within three weeks of publication.
Footnotes
Contributors: All authors contributed to the study design and the execution and interpretation of the data. CL and RJM collected the data. CL carried out the data analysis. CL and AM drafted the paper, with revisions contributed by all authors. All authors approved the final version. CL and AM are the guarantors.
Funding: No external funding.
Competing interests: The authors are employees of McMaster University. McMaster University owns the intellectual property for some of the processes described in the study including the McMaster premium literature service and the McMaster online rating of evidence (MORE), which are used to appraise and select articles for ACP Journal Club, Evidence-Based Medicine, Evidence-Based Nursing, bmjupdates+, BMJ Clinical Evidence, Medscape Best Evidence Alerts, Physicians Information and Education Resource, and Harrison’s Practice. All of the coauthors are or have been employed in part by contracts between McMaster University and the publishers of these information services.
Three BMJ Group products are mentioned in this paper: Evidence-Based Medicine, Evidence-Based Nursing, and bmjupdates+.
Ethical approval: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.