ABSTRACT
Background Integrity of academic publishing is increasingly undermined by fake science publications massively produced by commercial “editing services” (so-called “paper mills”). They use AI-supported, automated production techniques at scale and sell fake publications to students, scientists, and physicians under pressure to advance their careers. Because the scale of fake publications in biomedicine is unknown, we developed a simple method to red-flag them and estimate their number.
Methods To identify indicators able to red-flag fake publications (RFPs), we sent questionnaires to authors. Based on author responses, a classification rule was applied initially using the two-indicators “non-institutional email AND no international authors” (“email+NIA”) to sub-samples of 15,120 PubMed®-listed publications regarding publication date, journal, impact factor, country and RFP citations. Using the indicator “hospital affiliation” (“email+hospital”), this classification (tallying) rule was validated by comparing 400 known fakes with 400 matched presumed non-fakes.
Results Two initial indicators (“email+NIA”) revealed a rapid rise of RFP from 2010 to 2020. Countries with the highest RFP proportion were Russia, Turkey, China, Egypt, India and China (39%-55%). When using the “email+hospital” tallying-rule, sensitivity of RFP identification was 86%, the false alarm rate 44%, and the estimated RFP rate in 2020 was 11.0%. Adding a RFP-citation indicator (“email+hospital+RFP-citations”) increased the sensitivity to 90% and reduced the false alarm rate to 37%. Given 1.3 million biomedical Scimago-listed publications, the estimated annual RFP number in 2020 is about 150,000.
Conclusions Potential fake publications can be red-flagged using simple-to-use, validated classification rules to earmark them for subsequent scrutiny. RFP rates are increasing, suggesting higher actual fake rates than previously reported. The large scale and proliferation of fake publications in biomedicine can damage trust in science, endanger public health, and impact economic spending and security. Easy-to-apply fake detection methods, as proposed here, or more complex automated methods can enable the retraction of fake publications at scale and help prevent further damage to the permanent scientific record.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
The study was funded by the Otto-v.-Guericke University of Magdeburg and by a grant from the State of Sachsen-Anhalt Ministery of Science, Magdeburg, Germany
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
In this revision we clarified which tallying rule was used for each experiment. In addition, we calculated the red flag incidence with the validated "institutional email+hospital" indicator and adjusted the total estimate of red flagged publications. We appreciate the suggestion of the readers suggesting it and clarified this in the text after analyzing the yr 2020 sample of the data again to obtain more valid estimate of the scope of fake publishing. The result does not change the overall conclusion of our paper
Data Availability
All data produced in the present study are available upon reasonable request to the authors
