Time to publication in medical education journals: An analysis of publication timelines during COVID-19 (2019-2022)

Introduction

Journal articles are an important metric of academic productivity, and publication success is a significant consideration in tenure and promotion decisions. However, the publication process and its timeliness are influenced by factors beyond the quality of the scholarship (e.g., reviewer availability, quality, and responsiveness). Prior to the COVID-19 pandemic it took, on average, 263 days from submission of an article to a medical education journal for it to appear in PubMed [1]. This finding was based on articles published between 2008-2018. Since this publication, the COVID-19 pandemic has changed the world and consequently scholarly communication [2]. While multiple studies have investigated the impact of COVID-19 on publishing broadly [3,4] and in biomedicine more specifically [5], the pandemic’s effect on medical education publishing is unstudied.

COVID-19 was a “game changer” in medical education [6]. Yet, it is unknown how COVID-19 shaped publishing in medical education and if publication-related factors observed in other fields (e.g., publication type and coverage of COVID-19) played a role in publication timelines. As publications and their corresponding timelines are crucial determinants of academic success, it is critical for the field to have an updated publication timeline. Accurate timelines enable stakeholders to set realistic expectations for productivity and, as needed, to advocate for their members. Publication timelines also provide the consumers of these publications with a sense of the recency of an article’s content, which is important because findings from such articles can often influence educational policy and practice.

COVID-19 shifted the dynamics of scholarly publishing. Researchers found that on the one hand, COVID-19 “turbocharged” some scholarly publishing [5] such that the speed and volume of publication increased to provide rapid evidence to front-line healthcare workers, policy makers, and the public [3,7]. For example, one study found that early in the pandemic, medical journals halved their time to publication [8] and articles about the pandemic itself were published at faster rates and higher volumes than those not related to the pandemic [2,9,10]. On the other hand, publication of non-COVID focused research slowed [2], the productivity of certain groups (e.g., females, physician scientist trainees, caregivers, medical faculty) diminished [11–15], and the lack of access to classrooms and labs delayed research that was in the data collection and analysis phase.

Journal publishers and editors responded to COVID-19 in a variety of unprecedented ways. For example, to expedite publication, 20 major publishers and scholarly organizations united to establish the C19 Rapid Review initiative through which they shared peer reviews and the identities of reviewers thus streamlining the flow of manuscripts across journals [16]. In two other instances, the journal eLife adapted its peer review policies such that reviewers were advised not to require additional experiments or analyses in revision requests, because these requests would hinder publication speed [17], and the Lancet expanded their ‘fast tracking’ system for articles to expedite the sharing of ‘critical knowledge’ [5]. Recognizing the pandemic’s toll on reviewers and authors, many journals, including those in medical education, extended peer review and revision timelines [18]. Across science broadly there is some evidence as to how such initiatives impacted publication timelines [19,20], but there is no evidence specific to medical education.

At the start of the pandemic, eLife’s editor-in-chief espoused: “Publishing will not and should not be anybody’s first priority in the coming months” [17]. However, now that we are well beyond the pandemic’s emergency period, it is hard to ignore academia’s long history of prioritizing journal articles as a marker of success [21], which has high stakes implications for the scholarly progression of individuals and the field. At the individual level, journal articles and their related publication timelines play a role in one’s ability to be promoted, to secure grant funding, and to graduate from publication-dependent degree programs. For example, a review of promotion and tenure guidelines for faculty of medicine at 92 universities identified that 95% of guidelines mentioned peer-reviewed publications with 35 institutions (38%) specifying a required number of articles [22]. Thus, it is important for individuals and administrators to have updated publication timelines that reflect the reality of publishing during COVID-19.

Although the pandemic has ended and COVID-19 is now endemic, the potential for future pandemics or natural disasters is high, such that understanding these timelines is still an important aim. First, it will help to ensure that realistic expectations for productivity are being set and that evidence is available for advocating for researchers who may have faced a publication timeline influenced by the pandemic. An updated publication timeline also informs evidence-based decision making. Ideally, individuals make decisions using the “best available evidence” [23] with the currency of that evidence being a factor that enables the user to determine the relevance of the evidence to their current context. Thus, in this study, we explore publication timelines in medical education just before and during the COVID-19 pandemic. To guide this study, we asked: did publication timelines significantly change around COVID-19 and, if so, what were those changes, both broadly and in relation to publishing characteristics such as publication type and whether or not an article was about COVID-19.

Methods

We conducted a bibliometric study that replicated and expanded upon a prior study on publication timelines in medical education [1]. That prior work drew heavily upon an earlier study that had investigated this topic broadly across scientific disciplines [24]. Where deviations were made from the original study [1] we make note and provide a rationale.

Sample

We included articles that appeared in the Medical Education Journals List (MEJ-24) and that are indexed by PubMed. The MEJ-24 is a seed set of journals that was derived using bibliometric methods to represent the field of medical education [25]. The MEJ-24 includes 24 journals and represents an expansion from the previous study, which considered 14 journals [1].

Our sample included articles published in any one of the MEJ-24 journals between January 1, 2018 and December 31, 2022. To investigate the potential differences in publication times before and after the COVID-19 pandemic, we clustered articles into the following three groups (See Figure 1). The creation of these three groups deviates from the earlier study, which considered all articles across the study period at the same.

• Download figure
• Open in new tab

Figure 1: Pre-COVID, COVID-overlap, and COVID-endemic periods

Group 1 (G1; pre-COVID) included articles submitted between January 1, 2018–March 10, 2020 and published prior to March 10, 2020. We selected March 10, 2020 as a cut off date for publication as the World Health Organization declared a global pandemic on this date [26]. This group served as our control, since the COVID-19 pandemic should not have affected these articles’ publication time. We conceptualized Group 2 (G2; COVID-overlap) as an overlap period which contains articles submitted before the COVID outbreak, but not published until after the COVID outbreak. G2 articles were submitted between January 1, 2018 and March 10, 2020 and published after March 10, 2020. Group 3 (G3; COVID-endemic) included articles submitted after March 10, 2020 and published by December 31, 2022. We descriptively summarize the number of days in publication timeline for these three periods and compare the mean-level differences for statistical significance.

Results

During the study period 16,544 articles were published in MEJ-24 journals (See Zenodo for dataset [33]). Of these journals, 15 supplied partial or complete publication timeline data representing 8,501 (51.4%) articles (See Online Appendix A). For the purposes of our research question, we focused on articles submitted after January 1, 2018 and published prior to December 31, 2022 with full timeline data. This represented 7,614 articles from 15 journals of which 23 articles included flawed metadata (e.g. the received date was after the acceptance date) so were removed resulting in 7,591 (45.9%) articles included for analysis.

The time to publish was on average 188.1 days (SD = 115.3). Time from submission to acceptance was on average 149.9 days (SD = 106.5). Processing time, which is the time between acceptance and appearing in PubMed, was on average 38.2 days (SD = 46.1).

One-way between-group ANOVA results showed the publication time was significantly different among Group 1 (pre-COVID), Group 2 (COVID-overlap), and Group 3 (COVID-endemic), F (2, 7588) = 237.66, p <.001. Significant differences were also found for acceptance time, F (2, 7588) = 177.10, p <.001, and process time, F (2, 7588) = 79.58, p <.001. These results indicated publication timeline was significantly different during pre-COVID (G1), COVID-endemic (G3), and when there is overlap during COVID (G2, i.e., submitted pre-COVID but published after-COVID).

The post-hoc comparisons using the Tukey HSD test indicated that articles of Group 2 (COVID-overlap, n = 761, M= 262.8; SD = 134.4) took significantly longer time in publication as compared to articles in Group 1 (both submitted and published before COVID, n = 2004; M= 200.7; SD = 118), and those in Group 3 (both submitted and published COVID-endemic, n = 4826; M= 171.1; SD = 105.2). Significant results with the same trend were found for acceptance time, and for processing time (see Table 1 for Ms and SDs).

The results showed that articles that were submitted when COVID was endemic (Group 3) were published significantly faster than those that were submitted and published before the pandemic (Group 1), p < .001. The same significant results were found between these two groups for acceptance time and process time (see Table 1 for Ms and SDs). Thus, these results indicate that the publication timelines were delayed specifically for those articles submitted prior to/close to the pandemic outbreak, but not for those that were submitted once COVID became endemic.

Discussion

The pandemic phase of COVID-19 is over; yet this short-lived period had a profound impact on science and academic publishing [34]. Our findings suggest that this effect was also felt by those publishing in medical education, but these effects can be considered both positive and negative. For example, articles submitted during the COVID-overlap period took almost a year and a half to be published, which may have had consequences for individuals and the currency of the field’s knowledge base. However, in the COVID-endemic period, we observed that even with the increased number of manuscripts submitted to and published by medical education journals, the timeline for publication was significantly shorter, suggesting that there may be efficiencies worth exploring and possibly retaining into the future.

This study confirms findings from the broader scientific literature (e.g., biomedicine[35]): we found that the publication timeline for COVID-19 topics was quicker than publication timelines for non-COVID topics. This effect may have been the result of the various mechanisms that journals implemented. For example, it was found that the peer review process could be made faster because editors collaborated with editors at other journals [36] or removed hurdles such as requests for additional data [17]. While these mechanisms were introduced in a time of crisis and for a specific topic, it is important for editors and publishers to consider which, if any, should be retained into the future. Additionally, researchers should study the costs that these mechanisms might bring to the community (e.g., stress on peer reviewer effort, lessening of article quality) and the degree to which they are tolerable.

This study extends the existing literature by providing updated publication timelines and evidence-based expectations for academic productivity. In examining the COVID-pandemic effect, articles submitted between January 1, 2018 and March 10, 2020, and those published after March 10, 2020, took substantially longer to move through the publication process. So, while many universities granted “the gift of time” by automatically extending faculty tenure clocks during the pandemic [37], tenure and promotion committees must consider that many articles were impacted, including not just those that were submitted during the pandemic phase, but also articles under review or in revisions prior to COVID that were significantly affected. Thus, faculty hired between 2018 and 2020 might be expected to have less research productivity (at least in terms of published articles). Faculty seeking tenure and promotion could use these findings in a statement about how COVID shaped their academic productivity. Lastly, we observed that the article type (e.g., commentary, review article) should also be taken into consideration when assessing productivity, as time to publication can vary considerably between publication types.

Previous research found that medical education articles took an average of 263 days to publish between 2008 and 2018 [1]. Since March 11, 2020, articles are taking an average of 226 days from submission to appearing in PubMed. While shorter, this more recent timeline stands in contrast to the estimated 100 days it takes for the biomedical articles to be published [24], which has been criticized as a lengthy publication delay with potential negative effects on scientific progress [38]. As a field, medical education must consider what such a delay means for the timeliness of the evidence that is used to make decisions and craft educational policies.