Transfer of Statistical Innovations of the 1990s-2000s in Oncology to the Biomedical Literature

Introduction

Translational medicine, aiming to expedite the discovery of new diagnostic tools and treatments by using a multi-disciplinary and collaborative approach, has grown interests in the last decade. Moreover, the commonly accepted delay of 17 years for research evidence to reach clinical practice recently appears to decrease [1] as exemplified by the mapping networks of publications and cross-references. [2] However, most translation gaps between knowledge and clinical application that have been investigated, notably in oncology, concerned translation of biological drivers into therapeutic benefits for patients. [3]

For the advancement of clinical research and eventually of patient care, the translation of innovative statistical methods into practice is also of crucial importance, though the average time elapsed for biostatical innovation to reach medical use is less known. In 1994, Altman showed that, while the key survival paper of Kaplan and Meier achieved only six citations in medical journals in the first 10 years after publication, evidence of decreasing lag times between the introduction and widespread use of innovative statistical methods was expected. [4] More than 2 decades later, we wondered whether the reported above decreasing dichotomy between basic or preclinical and clinical research [2] could be also true in the transfer of innovative statistical methods, confirming the Altman’s hypothesis. [4]

We first focused on competing risks methods, the key innovative survival methods of the last decades, [5] common in cancer. Indeed, competing risks exist whenever the probability of the main event (e.g., cancer) is prevented (e.g., by death) or altered (e.g., by transplantation) to occur. In this setting, standard survival methods that ignore the informative censoring of these competing events may result in biased effects of prognostic factors or treatments, [6] illustrating the need for bridging the gap of innovation into practice. Two innovative papers that first proposed a statistical test [7] and a regression model [8] for such data, were selected as the first innovative set.

With the aim of focusing on statistical innovations of potential large expected therapeutic benefits for patients, we secondly considered the setting of phase I clinical trials, which is also of great interest in the oncology area though with a limited diffusion as compared to phase III. [9] We chose as the most influential statistical pioneering approach, the continual reassessment method (CRM) that is, a model-based approach for dose-finding studies where only empiric 3+3 designs (formerly Fibonacci) were long available. We thus included in the second innovative set the original paper of the CRM [10] together with its close proposed modifications (Table 2).

The primary objective of this paper was to assess whether Altman’s prediction was fulfilled in the field of oncology using the competing risks and the phase I clinical trials as examples. Secondary objectives were to study the relative importance of the applied and statistical literature in papers citing statistical innovations and to analyze which medical areas the methods are mainly used.

Methods

Results

The innovative sets of papers used for both settings are reported in Table 2. They included 2 articles for the competing risks set, and 28 for the phase I set.

Competing risks set

A total of 6,727 citations (5,863 unique articles), including 2,431 for Gray’s article and 4,296 for the Fine and Gray paper, were found (Table 1). Time to translational gap was 6.2 years for the Gray’s paper and 4.5 years for the Fine and Gray paper. The vast majority (91%) of the citations originated from applied journals, with a sharp and continuous increase in citations over time (Figure 1); in contrast, the citations from methodological and semi-applied journals represented only small percentages (6% and 2%, respectively) of the citing papers over the entire period. This was confirmed by the representation of the network of these journals (Figure 2).

• Download figure
• Open in new tab

Figure 1.

Cumulative number of citations for the competing risks setting (Panel A) and the phase I trials setting (Panel B) retrieved from the Web of Science and classified in each category of journals.

• Download figure
• Open in new tab

Figure 2.

Citation network of journals for the two innovative sets (Competing risks- Fig 2A, Phase I trials- Fig 2B). For clarity, only journals with at least 5 citations are represented.

Beside a methodological cluster, two major clusters confirmed the two main areas of application of competing risks, namely oncology (e.g., Journal of Clinical Oncology, European Journal of Cancer and Hematology) and hematology (e.g., Blood, Haematologica, and Bone Marrow Transplantation). Other clusters represent other areas of application, such as cardiovascular diseases.

Discussion

This study aimed to check whether the time lags in the statistical translation process was actually shortened, focusing on two areas with a large potential for clinical research improvements and widely encountered in the oncology literature, namely, the survival methods for competing risks data and the new designs for phase I clinical trials. The answer was bifid, with a short translational gap, defined by the time to reach 25 citations, [4] of about 5 years for the former, but delayed above 10 years for one half of the later. This could be expected given the two statistical innovative sets are used at different stages of clinical studies, with different levels of complexity. Indeed, the first set of competing risks articles deals with method of data analysis, that can easily be performed using modern statistical software without the need for major expertise - although whether such studies are used, performed and interpreted correctly can be another matter. [15] Therefore, such a lag time of about 5 years for using an innovative data analytic method is in agreement with the time to publication after completion of data collection and analysis recently estimated at about 3 years in six journals with high impact factors. [16] In contrast, the second set of innovative methods concern a change in trial design and logistics; thus, beside the time of analysis and publication, the transfer of innovative clinical trial design into the medical literature is obviously impacted by the additional constraints of patient enrollment time, and the follow-up period for the end point. It moreover requires statisticians to engage since the planning phase, [17] up to the analysis of the data, contrary to traditional methods such as the ‘3+3’ design, which can be used without involving statisticians and computer programs and remains the most common choice among clinicians for phase I dose-escalation oncology trials. [18]

Moreover, difference in complexity of both settings was illustrated in terms of citation patterns. Although the competing risks methodology was widely diffused over the medical (in particular, oncology) community, methodology relating to innovative designs for cancer Phase I trials failed to translate easily into practice, consistent with the results of a previous study that showed a very slow transfer of phase I design improvements into clinical practice. [19]

Nevertheless, the two settings share large implications for clinical research, and researchers trying to apply the statistical innovative methods should not be delayed in using the new knowledge. This is notably true in the setting of phase I trials in oncology, where improved selection of patients due to improvements in translational medicine should translate into faster and more precise dose determination. [20] Diffusing more widely into the applied literature and the medical community could be achieved in several ways. First, communication between biostatisticians and clinical colleagues should be improved, although many reviews have been published on these methods in the medical literature, as illustrated by the non-negligible fraction of semi-applied and applied papers that are different from the original clinical trials used in our study. This could be driven through key gateway journals, as suggested by the clusters of citation journals where Journal of Clinical Oncology and Statistics in Medicine appear major players (Figure 2). Concerning competing risks analysis, for which adequate modern methods have been integrated into all modern statistical software, a key step may be to educate clinicians to recognize the settings in which competing risks are of concern and to persuade them of the importance of using those methods. For the design of phase I studies, improvements in providing concrete guidance for designing such trials and facilitating their implementation in practice is still mandatory for bridging the gap between statistical innovation and practical implementation. Model-based designs are cited in the US FDA guidance for industries that are related to adaptive clinical trials as “less understood models,” and the FDA highlights some of their disadvantages. [21]

Our study has some limitations. First, we used citations as a measure of the transfer of knowledge between researchers and physicians. Such citation counts, although not a direct measure of the intrinsic significance of a research idea, provide a measure of statistical technology transfer and of its impact. [4] However, in addition to papers that focus on ranking journals and impact factors [22] many others have considered the citations only qualitatively. [23] Second, we segregated articles into three categories of applied, semi-applied and methodological journals, as previously reported. [5] However, this distinction is somewhat simplistic because the applied fractions of articles published in statistical journals may increase over time; this is reportedly highest for statistical medicine. [5] Third, we focused on two main and distinct topics of biostatistics, and the results might differ for other statistical innovations, such as dynamic prediction modelling, the joint modelling of longitudinal data, and multiple imputation techniques for handling missing data.

Competing interests

The author(s) declare no competing interests.

Funding

There was no specific funding for this study.

Authors’ contributions

SC had the original idea. SC and AV designed the study. AV conducted the literature search, extracted and analyzed data. AV, VL, and SC participated in the interpretation of the data. AV drafted the manuscript. AV, VL, and SC authors reviewed the manuscript and approved the final version.

Availability of data and material

Data used in this study are publicly available data. The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.