Abstract
In the midst of epidemics such as COVID-19, therapeutic candidates are unlikely to be able to complete the usual multi-year clinical trial and regulatory approval process within the course of an outbreak. We apply a Bayesian adaptive patient-centered model—which minimizes the expected harm of false positives and false negatives—to optimize the clinical trial development path during such outbreaks. When the epidemic is more infectious and fatal, the Bayesian-optimal sample size in the clinical trial is lower and the optimal statistical significance level is higher. For COVID-19 (assuming a static R0 = 2 and initial infection percentage of 0.1%), the optimal significance level is 7.1% for a clinical trial of a non-vaccine anti-infective therapeutic clinical trial and 13.6% for that of a vaccine. For a dynamic R0 ranging from 2 to 4, the corresponding values are 14.4% and 26.4%, respectively. Our results illustrate the importance of adapting the clinical trial design and the regulatory approval process to the specific parameters and stage of the epidemic.
1 Introduction
With growing public concern over the outbreak of Coronavirus Disease 2019 (COVID-19), significant efforts have been undertaken by global biomedical stakeholders to develop effective diagnostics, vaccines, anti-viral drugs, medical devices, and other therapeutics against this highly infectious and deadly pandemic. While in the past, the traditional randomized clinical trial (RCT) and regulatory approval process often took several years (U.S. Food & Drug Administration, 2018)—longer than the typical duration of an epidemic outbreak (Pronker et al. 2013)—recently the FDA has responded with actions such as the Breakthrough Devices Program, Emergency Use Authorization (EUA) authority, and Immediately in Effect guidance documents to prevent novel diagnostics and therapeutics from lagging behind the urgent needs of the population. In this paper, we propose adapting yet another tool that the FDA has already been exploring for medical devices (Chaudhuri et al. 2018, 2019) to therapeutics for treating COVID-19 that are currently under development.
In recent years, Bayesian adaptive RCT protocols have been increasingly used to expedite the clinical trial process of potentially transformative therapies for diseases with high mortality rates (Berry, 2015). Currently, these protocols have mainly been applied within the oncology domain, such as I-SPY for breast cancer (Barker et al., 2009) and GBM AGILE for glioblastoma (Alexander et al. 2018). These studies use Bayesian inference algorithms to greatly reduce the number of patients needed to assess the therapeutic effects of a drug candidate, without lowering the statistical power of the final approval decision, as measured by Type I and II error rates. As a result, therapeutic candidates can progress more quickly through the regulatory process and reach patients faster and at lower cost.
For severe diseases with no curative treatments, such as pancreatic cancer, patients tend to tolerate a higher Type I error of accepting an ineffective therapy in exchange for a lower Type II error of rejecting an effective therapy as well as expedited approvals of potentially effective treatments. Based on this observation, a patient-centered Bayesian protocol was proposed (Isakov et al., 2018; Montazerhodjat et al., 2017) that incorporates patient values into clinical trial design and identify the optimal balance between the possibilities of false positives (Type I error) and false negatives (Type II error). For more severe diseases, this protocol sets a tolerated Type I error rate much larger than the traditional 5% threshold, which leads to higher rates of approvals and expedited approval decisions.
However, the original Bayesian adaptive RCT framework does not take into account patient risk preferences. To address this gap, Chaudhuri and Lo (2018) developed an adaptive version of the Bayesian patient-centered model that achieves an optimal balance between Type I and Type II error rates, significantly reducing the number of subjects needed in trials to achieve a statistically significant conclusion. A key feature of this model is the time evolution of the loss function of the Bayesian decision algorithm. This mechanism favors the expedited approval of diagnostic or therapeutic candidates that show early positive effects, since patients place a lower value on delayed approval of an effective diagnostic or therapy.
There is a natural but subtle analog to this dilemma in the case of therapeutics for an infectious disease during the course of an epidemic outbreak. Approving an effective therapeutic early will prevent future infections and deaths, while approving it later will save fewer people from infection. On the other hand, approving an ineffective therapeutic early will not prevent any future casualties. Worse still, it may prevent people from taking adequate precautions against infection, since they will falsely believe that they are safe from the disease after the advent of the ineffective therapy.
Moreover, the cost of Type I versus Type II error can differ from therapy to therapy. A novel vaccine that could trigger a significant immune response such as a cytokine storm has a much higher cost of a Type I error than a medical device such as an air filtration system designed to destroy virions through intense ultraviolet light. Therefore, the appropriate statistical threshold for approval should depend on the specific therapy, as well as the circumstances of the current burden of disease.
In this article, we apply the Bayesian adaptive protocol to anti-infective therapeutic development using a loss function that evolves over the course of an epidemic outbreak. We achieve an optimal balance between Type I and Type II errors for therapeutics that treat infectious diseases and identify the optimal time to reach the approval decision based on the accumulation of clinical evidence. Our results show that when the epidemic is more infectious, the necessary sample size of the RCT decreases, while the tolerable Type I error increases. This confirms our earlier intuition that potentially effective therapies that are known to be safe should receive expedited approval when an epidemic is spreading rapidly.
2 Multi-Group SEIR Epidemic Model
The starting point for our analysis is the Susceptible-Exposure-Infective-Removed (SEIR) epidemic model, which has been applied to model the outbreak of COVID-19 in China in a number of recent studies (Yang et al. 2020; Wu et al. 2020). The population of N subjects is partitioned into four distinct groups: susceptible (S), exposed (E), infectious (I), and removed (R). The time evolution of the epidemic is specified by the following group of ordinary differential equations:
Here we use the convention that S(t), E(t), I(t), and R(t) are the proportions of the susceptible, exposed, infectious and removed populations, respectively, satisfying the conservation constraint for all t:
The parameters β, a, and γ denote the average rates of infection, incubation and recovery, respectively, and μ ∈ (0%, 100%) denotes the mortality rate of the epidemic. For example, if μ = 5%, we expect 5% of infected subjects will die from the disease. At time t, μR(t)N subjects will have died, and (1 - μ)R(t)N will have recovered.
A critical measure of the infectivity of an epidemic is its basic reproduction number, defined as R0 = β/γ in the SEIR model. This is the expected number of secondary infections caused by each infected subject in a population with no public health measures (such as quarantine, social-distancing, or vaccination).
A number of studies have used different statistical schemes to estimate R0 for COVID-19 during its initial outbreak period in central China in January 2020. These estimated values of R0 range from 2.2 (95% CI, 1.4 to 3.9) (Li et al., 2020) to 3.58 (95% CI, 2.89 to 4.39) (Zhao et al., 2020). Given the large uncertainty in the value of R0, we simulate therapeutic development under scenarios with constant R0 values of 2 and 4.
In addition, to model the impact of governmental nonpharmaceutical interventions (NPIs) on containing the spread of the epidemic, we consider a dynamic transmission SEIR model where the infection rate β(t) monotonically decreases in time as a result of the NPI. Specifically, we assume that β(t) takes the sigmoid functional form:
Here β0 and β∞ denote the infection rates in the initial and final stages of the epidemic (with β0 > β∞), respectively, t2 denotes the half-life of the decay in infection rate, and τ the length of the time window when this decay occurred. A larger difference β0 - β∞ corresponds to more significant reduction of epidemic transmission, a smaller value of t2 corresponds to a speedier decision to enforce the NPI, and a smaller value of τ corresponds to more strict enforcement of the NPI since β(t) decays more rapidly. We calibrate the values β0 = 3 and β∞ = 1.5 based on the estimates of the dynamic transmission rate of COVID-19 in Wuhan, China from December 2019 to February 2020 (Kucharski et al., 2020). We consider different values of t2 and τ to reflect the variability in timing and stringency of NPIs enforced by governments around the globe.
To model the significant variability in mortality rates of COVID-19 for patients in different age groups, we extend this basic model to a multi-group SEIR model, where the population is partitioned into five age groups, (1) below 49, (2) 50 to 59, (3) 60 to 69, (4) 70 to 79, and (5) above 80. We use Si, Ei, Ii, and Ri to denote the corresponding type in each group (and continue to use S, E, I, and R for the total proportion of each type in all groups). The dynamics of the epidemic are specified by the modified ordinary differential equations:
Here ci denotes the contact rate of the susceptible subjects in the ith age group with the total infected population I of all groups. This contact rate is measured relative to group 1, which we normalize to cl = 1. In the case of COVID-19, although the mortality rate is much higher for senior populations (Onder et al., 2020), the elderly also tends to have less frequent contact with the infected population outside the household (Walker et al., 2020).
We solve the differential equations in the multi-group SEIR model using the ODE45 solver in MATLAB 2019a with initial conditions for each age group:
The parameter I0 denotes the proportion of the initially infected population, re is the ratio of initially exposed and infected subjects, and Pi is the percentage of the ith age group in the population. The assumed demographic, contact rate, and mortality rate values are summarized in Table 2.
3 A Bayesian Patient-Centered Approval Process
Similar to Chaudhuri and Lo (2018), we develop a Bayesian patient-centered decision model for RCT approval which minimizes the expected loss (or harm) incurred on the patients by optimally balancing the losses of Type I and Type II errors. Here the loss does not refer to financial costs afforded by the patients, but rather the loss in patient value (i.e. how much patients weigh the relative harms of infection and death). We assign the losses per patient of being susceptible, infected, and deceased. Since Bayesian decision thresholds are invariant under the rescaling of the losses, we normalize by setting the loss per patient infection Li = 1. We then assign the loss per patient death relative to Li as LD, and the loss due to susceptibility to the disease as LS. The parameter values we assume, summarized in Table 1, are meant to represent one reasonable valuation of the relative losses. However, in practice patient value will differ from one patient group another, especially given the large variability of mortality rate of COVID-19 in different age groups (Onder et al., 2020). Here we report the main results of optimal sample size and statistical significance (Table 3 and 4) assuming LD = 100. The results for LD = 10 are provided in Supplementary Materials.
Demographic (US Census Bureau, 2018), relative contact rate (Walker et al. 2020) and mortality (Onder et al. 2020; World Health Organization 2003; 2019; 2020) profile of various age groups for COVID-19, SARS and MERS.
Simulation results of a Bayesian adaptive RCT on non-vaccine anti-infective therapeutics obtained from 10,000 Monte Carlo runs and assuming L-D = 100. R0 denotes the basic reproduction number, µ the disease morality, and I0 the proportion of initial infected subjects. Sample size refers to the number of subjects enrolled in each arm of the RCT. SD denotes standard deviation, and IQR the interquartile range about the median. R0(t) denotes the use of a dynamic transmission model with half-life t2 = 3 weeks and window length τ = 1 week.
Simulation results of Bayesian adaptive RCT for vaccines obtained from 10,000 Monte Carlo runs and assuming LD = 100. R0 denotes the basic reproduction number, µ the disease morality, and I0 the proportion of initial infected subjects. Sample size refers to the number of subjects enrolled in each arm of the RCT. SD denotes standard deviation, and IQR the interquartile range about the median. R0(t) denotes the use of a dynamic transmission model with half-life t2 = 3 weeks and window length τ = 1 week.
We simulate the multi-group SEIR model over a time period of T weeks, where T is the duration of the epidemic outbreak. Let κ denote the weekly subject enrollment rate in each arm of the clinical trial. We assume that the value of R0 is known (or well-estimated) at initial time t = 0 and stays constant during the course of the outbreak. At time t ∈ [0, T], the Bayesian loss Cij(t) of choosing the action Ĥ = i under H = j is defined as:
where we define the cumulative number of infected patients CI(t) until time t:
By design, this loss function penalizes Type I errors early in the epidemic by the susceptible term, (S(t) - S(T))NLS. We subtract the base level S(T) from S(t) since the multi-group SEIR model predicts that S(T)N subjects will not be infected by the epidemic. A Type I error at an earlier time will expose more currently susceptible population to the epidemic, since they will falsely believe that they are safe from the disease after the advent of the ineffective therapeutic. On the other hand, the loss function also penalizes correct approval decisions made at later stages of an epidemic via to the cumulative infected and death terms, CI(t) and μR(t). A correct but delayed approval decision for the therapeutic is less valuable since it will save fewer susceptible people from infection and death.
The Bayesian decision model considers the null hypothesis H = 0 that the anti-infective therapeutic (or vaccine) has no clinical effect, against the alternative hypothesis that it has positive clinical effect with signal-to-noise ratio ρ (Chaudhuri and Lo, 2018). We use p0and pl to denote the Bayesian prior probabilities of H = 0 and H = 1, respectively.
This patient-value model imposes higher losses for incorrect approvals at earlier stages and correct approvals at later stages of an epidemic. Under these constraints, the Bayesian decision algorithm yields the sample size and statistical significance threshold of the RCT that optimally balances Type I and Type II error.
4 Results
We simulate an epidemic outbreak over a time period of T weeks, where T is the duration of the outbreak. For an epidemic with higher infectivity, its duration is shorter, which puts more pressure to reach a timely approval decision. To avoid numerical instability, we formally define T as the first time when the number of cumulative infected patients reaches 99.9% of total infections predicted by the SEIR model. We assume an age-specific mortality rate μ at the level of COVID-19 (Onder et al. 2020; World Health Organization, 2020), and incubation and recovery periods of 7 days each (Yang et al. 2020). These estimated parameters can all be challenged to varying degrees, depending on the specific drug-indication pair under consideration and the particular circumstances of the epidemic, but they are meant to be representative for a typical anti-infective therapeutic during the midst of a growing epidemic.
We also assume that it takes 7 days after injection to assess the efficacy of the therapeutic on each subject. We adopt the optimization scheme of Montazerhodjat et al. (2017) to find the optimal Type I and Type II error rates of the non-adaptive Bayesian RCT. To represent typical practice of the pharmaceutical industry, we optimize under the upper bound on the model’s power Powermax = 90% (Isakov et al. 2019). We then use these optimal error rates as our stopping criteria to simulate the sequential decision process of a Bayesian adaptive RCT via Monte Carlo simulation (Chaudhuri and Lo, 2018). The simulation results are summarized in Table 3.
We separate the results into two distinct types of therapeutics—non-vaccine anti-infectives and vaccines—because of the differences in their historical probabilities of success. Vaccine development programs have an estimated PoS of 40% as of 2019Q4 (https://projectalpha.mit.edu) whereas the corresponding figure for non-vaccine anti-infectives is 23% (Wong et al., 2020).
4.1 Non-Vaccine Anti-Infective Therapeutics
Static Transmission Rate
We first analyze the case when the infectivity R0 remains constant in time (e.g. in the absence of effective NPIs). For the fixed-sample Bayesian RCT on a non-vaccine anti-infective therapeutic, as R0 increases from 2 to 4 (Rows 1 to 2 of Table 3), the optimal sample size of each experimental arm decreases from 242 to 152 and the optimal Type I error rate drastically increases from 7.1% to 17.3% (Figure 1), much higher than the traditional 5% threshold. As the epidemic spreads across the population more rapidly, the Bayesian RCT model has greater pressure to expedite the approval process and a much higher tolerance of false positive outcomes.
Optimal Type I error rate α of a non-adaptive Bayesian RCT vs. basic reproduction number R0 (assuming I0 = 0. 1%, LD = 100, disease mortality of COVID- 19, and constant R0). The Bayesian decision model yields a higher α for epidemics with high and low infectivity.
For the Bayesian adaptive RCT, when the therapeutic is ineffective (H = 0), the average sample size required to reject the therapeutic is much smaller than that of the non-adaptive version (Columns 7 and 8 of Table 3). Also, the required sample size decreases with the infectivity R0 in both mean and quartiles, yet always achieves Type I error rate α below that of the non-adaptive version (Column 11). The adaptive Bayesian decision model is able to reject an ineffective therapeutic with a relatively small sample size and a bounded false-positive rate.
On the other hand, when the therapeutic is effective (H = 1), as R0 increases from 2 to 4, the average sample size required by the Bayesian adaptive RCT decreases from 148 to 78 (Columns 9 and 10 of Table 3). The Bayesian adaptive model places more weight on approving an effective therapeutic earlier to prevent future infections when the epidemic is more infectious. Despite the smaller sample size, the model still retains an empirical power above 91.0% for all values of R0 (Column 12). The Bayesian adaptive model simultaneously expedites the approval of an effective therapeutic and retains a bounded false-negative rate. The results are illustrated in Figure 2.
Subject sample size in each arm of a Bayesian adaptive RCT under H = 1 decreases with the basic reproduction number R0 (assuming I0 = 0. 1%, LD = 100 and disease mortality of COVID-19). BNA denotes Bayesian non-adaptive optimal; BA 50% denotes median patient size of Bayesian adaptive. The 25% and 75% quantiles of Bayesian adaptive patient size are shown as lower and upper ends of the error bar.
Furthermore, as the proportion of the initially infected population I0 decreases from 0.1% to 0.01% (Rows 4 to 6 of Table 3), the optimal sample sizes for non-adaptive and adaptive RCTs both increase, while the optimal Type I error rates decrease. Beginning the clinical trials for a therapeutic during the earlier stages of an epidemic outbreak reduces the need to expedite the approval process in order to contain its future spread. Clinicians and researchers have more time to evaluate the efficacy of a therapeutic and record adverse effects by testing it on a larger number of subjects, which leads to a lower Type I error rate.
Finally, when the mortality rate μ increases from the level of COVID-19 (Onder et al. 2020; World Health Organization, 2020), to the level of SARS (World Health Organization, 2003), and further to the level MERS (World Health Organization, 2019), the optimal sample sizes for both non-adaptive and adaptive Bayesian models decrease and the optimal Type I error rates increase (Rows 7 to 12 of Table 3). When the epidemic is more lethal, the Bayesian adaptive model requires fewer subjects in the RCT, since both Type I and Type II errors will lead to greater losses due to death by infection. The higher death tolls provides significantly more incentive in the Bayesian adaptive framework to approve the therapeutic in the hopes of saving more people from future infection and death.
One interesting feature of the Bayesian decision model is that the optimal Type I error rate is not a monotonic function R0, but rather has a global minimum of 8% at R0 = 1.7 for COVID-19, as shown in Figure 1. As R0 decreases below 1.7, the optimal Type I error rate increases. The intuition for this result that we define the loss of Type I error as the excess risk of being susceptible to infection (S(t) - S(T))NLS, where S(T) is the fraction of the population that remains uninfected throughout the epidemic outbreak. When R0 is small, S(T) is close to 100% and the excess risk (S(t) - S(T))NLS is small compared to the benefit of preventing future deaths. Therefore, when the epidemic is not very infectious, the Bayesian decision model expedites the approval decision. This also confirms the intuition that smaller sample sizes are required in adaptive trials for diseases that affect a small fraction of the population. If we instead define the loss of Type I error as the absolute risk of being susceptible S(t)NLS, we find that the optimal Type I error indeed monotonically increases with R0, as shown in Figure 11 of Supplementary Materials.
Dynamic Transmission Rate
The results for the dynamic transmission model with β0 = 3, β∞ = 1.5, t2 = 3 weeks and τ = 1 week are summarized in Table 3. For COVID-19 (rows 3 and 6), we find that the Bayesian optimal sample size and Type I error rate of the dynamic transmission model lie in between the scenarios R0 = 2 and R0 = 4. This suggests that timely and effective government interventions will protect more subjects from infection and allow more time for the RCT.
However, for the more fatal SARS and MERS (rows 9 and 12), the dynamic transmission model sets higher optimal Type I error α and smaller sample size than R0 = 4. This is due to the U-shaped curve of optimal α vs. R0, shown in Figure 1. When the NPI reduces R0(t) below a certain threshold, the optimal α starts to increase. For highly fatal epidemics, when the government adopts NPIs to protect most of the susceptible population from infection, the regulatory priority should be to expedite potentially effective treatments that can help current patients since the loss of Type I error is much lower than that of the Type II error.
In addition, we investigate the impact of the timing and stringency of NPIs enforced by the government with different values of t2 and τ. The results are summarized in Table 5. We find that the optimal Type I error is larger for t2 = 3 weeks than t2 = 6 weeks. Therefore, if the government adopts well-enforced NPIs early on (such as the lockdown of Wuhan, China) to protect the susceptible population, this will reduce the loss associated with Type I error, leading to expedited approvals of potentially effective therapeutics. Furthermore, the sooner an effective therapeutic is approved, the sooner will NPIs be lifted.
Optimal sample size and Type I error α of Bayesian non-adaptive RCT for non-vaccine anti-infective therapeutics for dynamic transmission model. R0 denotes the basic reproduction number, µ the disease morality, and I0 the proportion of initial infected subjects. Sample size refers to the number of subjects enrolled in each arm of the RCT. R0(t) denotes the use of a dynamic transmission model with β0 = 3, β∞ = 1. 5.
4.2 Vaccines
We repeat the above analysis for RCT of vaccines using a prior probability of having an effective vaccine as reported at https://projectalpha.mit.edu for 2019Q4. The simulation results are summarized in Table 4. Overall, we observe the same pattern in the optimal sample size and Type I error rates on infectivity, mortality, and proportion of initial infections. However, since pl is higher for vaccines, the Bayesian decision model requires fewer subjects on average in the RCT to ascertain the positive effects of the vaccine, compared to the case of anti-infective therapeutics in Table 3. We find that vaccines should receive even more expedited evaluation.
To assess the robustness of our model’s predictions against the assumed values of model parameters, we perform a sensitivity analysis on the parameter values. The results are summarized in Supplementary Materials.
5 Discussion
A natural consequence of using a patient-centered framework for determining the approval threshold is, of course, more false positives—and the potential for a greater number of patients with adverse side-effects—in cases where the burden of disease is high. These false positives can be addressed through more vigilant post-approval surveillance by regulatory agencies and greater requirements for drug and device companies to provide such patient-level data to the regulator following approval. Failure to provide such data or evidence of an ineffective therapy can be grounds for revoking the approval.
However, past experience shows that withdrawing an approved drug can be challenging and disruptive for several reasons (Onakpoya, Heneghan, and Aronson, 2016). Therefore, implementing the patient-centered approach may require creating a new category of temporary approvals for crisis situations involving urgent needs at national or international levels, similar to the FDA’s EUA program. Such a program might involve provisional approval of a candidate therapy consisting of a one- or two-year license—depending on the nature of the drug-indication pair—to market the therapy to a pre-specified patient population, no off-label use of the therapy, and regular monitoring and data reporting to the regulator by the manufacturer and/or patients’ physicians during the licensing period (Lo, 2017). At the end of this trial period, one of two outcomes would occur, depending on the accumulated data during this period: (a) the “urgent needs” license expires; or (b) the license converts to the traditional regulatory license. Of course, at any point during the trial period, the regulator can terminate the license if the data show that the therapeutic is ineffective and/or unsafe.
While such a process may impose greater burdens on patients, manufacturers, and regulators, it may still be worthwhile if it brings faster or greater relief to patients facing mortal illnesses and extreme suffering. In this respect, an urgent-needs program may be viewed as a middle ground between a standard clinical trial and an approval, similar in spirit to the adaptive designs of sophisticated clinical trials with master protocols such as I-SPY 2, LUNG-MAP, and GBM-AGILE, in which patient care and clinical investigations are simultaneously accomplished. Also, because the Centers for Medicare and Medicaid Services (CMS) has demonstrated a willingness to cover the cost of certain therapeutics for which evidence is still being generated (see, for example, CMS’s “coverage with evidence” programs listed at https://go.cms.gov/2v6ZxWm), additional economic incentives may be available to support such temporary licenses.
6 Conclusion
We apply the Bayesian adaptive patient-centered model of Chaudhuri and Lo (2018) to clinical trials for therapeutics that treat infectious diseases during an epidemic outbreak. Using a simple epidemiological model, we find that the optimal sample size in the clinical trial decreases with the infectivity of the epidemic, measured by the basic reproduction number R0. At the same time, the optimal Type I error rate increases with R0. Lower levels of initial infection increase the number of subjects required to verify the therapeutic efficacy of the therapeutic under investigation, while higher levels of mortality increase the optimal sample size. The results confirm our intuition that clinical trials should be expedited and a higher false positive rate should be tolerated when the epidemic spreads more rapidly through the population, has a higher mortality rate, and has already infected a sizable portion of the population at the beginning of the RCT.
To provide transparency for how a patient-centered approach differs from the traditional statistical framework in the anti-infectives context, we used a relatively simple mathematical model of epidemic disease dynamics to estimate the societal loss in an outbreak. More sophisticated epidemiological models can easily be incorporated into our framework at the cost of computational tractability and transparency.
One interesting trade-off to be explored is the difference between COVID-19 vaccine and an anti-viral treatment that can cure an infected patient. While prevention through vaccination is the ultimate solution, a successful treatment for the disease using repurposed drugs that have already been approved for other indications (and whose safety profile has already been established) may be even more valuable, especially if it can be deployed in the nearer term and reduce the growing fear and panic among the general population. In such cases, the approval threshold should clearly reflect these cost/benefit differences.
Of course, in practice regulators consider many factors beyond p-values in making its decisions. However, that process is opaque even to industry insiders, and the role of patient preferences is unclear. The proposed patient-centered approach provides a systematic, objective, adaptive, and repeatable framework for explicitly incorporating patient preferences and burden-of-disease data in the therapeutic approval process. This framework also fulfills two mandates for the FDA, one from the fifth authorization of the Prescription Drug User Fee Act (PDUFA) for an enhanced quantitative approach to the benefit-risk assessment of new drugs (U.S FDA, 2013), and the other from Section 3002 of the 21st Century Cures Act of 2016 requiring the FDA to develop guidelines for patient-focused drug development, which includes collecting patient preference and experience data and explicitly incorporating this information in the drug approval process.
We hope this work will shed further insight into improving the current clinical trial process for infectious disease therapeutics and contribute to the timely development of effective treatments for COVID-19 patients in particular.
Supplementary Materials
Sensitivity of optimal sample size (in each arm of non-adaptive Bayesian RCT) to the weekly subject enrollment rate κ. From the lower to upper end of each box plot, the five-parameter summary corresponds to [50%, 75%, 100%, 125%, 150%] of the baseline value κ = 100 used in our analysis (Table 1).
Sensitivity of optimal Type I error rate to the weekly subject enrollment rate κ. From the upper to lower end of each box plot, the five-parameter summary corresponds to [50%, 75%, 100%, 125%, 150%] of the baseline value κ = 100 used in our analysis (Table 1).
Sensitivity of optimal sample size (in each arm of non-adaptive Bayesian RCT) to the signal to noise ratio (SNR) ρ of treatment effect (Chaudhuri and Lo, 2018). From the upper to lower end of each box plot, the five-parameter summary corresponds to [50%, 75%, 100%, 125%, 150%] of the baseline value ρ = 0. 25 used in our analysis (Table 1).
Sensitivity of optimal Type I error rate to the signal to noise ratio (SNR) ρ of treatment effect (Chaudhuri and Lo, 2018). From the upper to lower end of each box plot, the five-parameter summary corresponds to [50%, 75%, 100%, 125%, 150%] of the baseline value ρ = 0. 25 used in our analysis (Table 1).
Sensitivity of optimal sample size (in each arm of non-adaptive Bayesian RCT) to the prior probability of having an ineffective treatment p0. From the lower to upper end of each box plot, the five-parameter summary corresponds to [70%, 85%, 100%, 115%, 125%] of the baseline value p0 = 77% used in our analysis (Table 1).
Sensitivity of optimal Type I error rate to the prior probability of having an ineffective treatment p0. From the upper to lower end of each box plot, the five-parameter summary corresponds to [70%, 85%, 100%, 115%, 125%] of the baseline value p0 = 77% used in our analysis (Table 1).
Sensitivity of optimal sample size (in each arm of non-adaptive Bayesian RCT) to the time to assess the efficacy of the non-vaccine anti-infective therapeutics Δt. From the upper to lower end of each box plot, the five-parameter summary corresponds to [0%, 50%, 100%, 150%, 200%] of the baseline value Δt = 1 week used in our analysis (Table 1).
Sensitivity of optimal Type I error rate to the time to assess the efficacy of the anti-infective therapeutic Δt. From the lower to upper end of each box plot, the five-parameter summary corresponds to [0%, 50%, 100%, 150%, 200%] of the baseline value Δt = 1 week used in our analysis (Table 1).
Sensitivity of optimal sample size (in each arm of non-adaptive Bayesian RCT) to the incubation rate a of the infectious disease. From the upper to lower end of each box plot, the five-parameter summary corresponds to [50%, 75%, 100%, 125%, 150%] of the baseline value a = 1 week used in our analysis (Table 1).
Sensitivity of optimal Type I error rate to the incubation rate a of the infectious disease. From the lower to upper end of each box plot, the five-parameter summary corresponds to [50%, 75%, 100%, 125%, 150%] of the baseline value a = 1 week used in our analysis (Table 1).
Optimal sample size and Type I error rate α for Bayesian non-adaptive RCT on anti-infective therapeutics with R0 (basic reproduction number) close to 1. µ denotes the disease morality, and I0 the proportion of initial infected subjects. Sample size denotes the number of subjects enrolled in each arm of the RCT.
Simulation results of Bayesian adaptive RCT on non-vaccine anti-infective therapeutics obtained from 10,000 Monte Carlo runs and assuming LD = 10. R0 denotes the basic reproduction number, µ the disease morality, and I0 the proportion of initial infected subjects. Sample size refers to the number of subjects enrolled in each arm of the RCT. SD denotes standard deviation, and IQR the interquartile range about the median. R0(t) denotes dynamic transmission model with half-life t2 = 3 weeks and window length τ = 1 week.
Simulation results of Bayesian adaptive RCT on vaccines obtained from 10,000 Monte Carlo runs and assuming LD = 10. R0 denotes the basic reproduction number, µ the disease morality, and I0 the proportion of initial infected subjects. Sample size refers to the number of subjects enrolled in each arm of the RCT. SD denotes standard deviation, and IQR the interquartile range about the median. R0 (t) denotes dynamic transmission model with half-life t2 = 3 weeks and window length τ = 1 week.
Optimal Type I error rate α of non-adaptive Bayesian RCT monotonically increases with the basic reproduction number R0 (assuming I0 = 0. 1%, LD = 100 and disease mortality of COVID-19) if we define the loss of making a Type I error as the absolute risk of being susceptible S(t)NLS.
This alternative definition is not very realistic. For an epidemic with R0 < 2, the loss of Type I error converges to a large positive value S(T)NLS as time approaches the end of the epidemic outbreak. However, at the end of the outbreak, there are no more infected patients and thus no susceptible subjects. Therefore, the loss of Type I error should approach zero as t → T. This is the case for the excess risk of susceptibility (S(t) - S(T))NLS but not for the absolute risk of susceptibility S(t)NLS.
Footnotes
* We thank Murray Sheldon, Chi Heem Wong, the editor, associate editor, and reviewer for many helpful comments and suggestions, and Jayna Cummings for editorial assistance. Research support from the MIT Laboratory for Financial Engineering is gratefully acknowledged. The views and opinions expressed in this article are those of the authors only, and do not represent the views, policies, and opinions any institution or agency, any of their affiliates or employees, or any of the individuals acknowledged above.