COVID-19 pandemic surges can induce bias in trials using response adaptive randomization: A simulation study
============================================================================================================

* Christopher J Yarnell
* Robert A Fowler
* Lillian Sung
* George Tomlinson

## Abstract

Response-adaptive randomization is being used in COVID-19 trials, but it is unknown whether outcome rate changes during surges of COVID-19 will lead to bias in trial results. In response-adaptive randomization, allocation ratios are adjusted according to interim analyses to assign more patients to promising interventions. Although it is known that response-adaptive randomization may give biased estimates if outcome rates drift over time, observed mortality fluctuations in the COVID-19 pandemic are more extreme than any previously tested in simulation. We hypothesized that pandemic surges induce bias in trials using response-adaptive randomization, and that adjustment for time will alleviate that bias. Bayesian 4-arm superiority trials with a mortality outcome were simulated to investigate bias in treatment effect, comparing complete and response-adaptive randomization under different pandemic scenarios based on data from New York, Spain, and Italy. Relative bias in the odds ratio associated with treatment ranged from 0.3% to 11% and was largest in trials with a surge and an effective intervention that did not adjust for time. Bias was attenuated by adjustment for time without compromising the false-positive rate. Trials using response-adaptive randomization were more likely to identify effective interventions but were slower to drop ineffective interventions. Even with variation in outcome rates similar to observed pandemic surges, COVID-19 trials using response-adaptive randomization that adjust for time can provide accurate estimates of treatment effects.

Keywords
*   Adaptive Clinical Trials As Topic
*   Critical Care
*   Computer Simulation
*   COVID-19
*   Pneumonia

## Introduction

The SARS-CoV-2 pandemic triggered a global effort to test interventions in clinical trials.1 Some of the proposed and ongoing trials employ response-adaptive randomization, defined as a method of changing allocation to interventions in a randomized trial according to trial outcomes.1–4 This is a method of allocating more patients to effective interventions and potentially reducing the sample size required to reach a conclusion 5–8, but uptake has so far been uncommon.9–11

Response-adaptive randomization (RAR), as compared to complete (conventional) randomization, is potentially suitable for COVID-19 trials because RAR generally increases the probability that patients are allocated to effective arms. This may appeal to patients considering enrollment, clinicians screening patients or joining trials, and regulators assigning trial funding.5,6,12–19 In the setting of 2 or more interventions in addition to control there may also be a decrease in sample size required to identify successful therapies, depending on the method of adaptive randomization.14,20–23. Trials that use RAR may face logistical challenges 19, require complex analyses limiting the accessibility of results 24, and give biased estimates if the risk of outcomes changes over time. 8,16,21,22,25 This last limitation is particularly relevant to COVID-19. For example, if COVID-19 cases have overburdened the healthcare system, the mortality rate may increase and then return to baseline as the surge abates. If the allocation ratios of patients to different treatment arms are different during the time of high as opposed to baseline mortality, a naïve comparison of outcomes by treatment assignment could create the illusion of harm or efficacy between equivalent treatments (Figure 1).

![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/15/2022.07.13.22277596/F1.medium.gif)

[Figure 1:](http://medrxiv.org/content/early/2022/07/15/2022.07.13.22277596/F1)

Figure 1: Potential bias due to adaptive randomization in a pandemic
This figure shows the 28-day mortality (red line) and number of patients allocated to each ofthe three treatment groups (yellow, green, red) and control (blue) versus time. The 28-day mortality rate doubles during a surge (translucent gray box). The tables below show the counts of patients enrolled either during baseline or surge conditions, with a third table showing the totals.

The yellow intervention enrolls many more patients than the other treatments during the surge, so an unadjusted comparison suggests that the yellow group has a higher mortality rate (0.27) even though within either baseline or surge intervals it is beneficial (OR 0.88). By contrast, the red intervention appears to be beneficial because during the surge it was allocated the fewest patients, but within each interval it is no different from control (OR 1.0).

Using logistic regression with time divided into intervals (visualized as blocks on the figure) provides a way of comparing patient outcomes within each time “block”.

COVID-19 outbreaks with surges in case numbers have occurred worldwide, causing shortages of medications and equipment, provision of intensive care in non-traditional settings, and wide variation in mortality rate for critically ill and hospitalized patients.26–31 The performance of adaptive randomization has been assessed in trials with linear drift, including drift based on the Ebola virus disease epidemiology.12,32 No work, to our knowledge, assesses the robustness of response-adaptive randomization to the nonlinear surges demonstrated by COVID-19.

### Aims

Use simulated trials of therapies for COVID-19 to: (1) compare bias in the estimated odds ratios between trials using response-adaptive and complete randomization; (2) assess whether using regression to adjust for time reduces the potential bias with RAR; and (3) identify whether RAR demonstrates other advantages in the setting of pandemic epidemiology.

## Methods

This simulation study followed recommendations of Morris et. al 33 and the STRESS simulation reporting guidelines 34. This research did not require research ethics board review according to the Tri-Council Policy Statement Article 2.1. 35 In this study we use a Bayesian statistical framework, similar to most contemporary trials that use RAR 10,11,36, although response-adaptive randomization can also be implemented using frequentist approaches 6,32,37. Additional details about the methods are available in the Supplement.

### Data-generating mechanism

Simulations generated randomized superiority trials of hospitalized patients with severe COVID-19 pneumonia comparing three interventions to control with a binary 28-day mortality outcome. Each trial ran for 365 days. Mortality was simulated as a binary random variable based on mortality rate, and daily enrollment was simulated as a Poisson random variable based on the mean daily enrollment. Important components of the data-generating mechanism detailed below were pandemic epidemiology, number and efficacy of intervention arms, randomization algorithms, statistical treatment of time-varying risk, and stopping rules.

### Pandemic epidemiology

Pandemic epidemiology impacted the mortality and enrollment rates. Based on available information we set the baseline 28-day mortality rate to 20%. 26,38 Surges compromising healthcare capacity have been observed in China 39, Italy 27, New York 26, Spain 31, and Brazil 40. Narrative reports describe provision of intensive care throughout the hospital during surge conditions, so we set the surge mortality rate to be similar to the intensive care unit mortality at 40%. 41,42 For enrollment rates we used information from completed and in-progress COVID-19 trials and set mean daily enrolment to be 15 patients at baseline and 25 patients during surge.2,43,44

The duration of each surge was set to 21 days, based on the large dataset from the United Kingdom.42 The first surge, when present, began on trial day 30. The cyclic surge scenario used three surges (spring, fall, winter) starting on trial day 30, 150, and 270, approximately similar to the 1918 pandemic.45 The base case had no surge, and alternative cases had one or three surges. Single surge cases with later surge timing (days 150 and 270) were investigated in sensitivity analyses.

### Number and efficacy of intervention arms

The trials evaluated three intervention arms and one control arm, mimicking large trials of COVID-19 evaluating multiple arms. 2,43,46 Preliminary results from COVID-19 trials suggested an odds ratio of 0.75 for benefit was feasible.43,47 We defined the “null” scenario as all interventions being equivalent (OR = 1) and the “nugget” scenario as one intervention effective (OR 0.75) and the remainder equivalent (OR = 1). The control arm was intervention 1 and if there was an effective intervention, it was intervention 2. As sensitivity analyses we conducted simulations where a single intervention was harmful with an odds ratio of 1.33 and where there was one effective, one harmful, and one equivalent intervention.44

### Randomization algorithms

The base case for randomization was complete randomization where patients have equal probability of being allocated to each arm remaining in the trial including control. Both randomization algorithms began interim analyses after 100 patient-outcomes were available and repeated them every 14 days thereafter.

The response-adaptive randomization algorithm used for these simulations was a mixed patient-benefit and power-oriented approach analogous to that used in the REMAP trial and algorithms shown to have better performance among adaptive randomization algorithms.2,23 For probability *ρ**g* of allocation to treatment group *g*, probability *O(g)* that intervention *g* is optimal, and number of patients *n**g* allocated to group *g*: ![Formula][1]</img>  The algorithm additionally preserved power by stipulating that if any *ρ**g* < 0.10, it is increased and all *ρ**g* are renormalized such that all allocation probabilities are at least 0.10.

### Statistical treatment of time-varying outcomes

Analyses of trials using complete randomization can adjust for time to improve precision.48 For trials using RAR, adjustment for time can also reduce bias 32,49 but may not be able to accommodate the extreme time-trend shifts seen in pandemic epidemiology. The simulations compared trials with and without adjustment for time in interim and final analyses by dividing the timeline into 14-day “blocks” and analyzing time block as a categorical variable. Smaller time blocks and lower enrollment rates were investigated as sensitivity analyses to explore the performance of time trend adjustment with fewer patients in each time block.

### Stopping rules

The simulated trials used stopping rules analogous to those in REMAP and Thall et al.2,10,11,21 An intervention was declared superior if the probability it was the optimal intervention was greater than 0.99, and inferior if the same probability was less than 0.01. The region of practical equivalence ranged from an odds ratio of 1/1.2 to 1.2 and an intervention was declared equivalent if the probability of the odds ratio (relative to control) falling in that region was 0.90 or greater. Arms were dropped when determined to be inferior or equivalent. The trial stopped at 365 days, if an intervention was superior, or if all interventions were either inferior or equivalent.

### Estimand

The estimand for these simulations was the log-odds ratio associated with treatment.33 The odds ratio was assumed constant across varying baseline risk within each model.

### Methods for trial analysis

Interim and final analyses used Bayesian logistic regression to calculate the posterior distribution of the log-odds of mortality associated with each intervention.50,51 Models including adjustment for time incorporated time-block of enrollment as a categorical variable. A case with no interim analyses was considered as a sensitivity analysis.

### Performance measures

Performance measures were calculated for each factorial combination of the simulation characteristics. Results were also calculated according to the subgroup of whether or not the trial reached a conclusion, defined as either a single intervention being declared superior or all interventions being declared equivalent or inferior according to the stopping rules.

### Bias

The primary outcome for this simulation study was bias in the estimate of the odds ratio. This was calculated as the relative (percentage) difference between the mean estimate across simulations and the true value. We calculated Monte Carlo standard error and the root mean-squared error for the log-odds ratio.

### Operating characteristics

True negative rate was calculated as the proportion of simulated trials where an intervention with OR ≥ 1 did not meet the criterion for superiority. True positive rate was calculated as the proportion of trials that declared an effective intervention to be superior. Practical efficiency characteristics included the proportion of trials that reached a conclusion, the duration among trials that reached a conclusion, and the sample size enrolled in each arm.

### Ethical import

The ethical import of the trials was assessed by the proportion of patients randomized to a superior intervention where one existed, the time to dropping inferior or equivalent arms, and the average 28-day mortality of participants in the trial.

### Computational details

All simulations were coded in R and Stan using RStudio and several supporting packages.52–58 Existing software does not accommodate the time-adjustment aspect of these analyses.59 The Niagara computer cluster run by Compute Canada was used to run the simulations.60,61 Each Bayesian regression was run in 2 chains for 3000 iterations with 1000 iterations warmup. The number of simulated trials was chosen to be 1000 to keep the Monte Carlo standard error for the log-odds of bias approximately below 0.005, corresponding to a 95% confidence interval for odds ratio of bias 0.99 to 1.01.33 Chains and r-hat values were inspected in test cases to ensure convergence.50 All R code is available in the supporting information.

## Results

We simulated 1000 trials for each of 24 factorial scenarios. This required 3.5 hours of computation across 4 nodes on the cluster each using 80 cores (1120 core-hours of computation).

### Bias

Relative bias ranged from 0.3% to 11%, with variation according to scenario across the simulations (Table 1, Table E1). Across all scenarios with time-adjusted analyses, bias was less than 15% of the size of the average 95% credible interval for the mean odds ratio. In the null scenario the odds ratios were biased towards harm (OR > 1), and bias was largest (11%) in the single surge scenario using complete randomization without adjustment for time. Results were similar for interventions with OR set to 1.0 in the nugget scenario (Table E1), where complete randomization was associated with greater bias than adaptive randomization.

View this table:
[Table 1:](http://medrxiv.org/content/early/2022/07/15/2022.07.13.22277596/T1)

Table 1: 
Bias for equivalent and effective interventions

In the nugget scenario, where one intervention had odds ratio 0.75, odds ratios generally overestimated benefit with the lowest odds ratio seen in the single surge nugget scenario using adaptive randomization without adjustment for time trends (OR estimate 0.69, relative bias 10.4%). Bias in this scenario decreased (OR 0.73, relative bias 4.0%) with the use of adjustment for time (Figure 2). Throughout interventions with odds ratio of 0.75, complete as opposed to adaptive randomization gave less biased estimates of the odds ratio.

![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/15/2022.07.13.22277596/F2.medium.gif)

[Figure 2:](http://medrxiv.org/content/early/2022/07/15/2022.07.13.22277596/F2)

Figure 2: Odds ratio densities for Intervention 2 by COVID scenario
This figure depicts the distribution of 1000 posterior mean odds ratios for intervention 2 according to simulation scenario, including COVID epidemiology (rows: no surge, single surge, or cyclic surges), intervention scenario (columns: null, nugget), adjustment for time trends (columns: no adjustment, with adjustment), and randomization algorithm (colour). A black vertical line marks the set odds ratio for intervention 2 in either the “null” (OR 1) or “nugget” (OR 0.75) intervention scenarios. The improvement in bias with adjustment for time trends is most evident for adaptive randomization in the single surge scenario.

### Operating characteristics

The proportion of trials that appropriately concluded equivalent interventions were not superior was high (lowest proportion 0.958, Table 2). After incorporating adjustment for time, the proportion of trials using adaptive randomization that appropriately concluded equivalent interventions were not superior was 0.99 or higher in all scenarios (Table 2, Table E2). The proportion of trials where an ineffective intervention was declared optimal was low across null scenarios.

View this table:
[Table 2:](http://medrxiv.org/content/early/2022/07/15/2022.07.13.22277596/T2)

Table 2: 
Trial conclusions and rejection for equivalent and effective interventions

The proportion of trials that reached a conclusion varied according to pandemic epidemiology from a low of 0.39 (null scenario, no surge, adaptive randomization, adjustment for time trends) to a high of 0.86 (nugget scenario, single surge, adaptive randomization, no adjustment for time trends).

The frequency with which superior interventions were identified as such was not high in any scenario. For interventions with odds ratio 0.75, the lowest probability of being declared superior (0.38) was seen in simulations with no surge, complete randomization, and adjustment for time trends. By contrast, the highest probability of being declared superior (0.77) was seen in simulations with a single surge, adaptive randomization, and no adjustment for time trends (Table 2, Figure E1). Trials using adaptive as opposed to complete randomization in the nugget scenario yielded a higher probability of identifying a superior intervention (Figure 3).

![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/15/2022.07.13.22277596/F3.medium.gif)

[Figure 3:](http://medrxiv.org/content/early/2022/07/15/2022.07.13.22277596/F3)

Figure 3: Trial conclusions for equivalent and effective interventions
This figure depicts the proportion of trials that concluded continue, equivalence, inferiority, or superiority for intervention 2 according to simulation scenario, including COVID epidemiology (rows: no surge, single surge, or cyclic surges), intervention scenario (columns: null, nugget), adjustment for time trends (columns: no adjustment, with adjustment), and randomization algorithm (colour). Adaptive as opposed to complete randomization results in a higher proportion of trials concluding that intervention 2 is superior in the nugget scenario.

Among trials that did not reach conclusions according to the stopping criteria, the probability that an effective intervention was the optimal arm was still high. In those trials, the highest mean posterior probability of an effective intervention being optimal (0.945) was seen in the cyclic surge scenario with adaptive randomization and adjustment for time trends (Table E3).

Sample size was generally large (Table 3). Because most trials were limited by the 365-day time-limit, sample sizes were similar across intervention scenarios.

View this table:
[Table 3:](http://medrxiv.org/content/early/2022/07/15/2022.07.13.22277596/T3)

Table 3: 
Outcomes of trial for equivalent and effective interventions

### Ethical import

In the nugget scenario, trials using adaptive randomization consistently randomized about twice as many patients to the superior intervention arm (Table 3). The average risks of 28-day mortality were lower in trials using adaptive randomization, although absolute differences were small (Table 3, Figure E2).

Equivalent or harmful interventions were generally dropped earlier from trials using complete as opposed to adaptive randomization (Figure E3). By contrast, there was minimal difference in time to recognition of a superior intervention.

A large number of null interventions (OR 1) had “Inferiority” conclusions. Across all factorial scenarios, in 95% of trials where intervention 3 (OR = 1) was deemed inferior the posterior probability of equivalence with respect to control was at least 0.78.

## Discussion

This simulation study of COVID-19 found that response-adaptive randomization can generate biased estimates of efficacy amidst surges of COVID-19, and that adjustment for time eliminates this bias. Simulations used realistic trial design and parameters. The bias was generally in the direction of overestimating the efficacy of effective interventions. Relative to complete randomization, response-adaptive randomization with time-adjustment increased the number of patients allocated to effective interventions, increased the probability of identifying effective interventions, and did not increase the probability of concluding that equivalent interventions were effective. However, it delayed the removal of ineffective interventions.

Adjustment for time trends during pandemic surges attenuated bias for both adaptive and complete randomization approaches, but it slightly increased bias if there was no surge. However, the slight increase in bias in the case of no surge is outweighed by the alleviation of large amounts of bias in the cases with surge, so where numerically feasible, adjustment for time trends should be the standard-of-design for Bayesian trials using either response-adaptive randomization or complete randomization with frequent interim analyses.

The simulations revealed important consequences of using complete versus response-adaptive randomization. Trials using complete randomization identified ineffective interventions earlier than trials using response-adaptive randomization. This occurred because in response-adaptive randomization fewer patients are allocated to interventions that are measured to be ineffective, so more time is required to recruit enough patients to definitively remove those interventions from the trial. The proportion of all COVID-19 patients enrolled in trials is low, making this an important consideration in trial design. Announcement of the RECOVERY trial hydroxychloroquine results on June 5, 2020 62, for example, allowed other trials such as the SOLIDARITY trial to stop allocating patients to hydroxychloroquine, and hopefully reduced the inappropriate use of hydroxychloroquine worldwide. More subtle potential benefits include redirection of resources to other avenues for reducing COVID-19 morbidity and mortality, such as additional experimental agents, methods of contact tracing and isolation, or public health campaigns to promote universal masking.

Trials using response-adaptive randomization were more likely to identify effective interventions than trials using complete randomization. Even among trials that did not reach a definite conclusion by 365 days, effective interventions studied with adaptive randomization had a higher probability of being the optimal intervention compared to those studied with complete randomization. Historically, the high proportion of trials with indeterminate or negative results has been a source of frustration for clinicians and trialists.63 Although adaptive randomization does not increase the chance that an intervention is effective, it does increase the chance that an effective intervention is appropriately recognized.

Response-adaptive randomization also led to more patients being allocated to an effective intervention, when one was present. This could help increase patient enrollment by satisfying an individual patient’s desire to have improved outcomes through participation in research.7 This may also encourage clinicians to participate in the trial by reducing the tension between desire to give an individual patient a potentially optimal therapy and the need for randomized evaluation where equipoise exists. However, increased complexity of the consent process and trial conduct may offset these benefits.64

On other aspects of trial performance, the two randomization algorithms were largely similar. Despite randomizing many more patients to effective interventions, trials using response-adaptive randomization did not show a clinically significant improvement in average patient outcomes, partially due to the small difference (20% versus 15.8%) in mortality rates outside of surge conditions. After incorporating adjustment for time trends, there was little difference in true negative rates.

This investigation has several limitations. First, response-adaptive randomization algorithms are not monolithic and different implementations of RAR may lead to different conclusions.32,65 Investigation of all possible RAR design choices is infeasible, so instead we modeled our algorithm after a large ongoing trial of critically ill patients with COVID-19. Second, our implementation of COVID epidemiology may not be a nuanced enough representation of the actual dynamics of mortality or enrollment rates during a surge in cases. Third, the validity of these results may depend on having enough patients in each “time block” such that categorical adjustment for time trends yields an accurate model. Fourth, the simulations modeled a multicenter trial with a surge taking place at the same time in every center, a pattern not seen with COVID-19 surges so far. If surges occurred throughout a trial, and at different times at different sites, the net effect would be regression to the mean event rate, so the issues with RAR could be less pronounced than in the scenarios modeled here. Finally, while the code for replication of this study is easily available, the computational demands of the simulations may limit efforts to plan Bayesian adaptive trials with adjustment for time trends.36,66

The decision to incorporate adaptive versus complete randomization into a trial of a therapy for COVID-19 also depends on factors beyond those captured in these simulations. Adaptive randomization introduces the possibility of clinicians inferring interim results due to detectable changes in treatment allocation as the trial progresses. Using adaptive randomization requires an added level of organization and communication across sites that may outweigh any statistical or ethical benefits amidst the chaos of a pandemic. If adaptive randomization is used with adjustment for time, meta-analyses of therapies would require individual patient data. Finally, research concerning COVID-19 is reaching a much wider audience than almost any medical research previously undertaken, potentially increasing the requirement for simple and transparent statistical analyses as opposed to the more complex analyses required for adaptive randomization. Optimal trial design requires consideration of both quantitative and qualitative aspects of design elements.

## Conclusion

In trials of therapies for COVID-19 during surges that may be associated with increased mortality rates, the use of response-adaptive randomization without adjustment for time estimates results in biased estimates of efficacy. However, this bias is eliminated with adjustment for time and response-adaptive as opposed to complete randomization has a higher probability of identifying effective interventions. Response-adaptive randomization with time adjustment is an appealing option for pandemic trials.

## Supporting information

Supplemental appendix [[supplements/277596_file02.pdf]](pending:yes)

## Data Availability

All data produced in the present study are available upon reasonable request to the authors

## Acknowledgements

We thank the following for helpful comments: Scott Berry.

## Footnotes

*   Funding and role of funders in study The study was funded by the Canadian Institutes for Health Research CGS-M program (Yarnell), the Clinician-Investigator Program of the University of Toronto (Yarnell), and the Eliot Phillipson Clinician Scientist Training Program of the University of Toronto (Yarnell).

*   Received July 13, 2022.
*   Revision received July 13, 2022.
*   Accepted July 15, 2022.


*   © 2022, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  1.World Health Organization. International Clinical Trials Registry Platform. [https://www.who.int/ictrp/en/](https://www.who.int/ictrp/en/). Published 2020. Accessed July 5, 2020.
    
    

2.  2.Angus DC, Berry S, Lewis RJ, et al. The Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia (REMAP-CAP) Study: Rationale and Design. Ann Am Thorac Soc. 2020. doi:[http://dx.doi.org/10.1513/AnnalsATS.202003-192SD](http://dx.doi.org/10.1513/AnnalsATS.202003-192SD)
    
    

3.  3.Houston BL, Lawler PR, Goligher EC, et al. Anti-Thrombotic Therapy to Ameliorate Complications of COVID-19 (ATTACC): Study design and methodology for an international, adaptive Bayesian randomized controlled trial. Clin Trials. August 2020:174077452094384. doi:10.1177/1740774520943846
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1740774520943846&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32815416&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

4.  4.Angus DC, Derde L, Al-Beidh F, et al. Effect of hydrocortisone on mortality and organ support in patients with severe COVID-19: The REMAP-CAP COVID-19 corticosteroid domain randomized clinical trial. JAMA - J Am Med Assoc. 2020;324(13):E1–E13. doi:10.1001/jama.2020.17022
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.17022&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32778601&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

5.  5.Rosenberger WF, Lachin JM. The use of response-adaptive designs in clinical trials. Control Clin Trials. 1993;14(6):471–484. doi:10.1016/0197-2456(93)90028-C
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/0197-2456(93)90028-C&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=8119063&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1993MQ65300002&link_type=ISI) 

6.  6.Hu F, Rosenberger WF. The Theory of Response-Adaptive Randomization in Clinical Trials. Wiley Blackwell; 2006. doi:10.1002/047005588X
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/047005588X&link_type=DOI) 

7.  7.Meurer WJ, Lewis RJ, Berry DA. Adaptive clinical trials: A partial remedy for the therapeutic misconception? JAMA - J Am Med Assoc. 2012;307(22):2377–2378. doi:10.1001/jama.2012.4174
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2012.4174&link_type=DOI) 

8.  8.Park JJ, Thorlund K, Mills EJ J.J.H. P K. T. Critical concepts in adaptive clinical trials. Clin Epidemiol. 2018;10:343–351. doi:[http://dx.doi.org/10.2147/CLEP.S156708](http://dx.doi.org/10.2147/CLEP.S156708)
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2147/CLEP.S156708&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29606891&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

9.  9.Bothwell LE, Avorn J, Khan NF, Kesselheim AS. Adaptive design clinical trials: A review of the literature and ClinicalTrials.gov. BMJ Open. 2018;8(2). doi:10.1136/bmjopen-2017-018320
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC8yL2UwMTgzMjAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wNy8xNS8yMDIyLjA3LjEzLjIyMjc3NTk2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10. 10.Park JW, Liu MC, Yee D, et al. Adaptive Randomization of Neratinib in Early Breast Cancer. N Engl J Med. 2016;375(1):11–22. doi:10.1056/NEJMoa1513750
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa1513750&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27406346&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

11. 11.Rugo HS, Olopade OI, DeMichele A, et al. Adaptive Randomization of Veliparib–Carboplatin Treatment in Breast Cancer. N Engl J Med. 2016;375(1):23–34. doi:10.1056/NEJMoa1513749
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa1513749&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27406347&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

12. 12.Brueckner M, Titman A, Jaki T, Rojek A, Horby P. Performance of different clinical trial designs to evaluate treatments during an epidemic. PLoS One. 2018;13(9). doi:10.1371/journal.pone.0203387
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0203387&link_type=DOI) 

13. 13.Lin J, Bunn V. Comparison of multi-arm multi-stage design and adaptive randomization in platform clinical trials. Contemp Clin Trials. 2017;54:48–59. doi:10.1016/j.cct.2017.01.003
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.cct.2017.01.003&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

14. 14.Wason JMSS, Trippa L. A comparison of Bayesian adaptive randomization and multi-stage designs for multi-arm clinical trials. Stat Med. 2014;33(13):2206–2221. doi:10.1002/sim.6086
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.6086&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24421053&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

15. 15.Rosenberger WF, Lachin JM. Randomization in Clinical Trials : Theory and Practice. New York: Wiley; 2002.
    
    

16. 16.Pallmann P, Bedding AW, Choodari-Oskooei B, et al. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018;16(1):29. doi:10.1186/s12916-018-1017-7
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12916-018-1017-7&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29490655&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

17. 17.Berry DA. Bayesian clinical trials. Nat Rev Drug Discov. 2006;5(1):27–36. doi:10.1038/nrd1927
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/nrd1927&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16485344&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000234555300015&link_type=ISI) 

18. 18.Berry D. Emerging innovations in clinical trial design. Clin Pharmacol Ther. 2016;99(1):82–91. doi:10.1002/cpt.285
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/cpt.285&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26561040&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

19. 19.Madani Kia T, Marshall JC, Murthy S. Stakeholder perspectives on adaptive clinical trials: a scoping review. doi:10.1186/s13063-020-04466-0
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s13063-020-04466-0&link_type=DOI) 

20. 20.Resch KL. Pragmatic randomised controlled trials for complex therapies. Forsch Komplementarmed. 1998;5(1):136–139. doi:10.1159/000057335
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1159/000057335&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9761995&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

21. 21.Thall PF, Fox P, Wathen J. Statistical controversies in clinical research: Scientific and ethical problems with adaptive randomization in comparative clinical trials. Ann Oncol. 2015;26(8):1621–1628. doi:10.1093/annonc/mdv238
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/annonc/mdv238&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25979922&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

22. 22.Korn EL, Freidlin B. Adaptive Clinical Trials: Advantages and Disadvantages of Various Adaptive Design Elements. J Natl Cancer Inst. 2017;109(6). doi:10.1093/jnci/djx013
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jnci/djx013&link_type=DOI) 

23. 23.Wathen JK, Thall PF. A simulation study of outcome adaptive randomization in multi-arm clinical trials. Clin Trials J Soc Clin Trials. 2017;14(5):432–440. doi:10.1177/1740774517692302
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1740774517692302&link_type=DOI) 

24. 24.Morgan CC, Huyck S, Jenkins M, et al. Adaptive Design: Results of 2012 Survey on Perception and Use. Ther Innov Regul Sci. 2014;48(4):473–481. doi:10.1177/2168479014522468
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/2168479014522468&link_type=DOI) 

25. 25.Karrison T, Huo D, Chappell R. A group sequential, response-adaptive design for randomized clinical trials. Control Clin Trials. 2003;24(5):506–522.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0197-2456(03)00092-8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=14500050&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000185713600002&link_type=ISI) 

26. 26.Richardson S, Hirsch JS, Narasimhan M, et al. Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the New York City Area. JAMA. April 2020. doi:10.1001/jama.2020.6775
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.6775&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32320003&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

27. 27.Grasselli G, Zangrillo A, Zanella A, et al. Baseline Characteristics and Outcomes of 1591 Patients Infected with SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy. JAMA - J Am Med Assoc. 2020;323(16):1574–1581. doi:10.1001/jama.2020.5394
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.5394&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32250385&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

28. 28.Myers LC, Parodi SM, Escobar GJ, Liu VX. Characteristics of Hospitalized Adults With COVID-19 in an Integrated Health Care System in California. JAMA. April 2020. doi:10.1001/jama.2020.7202
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.7202&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32329797&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

29. 29.Guan WJ, Ni ZY, Hu Y, et al. Clinical Characteristics of Coronavirus Disease 2019 in China. N Engl J Med. April 2020. doi:10.1056/NEJMoa2002032
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2002032&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32109013&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

30. 30.Pan A, Liu L, Wang C, et al. Association of Public Health Interventions With the Epidemiology of the COVID-19 Outbreak in Wuhan, China. JAMA. 2020;323(19):1915. doi:10.1001/jama.2020.6130
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.6130&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32275295&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

31. 31.Soriano V, Barreiro P. Why such excess of mortality for COVID-19 in Spain? Ther Adv Infect Dis. 2020;7:204993612093275. doi:10.1177/2049936120932755
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/2049936120932755&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

32. 32.Villar SS, Bowden J, Wason J. Response-adaptive designs for binary responses: How to offer patient benefit while being robust to time trends? Pharm Stat. 2018;17(2):182–197. doi:10.1002/pst.1845
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/pst.1845&link_type=DOI) 

33. 33.Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38(11):2074–2102. doi:10.1002/sim.8086
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.8086&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

34. 34.Monks T, Currie CSM, Onggo BS, Robinson S, Kunc M, Taylor SJE. Strengthening the reporting of empirical simulation studies: Introducing the STRESS guidelines. J Simul. 2019;13(1):55–67. doi:10.1080/17477778.2018.1442155
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/17477778.2018.1442155&link_type=DOI) 

35. 35.Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humanities Research Council of Canada. Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans.; 2018. [https://ethics.gc.ca/eng/documents/tcps2-2018-en-interactive-final.pdf](https://ethics.gc.ca/eng/documents/tcps2-2018-en-interactive-final.pdf).
    
    

36. 36.Ventz S, Barry WT, Parmigiani G, Trippa L. Bayesian Response-Adaptive Designs for Basket Trials. Biometrics. 2017;73(3):905–915. doi:10.1111/biom.12668
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/biom.12668&link_type=DOI) 

37. 37.Villar SS, Wason J, Bowden J. Response-adaptive randomization for multi-arm clinical trials using the forward looking Gittins index rule. Biometrics. 2015;71(4):969–978. doi:10.1111/biom.12337
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/biom.12337&link_type=DOI) 

38. 38.Horby P, Lim WS, Emberson J, et al. Dexamethasone for COVID-19-Preliminary Report Effect of Dexamethasone in Hospitalized Patients with COVID-19-Preliminary Report RECOVERY Collaborative Group*. medRxiv. June 2020:2020.06.22.20137273. doi:10.1101/2020.06.22.20137273
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNi4yMi4yMDEzNzI3M3YxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMTUvMjAyMi4wNy4xMy4yMjI3NzU5Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

39. 39.Wu Z, McGoogan JM. Characteristics of and Important Lessons from the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72314 Cases from the Chinese Center for Disease Control and Prevention. JAMA - J Am Med Assoc. 2020;323(13):1239–1242. doi:10.1001/jama.2020.2648
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.2648&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32091533&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

40. 40.Lobo A de P, Cardoso-Dos-Santos AC, Rocha MS, et al. COVID-19 epidemic in Brazil: where we at? Int J Infect Dis. 2020;97:382. doi:10.1016/j.ijid.2020.06.044
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijid.2020.06.044&link_type=DOI) 

41. 41.Armstrong RA, Kane AD, Cook TM. Outcomes from intensive care in patients with COVID-19: a systematic review and meta-analysis of observational studies. Anaesthesia. June 2020:anae.15201. doi:10.1111/anae.15201
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/anae.15201&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

42. 42.Intensive Care National Audit and Research Center. Report on COVID-19 in Critical Care. London, England; 2020. [https://www.icnarc.org/Our-Audit/Audits/Cmp/Reports](https://www.icnarc.org/Our-Audit/Audits/Cmp/Reports).
    
    

43. 43.The RECOVERY Collaborative Group. Dexamethasone in Hospitalized Patients with Covid-19 — Preliminary Report. N Engl J Med. July 2020. doi:10.1056/nejmoa2021436
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/nejmoa2021436&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32678530&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

44. 44.Horby P, Mafham M, Linsell L, et al. Effect of Hydroxychloroquine in Hospitalized Patients with COVID-19: Preliminary results from a multi-centre, randomized, controlled trial. medRxiv. July 2020:2020.07.15.20151852. doi:10.1101/2020.07.15.20151852
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNy4xNS4yMDE1MTg1MnYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMTUvMjAyMi4wNy4xMy4yMjI3NzU5Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

45. 45.Martini M, Gazzaniga V, Bragazzi NL, Barberis I. The Spanish Influenza Pandemic: a lesson from history 100 years after 1918 War and disease: the spread of the global influenza pandemic. J Prev Med Hyg. 2019;60(1):E64. doi:10.15167/2421-4248/jpmh2019.60.1.1205
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15167/2421-4248/jpmh2019.60.1.1205&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

46. 46.World Health Organization. Solidarity clinical trial for COVID-19 treatments. [https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments](https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments). Published 2020. Accessed July 5, 2020.
    
    

47. 47.Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the Treatment of Covid-19 — Preliminary Report. N Engl J Med. May 2020. doi:10.1056/nejmoa2007764
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/nejmoa2007764&link_type=DOI) 

48. 48.Hauck WW, Anderson S, Marcus SM. Should we adjust for covariates in nonlinear regression analyses of randomized trials? Control Clin Trials. 1998;19(3):249–256. doi:10.1016/S0197-2456(97)00147-5
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0197-2456(97)00147-5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=9620808&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000073761100002&link_type=ISI) 

49. 49.Villar SS, Wason J, Bowden J. Patient drift and response-adaptive randomisation: impact and solutions. Trials. 2015;16(2). doi:10.1186/1745-6215-16-S2-P232
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1745-6215-16-S2-P232&link_type=DOI) 

50. 50.McElreath R. Statistical Rethinking: A Bayesian Course with Examples in R and Stan.; 2018. doi:10.1201/9781315372495
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1201/9781315372495&link_type=DOI) 

51. 51.Spiegelhalter DJ, Abrams KR (Keith R., Myles JP. Bayesian Approaches to Clinical Trials andHealth-Care Evaluation. John Wiley & Sons; 2004.
    
    

52. 52.Corporation M, Weston S. doParallel: Foreach Parallel Adaptor for the “parallel” Package. 2019. [https://cran.r-project.org/package=doParallel](https://cran.r-project.org/package=doParallel).
    
    

53. 53.Microsoft, Weston S. foreach: Provides Foreach Looping Construct. 2020. [https://cran.r-project.org/package=foreach](https://cran.r-project.org/package=foreach).
    
    

54. 54.Wickham et. al. Welcome to the Tidyverse. J Open Source Softw. 2019;4(43):1686. [https://doi.org/10.21105/joss.01686](https://doi.org/10.21105/joss.01686).
    
    

55. 55.Gómez-Rubio V. Ggplot2 - Elegant Graphics for Data Analysis (2nd Edition). Vol 77. Springer-Verlag New York; 2017. doi:10.18637/jss.v077.b02
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.18637/jss.v077.b02&link_type=DOI) 

56. 56.Carpenter B, Gelman A, Hoffman MD, et al. Stan: A probabilistic programming language. J Stat Softw. 2017;76(1):1–32. doi:10.18637/jss.v076.i01
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.18637/jss.v076.i01&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16401268&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

57. 57.Bürkner PC. brms: An R package for Bayesian multilevel models using Stan. J Stat Softw. 2017;80(1):1–28. doi:10.18637/jss.v080.i01
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.18637/jss.v080.i01&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31532012&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

58. 58.R Core Team. R: A Language and Environment for Statistical Computing. 2020. [https://www.r-project.org](https://www.r-project.org).
    
    

59. 59.Grayling MJ, Wheeler GM. A review of available software for adaptive clinical trial design. Clin Trials. 2020;17(3). doi:10.1177/1740774520906398
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1740774520906398&link_type=DOI) 

60. 60.Loken C, Gruner D, Groer L, et al. {SciNet}: Lessons Learned from Building a Power-efficient Top-20 System and Data Centre. J Phys Conf Ser. 2010;256:12026. doi:10.1088/1742-6596/256/1/012026
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1088/1742-6596/256/1/012026&link_type=DOI) 

61. 61.Ponce M, Van Zon R, Northrup S, et al. Deploying a top-100 supercomputer for large parallel workloads: The Niagara supercomputer. In: ACM International Conference Proceeding Series. New York, NY, USA: Association for Computing Machinery; 2019:1–8. doi:10.1145/3332186.3332195
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1145/3332186.3332195&link_type=DOI) 

62. 62.RECOVERY Trial Chief Investigators. No Clinical Benefit from Use of Hydroxychloroquine in Hospitalised Patients with COVID-19.; 2020. [https://www.recoverytrial.net](https://www.recoverytrial.net). Accessed July 10, 2020.
    
    

63. 63.Santacruz CA, Pereira AJ, Celis E, Vincent JL. Which Multicenter Randomized Controlled Trials in Critical Care Medicine Have Shown Reduced Mortality? A Systematic Review. Crit Care Med. 2019;47(12):1680–1691. doi:10.1097/CCM.0000000000004000
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/CCM.0000000000004000&link_type=DOI) 

64. 64.Hey SP, Kimmelman J. Are outcome-adaptive allocation trials ethical? Clin Trials. 2015;12(2):102–106. doi:10.1177/1740774514563583
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1740774514563583&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25649106&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 

65. 65.Viele K, Saville BR, McGlothlin A, Broglio K. Comparison of response adaptive randomization features in multiarm clinical trials with control. Pharm Stat. 2020. doi:10.1002/pst.2015
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/pst.2015&link_type=DOI) 

66. 66.Grieve AP, Krams M. ASTIN: A Bayesian adaptive dose-response trial in acute stroke. In: Clinical Trials. Vol 2. Clin Trials; 2005:340–351. doi:10.1191/1740774505cn094oa
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1191/1740774505cn094oa&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16281432&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F15%2F2022.07.13.22277596.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000232648100013&link_type=ISI)

 [1]: /embed/graphic-2.gif