Abstract
Clinical trials of new treatments in different progressive diseases use power analysis to determine the sample size needed for a trial to obtain a statistically significant estimate for an anticipated treatment effect. In trials with parallel designs, the standard power analysis approach is based on a two-sample t-test. For example, the standard t-test approach was used in determining the sample size for the Phase 3 trials of aducanumab, the first drug approved by the United States Food and Drug Administration (FDA) to potentially slow cognitive decline in early-stage Alzheimer’s disease. However, t-tests contain normality assumptions, and t-test-based power analyses do not implicitly factor in the uncertainty about anticipated treatment effects that arises due to inter-subject heterogeneity in disease progression. These limitations may lead to recommended sample sizes that are too small, potentially making a trial blind to a treatment effect that is truly present if the cohort’s endpoints are not normally distributed and/or the anticipated treatment effect is overestimated.
To address these issues, we present a novel power analysis method that (1) simulates clinical trials in a progressive disease using real-world data, (2) accounts for inter-subject heterogeneity in disease progression, and (3) does not depend on normality assumptions. As a showcase example, we used our method to calculate power for a range of sample sizes and treatment effects in simulated trials similar to the Phase 3 aducanumab trials EMERGE and ENGAGE. As expected, our results show that power increases with number of subjects and treatment effect (here defined as the cohort-level percent reduction in the rate of cognitive decline in treated subjects vs. controls). However, inclusion of realistic inter-subject heterogeneity in cognitive decline trajectories leads to increased sample size recommendations compared to a standard t-test power analysis. These results suggest that the sample sizes recommended by the t-test power analyses in the EMERGE and ENGAGE Statistical Analysis Plans were possibly too small to ensure a high probability of detecting the anticipated treatment effect. Insufficient sample sizes could partly explain the statistically significant effect of aducanumab being detected only in EMERGE. We also used our method to analyze power in simulated trials similar the Phase 3 lecanemab trial Clarity AD. Our results suggest that Clarity AD was adequately powered, and that power may be influenced by a trial’s number of analysis visits and the characteristics of subgroups within a cohort.
By using our simulation-based power analysis approach, clinical trials of treatments in Alzheimer’s disease and potentially in other progressive diseases could obtain sample size recommendations that account for heterogeneity in disease progression and uncertainty in anticipated treatment effects. Our approach avoids the limitations of t-tests and thus could help ensure that clinical trials are more adequately powered to detect the treatment effects they seek to measure.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer's Association; Alzheimer's Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California. This article was supported by a donation from the Famille Louise & André Charron, and a research grant (Principal Investigator: D. Louis Collins) from the Canadian Institutes of Health Research (CIHR). Daniel Andrews is supported by McGill University and by a doctoral research scholarship from the Canadian Institutes of Health Research (CIHR). Dr. Louis Collins reports receiving research funding from the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC), Brain Canada, the Weston Foundation, and the Famille Louise & André Charron.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (https://adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf Ethical approval for data collection within ADNI was given by the institutional ethics review boards / ethics committees in all participating institutions of the ADNI studies. The ADNI studies received informed consent from all participants before inclusion in the studies. To obtain access to ADNI data, we first registered in the online ADNI system and submitted a written data access application to ADNI describing our research project proposal. Our proposal was evaluated by research personnel at ADNI and was approved. Subsequently, we were granted access to ADNI data, which we used in preparation of this article. ADNI data can be accessed by registering and applying for access at https://adni.loni.usc.edu/data-samples/access-data/.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (https://adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf To obtain access to ADNI data, we first registered in the online ADNI system and submitted a written data access application to ADNI describing our research project proposal. Our proposal was evaluated by research personnel at ADNI and was approved. Subsequently, we were granted access to ADNI data, which we used in preparation of this article. ADNI data can be accessed by registering and applying for access at https://adni.loni.usc.edu/data-samples/access-data/. To maintain data security and save computer storage space, simulated datasets used in preparation of this article were automatically deleted after each simulation experiment described. However, our simulation experiments can be replicated upon request. Researchers can submit a research proposal to ADNI to apply for access to ADNI data; after receiving access approval from ADNI, the ADNI dataset can be used to generate simulated datasets similar to ours by following the procedure outlined in our submitted article.
