Abstract
Numerous studies assessing the effectiveness of non-pharmaceutical interventions (NPIs) against COVID-19 have produced conflicting results, partly due to methodological differences. This study aims to clarify these discrepancies by comparing two frequently used approaches in terms of parameter bias and confidence interval coverage of NPI effectiveness parameters. We compared two-step approaches, where NPI effects are regressed on by-products of a first analysis, such as the effective reproduction number ℛt, with more integrated models that jointly estimate NPI effects and transmission rates in a single-step approach. We simulated datasets with mechanistic and an agent-based models and analyzed them with both mechanistic models and a two-step regression procedure. In the latter, ℛt was estimated first and then used as the outcome in a linear regression with NPI variables as predictors. Mechanistic models consistently outperformed two-step regressions, exhibiting minimal bias (0-5%) and accurate confidence interval coverage. Conversely, the two-step regression showed up to 25% bias, with significantly lower-than-nominal confidence interval coverage, reflecting challenges in uncertainty propagation. We identified additional challenges in the two-step regression method, such high depletion of susceptibles and time lags in observational data. Our findings suggest caution when using two-step regression methods for estimating NPI effectiveness.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study did not receive any funding.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present work are simulated. Data produced in the study are available upon reasonable request to the authors or can be re-simulated from the scripts available on GitHub (https://github.com/sistm/SEIR\_vs\_RTreg)
