PT  - JOURNAL ARTICLE
AU  - Miller, Sarah L.
AU  - Tuia, Jordan
AU  - Prasad, Vinay
TI  - Interpretation of wide confidence intervals in meta-analytic estimates: Is the ‘Absence of Evidence’ ‘Evidence of Absence’?
AID  - 10.1101/2023.07.11.23292513
DP  - 2023 Jan 01
TA  - medRxiv
PG  - 2023.07.11.23292513
4099  - http://medrxiv.org/content/early/2023/07/14/2023.07.11.23292513.short
4100  - http://medrxiv.org/content/early/2023/07/14/2023.07.11.23292513.full
AB  - Introduction Recently, a Cochrane review by Jefferson et al. on physical interventions to slow the spread of respiratory viruses concluded that, “Wearing masks in the community probably makes little or no difference to the outcome of laboratory-confirmed influenza/SARS-CoV-2 compared to not wearing masks”, though this finding had a wide confidence interval. Cochrane issued a rare clarifying statement, fueling controversy. We sought to contextualize the findings of the review by Jefferson et al.Methods We searched for consecutive reviews by Cochrane published on or before March 9th, 2023. We included studies where a central finding showed an intervention offered no statistically significant benefit, and ascertained the language used by reviewers to describe that result. We compare this to the report by Jefferson et al., and deemed it consistent or inconsistent with the language of their report.Results We found between November 21st, 2022, and March 9th, 2023, there were 20 Cochrane reviews that met the inclusion criteria. We found that 95% (n = 19) of the reviews used language that was consistent with Jefferson’s findings, while 5% (n = 1) used language inconsistent with Jefferson’s conclusion, describing the effect of the intervention on the outcome as “unclear”.Discussion Most reviews performed by Cochrane conclude that interventions which fail to show statistically significant benefits make “no difference” have “no effect” or do not “increase or decrease” the outcome, and this occurs despite wide confidence intervals. The conclusions by Jefferson et al. are consistent with Cochrane reporting guidelines and clarification from the organization was unjustified.Competing Interest StatementDisclosure: Vinay Prasad′s Disclosures. (Research funding) Arnold Ventures (Royalties) Johns Hopkins Press, Medscape, and MedPage (Honoraria) Grand Rounds/lectures from universities, medical centers, non–profits, and professional societies. (Consulting) UnitedHealthcare and OptumRX. (Other) Plenary Session podcast has Patreon backers, YouTube, and Substack. All other authors have no financial nor non–financial conflicts of interest to report.Funding StatementNone. VP is funded by Arnold Ventures to study Low Value Care. This study received no specific funding.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesI confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesAll data produced in the present work are contained in the manuscript