Abstract
Objectives A recent paper by Doi et al. advocated completely replacing the relative risk (RR) with the odds ratio (OR) as the effect measure used to report the association between a treatment and a binary outcome in clinical trials and meta-analyses. Besides some practical advantages of RR over OR and the well-known issue of the OR being non-collapsible, Doi et al.’s key assumption that the OR is “portable” in the meta-analysis, i.e., study-specific ORs are likely not correlated with baseline risks, was not well justified.
Study designs and settings We summarized the Spearman’s rank correlation coefficient between study-specific OR and the baseline risk in 40,243 meta-analyses from the Cochrane Database of Systematic Reviews (CDSR).
Results Study-specific ORs are negatively correlated with baseline risk of disease (i.e., higher ORs tend to be observed in studies with lower baseline risks of disease) for most meta-analyses in CDSR. Using a meta-analysis comparing the effect of oral sumatriptan (100 mg) versus placebo on mitigating the acute headache at 2 hours after drug administration, we demonstrate that there is a strong negative correlation between OR (RR or RD) with the baseline risk and the conditional effects notably vary with baseline risks.
Conclusions Replacing RR or RD with OR is currently unadvisable in clinical trials and meta-analyses. It is possible that no effect measure is “portable” in a meta-analysis. In cases where portability of the effect measure is challenging to satisfy, we suggest presenting the conditional effect based on the baseline risk using a bivariate generalized linear mixed model. The bivariate generalized linear mixed model can be used to account for correlation between the effect measure and baseline disease risk. Furthermore, in addition to the overall (or marginal) effect, we recommend that investigators also report the effects conditioning on the baseline risk.
Key findings
In most meta-analyses in Cochrane Database of Systematic Reviews, there is notable negative correlation between ORs and baseline risks.
When such a correlation is not negligible, the OR is not “portable” across studies with different baseline risks.
When an effect measure is not “portable”, one may derive the effects conditioning on the baseline risk from a bivariate generalized linear mixed model.
What this study adds to what was known
The recommendation to replace the RR with the OR in clinical trials and meta-analyses is misguided.
The OR is not a better effect summary than RR and RD in a single study or in meta-analyses; the noncollapsibility of OR can lead to misleading results in a single study and the OR is generally not portable in the meta-analysis.
In addition to reporting effect measures such as the OR, RR or RD, it is also important to present the baseline risk.
What is the implication and what should change now?
When none of the effects are “portable” in a meta-analysis, in addition to report the overall (or marginal) effect, one should also report the effects conditioning on the baseline risk, regardless of the measure of choice.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
The authors gratefully acknowledge the NIH National Library of Medicine (R01LM012982).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Not applicable.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The data comes from the Cochrane Database of Systematic Reviews.
