PT  - JOURNAL ARTICLE
AU  - Raju Kanukula
AU  - Joanne E McKenzie
AU  - Lisa Bero
AU  - Zhaoli Dai
AU  - Sally McDonald
AU  - Cynthia M Kroeger
AU  - Elizabeth Korevaar
AU  - Andrew Forbes
AU  - Matthew J Page
TI  - Investigation of bias due to selective inclusion of study effect estimates in meta-analyses of nutrition research
AID  - 10.1101/2022.11.01.22281823
DP  - 2022 Jan 01
TA  - medRxiv
PG  - 2022.11.01.22281823
4099  - http://medrxiv.org/content/early/2022/11/02/2022.11.01.22281823.short
4100  - http://medrxiv.org/content/early/2022/11/02/2022.11.01.22281823.full
AB  - We aimed to explore, in a sample of systematic reviews with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available, and, (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects. We randomly selected systematic reviews of food/diet and health-related outcomes published between January 2018 and June 2019. We selected the first presented meta-analysis in each review (index meta-analysis), and extracted from study reports all study effect estimates that were eligible for inclusion in the meta-analysis. We calculated the Potential Bias Index (PBI) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 suggest selective inclusion of effect estimates more or less favourable to the intervention, respectively. We investigated the impact of any potential selective inclusion by comparing the index meta-analytic estimate to the median of a randomly constructed distribution of meta-analytic estimates. Thirty-nine systematic reviews with 312 studies were included. The estimated PBI was 0.49 (95% CI 0.42 to 0.55), suggesting the selection of study effect estimates was consistent with a process of random selection. In addition, the impact of any potential selective inclusion on the meta-analytic effects was negligible. Despite this, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta-analyses, which can help readers understand the risk of selective inclusion bias in the systematic reviews.Competing Interest StatementThe authors have declared no competing interest.Clinical Protocols https://f1000research.com/articles/8-1760 Funding StatementThis project was funded by an Australian National Health and Medical Research Council (NHMRC) project grant (APP1139997). RK is supported by a Monash Graduate Scholarship and a Monash International Tuition Scholarship. MJP is supported by an Australian Research Council Discovery Early Career Researcher Award (DE200101618). JEM is supported by an Australian NHMRC Investigator Grant (GNT2009612). SM is supported by the Country Women's Association (NSW) and Edna Winifred Blackman Postgraduate Research Scholarship. The funders had no role in the study design, data collection and analysis, or preparation of the manuscript.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesI confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesData and analytic code are available on the Open Science Framework (https://osf.io/umk62/). https://osf.io/umk62/