Abstract
In the field of health care, researchers and decision makers are increasingly turning toward retrospective observational studies of administrative claims and electronic health record databases to improve outcomes for patients. For many important questions, randomized studies have not been conducted, and even when they have been, such studies often inadequately reflect the realities of patients’ lives or care. However, use of retrospective studies not only increases methodological complexity but also requires more subjectivity for those attempting to perform statistical analysis. The hurdles for establishing the reproducibility of such research to ensure accuracy and generalizability are therefore also higher, as are the requirements for transparency to limit the impact of bias. The ethical statistical practitioner will therefore need to take additional steps to enable results to be interpreted and acted upon with confidence. These include increased transparency regarding the impact of database selection, database quality, database content, and design decisions on the robustness of statistical conclusions. A number of approaches to increase the reproducibility of retrospective health care research are also presented, along with some discussion regarding responsibilities of data owners, statistical practitioners, publishers, and users of results.
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
World Medical Association. WMA Declaration of Helsinki—Ethical Principles for Medical Research Involving Human Subjects. http://www.wma.net/en/30publications/10policies/b3/. Accessed November 3, 2014.
US Department of Health and Human Services. Health information privacy. http://www.hhs.gov/ocr/privacy/. Accessed November 3, 2014.
Faden RR, Beauchamp TL, Kass NE. Informed consent, comparative effectiveness, and learning health care. N Engl J Med. 2014;370(8):766–768.
Platt R, Kass NE, McGraw D. Ethics, regulation, and comparative effectiveness research: time for a change. JAMA. 2014;311(15):1497–1498.
Patient-Centered Outcomes Research Institute Methodology Committee. The PCORI methodology report. http://www.pcori.org/content/pcori-methodology-report. Posted November 19, 2013. Updated July 15, 2014. Accessed February 10, 2015.
vonElm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370:1453–1457.
Berger ML, Mamdani M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force report—part I. Value Health. 2009;12(8):1044–1052.
Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML. Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force report—part II. Value Health. 2009;12(8):1053–1061.
Johnson ML, Crown W, Martin BC, Dormuth CR, Siebert U. Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force report—part III. Value Health. 2009;12(8):1062–1073.
Methods guide for effectiveness and comparative effectiveness reviews. AHRQ Publication No. 10(14)-EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality. http://www.effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&mp=1&productID=318. Accessed April 1, 2015.
Dreyer NA, Schneeweiss S, McNeil BJ, et al. GRACE principles: recognizing high-quality observational studies of comparative effectiveness. Am J Manag Care. 2010;16(6):467–471.
Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–2012.
American Statistical Association. Ethical guidelines for statistical practice. http://www.amstat.org/committees/ethics/. Accessed November 3, 2014.
Mini-Sentinel. Mini-Sentinel Methods Website. http://www.mini-sentinel.org/methods/default.aspx. Accessed April 1, 2015.
World Health Organization. International Classification of Diseases. http://www.who.int/classifications/icd/en/. Accessed April 1, 2015.
Girman CJ, Faries D, Ryan P, et al. Pre-study feasibility and identifying sensitivity analyses for protocol pre-specification in comparative effectiveness research. J Comp Eff Res. 2014;3(3):259–270.
Walker AM, Patrick AR, Lauer MS, et al. A tool for assessing the feasibility of comparative effectiveness research. Comp Eff Res. 2013;3:11–20.
Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15:291–303.
Ryan PB, Madigan D, Stang PE, Overhage JM, Racoosin JA, Hartzema AG. Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership. Stat Med. 2012;31:4401–4415.
Observational Medical Outcomes Partnership. OMOP 2012 Symposium Presentations. http://omop.org/2012SymposiumPresentations. Accessed November 3, 2014.
Royall RM. The effect of sample size on the meaning of significance tests. Am Stat. 1986;40(4):313–315.
Ioannidis JPA. Why most discovered true associations are inflated. Epidemiology. 2008;19(5):640–648.
Christley RM. Power and error: increased risk of false positive results in underpowered studies. Open Epidemiol J. 2010;3:16–19.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rotelli, M.D. Ethical Considerations for Increased Transparency and Reproducibility in the Retrospective Analysis of Health Care Data. Ther Innov Regul Sci 49, 342–347 (2015). https://doi.org/10.1177/2168479015578155
Published:
Issue Date:
DOI: https://doi.org/10.1177/2168479015578155