Skip to main content

Advertisement

Log in

Ethical Considerations for Increased Transparency and Reproducibility in the Retrospective Analysis of Health Care Data

  • Ethics Special Section: Analytical Report
  • Published:
Therapeutic Innovation & Regulatory Science Aims and scope Submit manuscript

Abstract

In the field of health care, researchers and decision makers are increasingly turning toward retrospective observational studies of administrative claims and electronic health record databases to improve outcomes for patients. For many important questions, randomized studies have not been conducted, and even when they have been, such studies often inadequately reflect the realities of patients’ lives or care. However, use of retrospective studies not only increases methodological complexity but also requires more subjectivity for those attempting to perform statistical analysis. The hurdles for establishing the reproducibility of such research to ensure accuracy and generalizability are therefore also higher, as are the requirements for transparency to limit the impact of bias. The ethical statistical practitioner will therefore need to take additional steps to enable results to be interpreted and acted upon with confidence. These include increased transparency regarding the impact of database selection, database quality, database content, and design decisions on the robustness of statistical conclusions. A number of approaches to increase the reproducibility of retrospective health care research are also presented, along with some discussion regarding responsibilities of data owners, statistical practitioners, publishers, and users of results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):e124.

    Article  Google Scholar 

  2. World Medical Association. WMA Declaration of Helsinki—Ethical Principles for Medical Research Involving Human Subjects. http://www.wma.net/en/30publications/10policies/b3/. Accessed November 3, 2014.

  3. US Department of Health and Human Services. Health information privacy. http://www.hhs.gov/ocr/privacy/. Accessed November 3, 2014.

  4. Faden RR, Beauchamp TL, Kass NE. Informed consent, comparative effectiveness, and learning health care. N Engl J Med. 2014;370(8):766–768.

    Article  CAS  Google Scholar 

  5. Platt R, Kass NE, McGraw D. Ethics, regulation, and comparative effectiveness research: time for a change. JAMA. 2014;311(15):1497–1498.

    Article  CAS  Google Scholar 

  6. Patient-Centered Outcomes Research Institute Methodology Committee. The PCORI methodology report. http://www.pcori.org/content/pcori-methodology-report. Posted November 19, 2013. Updated July 15, 2014. Accessed February 10, 2015.

  7. vonElm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370:1453–1457.

    Article  Google Scholar 

  8. Berger ML, Mamdani M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force report—part I. Value Health. 2009;12(8):1044–1052.

    Article  Google Scholar 

  9. Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML. Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force report—part II. Value Health. 2009;12(8):1053–1061.

    Article  Google Scholar 

  10. Johnson ML, Crown W, Martin BC, Dormuth CR, Siebert U. Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force report—part III. Value Health. 2009;12(8):1062–1073.

    Article  Google Scholar 

  11. Methods guide for effectiveness and comparative effectiveness reviews. AHRQ Publication No. 10(14)-EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality. http://www.effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&mp=1&productID=318. Accessed April 1, 2015.

  12. Dreyer NA, Schneeweiss S, McNeil BJ, et al. GRACE principles: recognizing high-quality observational studies of comparative effectiveness. Am J Manag Care. 2010;16(6):467–471.

    PubMed  Google Scholar 

  13. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–2012.

    Article  CAS  Google Scholar 

  14. American Statistical Association. Ethical guidelines for statistical practice. http://www.amstat.org/committees/ethics/. Accessed November 3, 2014.

  15. Mini-Sentinel. Mini-Sentinel Methods Website. http://www.mini-sentinel.org/methods/default.aspx. Accessed April 1, 2015.

  16. World Health Organization. International Classification of Diseases. http://www.who.int/classifications/icd/en/. Accessed April 1, 2015.

  17. Girman CJ, Faries D, Ryan P, et al. Pre-study feasibility and identifying sensitivity analyses for protocol pre-specification in comparative effectiveness research. J Comp Eff Res. 2014;3(3):259–270.

    Article  Google Scholar 

  18. Walker AM, Patrick AR, Lauer MS, et al. A tool for assessing the feasibility of comparative effectiveness research. Comp Eff Res. 2013;3:11–20.

    Google Scholar 

  19. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15:291–303.

    Article  Google Scholar 

  20. Ryan PB, Madigan D, Stang PE, Overhage JM, Racoosin JA, Hartzema AG. Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership. Stat Med. 2012;31:4401–4415.

    Article  Google Scholar 

  21. Observational Medical Outcomes Partnership. OMOP 2012 Symposium Presentations. http://omop.org/2012SymposiumPresentations. Accessed November 3, 2014.

  22. Royall RM. The effect of sample size on the meaning of significance tests. Am Stat. 1986;40(4):313–315.

    Google Scholar 

  23. Ioannidis JPA. Why most discovered true associations are inflated. Epidemiology. 2008;19(5):640–648.

    Article  Google Scholar 

  24. Christley RM. Power and error: increased risk of false positive results in underpowered studies. Open Epidemiol J. 2010;3:16–19.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Matthew D. Rotelli PhD.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rotelli, M.D. Ethical Considerations for Increased Transparency and Reproducibility in the Retrospective Analysis of Health Care Data. Ther Innov Regul Sci 49, 342–347 (2015). https://doi.org/10.1177/2168479015578155

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1177/2168479015578155

Keywords

Navigation