RT Journal Article
SR Electronic
T1 Expert-validated estimation of diagnostic uncertainty for deep neural networks in diabetic retinopathy detection
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 19002154
DO 10.1101/19002154
A1 Ayhan, Murat Seçkin
A1 Kühlewein, Laura
A1 Aliyeva, Gulnar
A1 Inhoffen, Werner
A1 Ziemssen, Focke
A1 Berens, Philipp
YR 2019
UL http://medrxiv.org/content/early/2019/07/15/19002154.abstract
AB Deep learning-based systems can achieve a diagnostic performance comparable to physicians in a variety of medical use cases including the diagnosis of diabetic retinopathy. To be useful in clinical practise, it is necessary to have well calibrated measures of the uncertainty with which these systems report their decisions. However, deep neural networks (DNNs) are being often overconfident in their predictions, and are not amenable to a straightforward probabilistic treatment. Here, we describe an intuitive framework based on test-time data augmentation for quantifying the diagnostic uncertainty of a state-of-the-art DNN for diagnosing diabetic retinopathy. We show that the derived measure of uncertainty is well-calibrated and that experienced physicians likewise find cases with uncertain diagnosis difficult to evaluate. This paves the way for an integrated treatment of uncertainty in DNN-based diagnostic systems.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis research was supported by the German Ministry of Science and Education (BMBF, 01GQ1601 and 01IS18039A) and the German Science Foundation (BE5601/4-1 and EXC 2064, project number 390727645).Author DeclarationsAll relevant ethical guidelines have been followed and any necessary IRB and/or ethics committee approvals have been obtained.Not ApplicableAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.Not ApplicableAny clinical trials involved have been registered with an ICMJE-approved registry such as ClinicalTrials.gov and the trial ID is included in the manuscript.Not ApplicableI have followed all appropriate research reporting guidelines and uploaded the relevant Equator, ICMJE or other checklist(s) as supplementary files, if applicable.Not ApplicableAll data used in this study are publicly available. More information can be found via the following links: Dataset 1: https://www.kaggle.com/c/diabetic-retinopathy-detection Dataset 2: https://idrid.grand-challenge.org/