PT  - JOURNAL ARTICLE
AU  - Watanabe, Naoki
AU  - Watari, Tomohisa
AU  - Akamatsu, Kenji
AU  - Miyatsuka, Isao
AU  - Otsuka, Yoshihito
TI  - Performance of Deep Learning Models in Predicting the Nugent Score to Diagnose Bacterial Vaginosis
AID  - 10.1101/2024.09.16.24313614
DP  - 2024 Jan 01
TA  - medRxiv
PG  - 2024.09.16.24313614
4099  - http://medrxiv.org/content/early/2024/09/16/2024.09.16.24313614.short
4100  - http://medrxiv.org/content/early/2024/09/16/2024.09.16.24313614.full
AB  - The Nugent score is a commonly used tool for diagnosing bacterial vaginosis; however, its accuracy depends on the skills of laboratory technicians. We aimed to evaluate the performance of deep learning models in predicting the Nugent score, with the goal of improving diagnostic consistency and accuracy. A total of 1,510 vaginal images collected from a hospital in Japan between 2021 and 2023 were assessed. Each image was annotated by laboratory technicians into one of four categories based on the Nugent score—normal vaginal flora, absence of vaginal flora, altered vaginal flora, or bacterial vaginosis. Deep learning models were developed to predict these categories, and their performance was evaluated by comparing the predicted scores with technician annotations. A high magnification model was further optimized and evaluated using an independent test set of 106 images to assess its performance relative to that of the technicians. The deep learning models demonstrated an accuracy of 84% at low magnification and 89% at high magnification in predicting the Nugent score categories. After optimization, the high magnification model achieved 94% accuracy, surpassing the average 92% accuracy of the technicians. The agreement between deep learning model predictions and technician annotations was 92% for normal vaginal flora, 100% for absence of vaginal flora, 91% for altered vaginal flora, and 100% for bacterial vaginosis. The deep learning models demonstrated accuracy comparable to that of laboratory technicians, which indicates their potential utility in improving the diagnostic accuracy of bacterial vaginosis.IMPORTANCE Bacterial vaginosis is a global health issue affecting women, causing symptoms such as abnormal vaginal discharge and discomfort. The Nugent score is the standard method for diagnosing bacterial vaginosis and is based on manual interpretation of Gram-stained vaginal smears. However, this method relies on the skill and experience of trained professionals, leading to variability in results and challenges in facilities with limited access to such experts. This poses significant challenges for settings with limited access to experienced technicians. The deep learning models developed in this study predict the Nugent score with high accuracy; thus, they can be used to standardize the diagnosis of bacterial vaginosis, reduce observer variability, and enable reliable diagnosis even in settings without experienced personnel. Although larger scale validation is needed, our results suggest that deep learning models may represent a new approach for the diagnosis of bacterial vaginosis.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis study did not receive any funding.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:Ethical approval was obtained from the Kameda Medical Center Ethics Committee (approval number 22-128). The requirement for written informed consent from the participants was waived by the Research Ethics Committee because of the exclusive use of anonymized data in this study.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesAll the data supporting the findings are provided in the article. Any additional data are available on request to the corresponding authors.