Diagnostic accuracy in detecting malignancy in suspicious skin lesions using Artificial Intelligence

Abstract

Background: Artificial Intelligence (AI) has demonstrated a high image processing capacity and improved diagnostic accuracy in dermatology. In this context, Computer-Aided Diagnosis (CAD) systems have shown a diagnostic performance comparable to that of specialists in classifying skin lesions, particularly pigmented lesions. The present study aims to validate Legit.Health is a reliable tool for diagnosing and assessing the severity of patients with skin lesions suspicious of malignacy. Objective: To validate that the Legit.Health medical device optimises clinical workflow by enhancing diagnostic accuracy and determining the malignancy or severity of patients with skin lesions suspicious of malignacy. Methods: An observational, prospective study was conducted, incorporating both longitudinal and retrospective cases. A total of 76 retrospective patients with 88 lesions and 32 prospective patients with 42 lesions attending Instituto de Dermatologia Integral Madrid, Spain, were recruited. The diagnostic performance of Legit.Health was compared with that of dermatologists in the retrospective images against a gold standard (biopsy results). In the prospective phase of the study, the performance of the current Legit.Health medical device was evaluated alongside dermatologists assisted by the device and the latest version of the device (Legit.Health Plus). Analyses were performed to calculate the AUC (area under the curve), accuracy, sensitivity, and specificity. Results: In the retrospective analysis, the device demonstrated an AUC of 0.76 compared to 0.79 for dermatologists in detecting malignant lesions. For these images, the device achieved the following accuracy scores: top-1 = 0.23, top-3 = 0.38, and top-5 = 0.47, whereas dermatologists achieved top-1 = 0.33 and top-3 = 0.45 (providing only three possible diagnoses). When the specific histologic subtype of naevus was not considered in the diagnosis, Legit.Health achieved an accuracy of top-1 = 0.50, top-3 = 0.71, and top-5 = 0.78, compared to dermatologists' top-1 = 0.50 and top-3 = 0.70. In the prospective analysis, we examined the performance of dermatologists using the Legit.Health medical device, the device alone, and the latest version of the device. In the malignancy analysis, they achieved an AUC of 0.94, 0.95, and 0.97, respectively. Regarding diagnostic accuracy, dermatologists assisted by the medical device achieved a top-1 accuracy of 0.30, while both the medical device alone and its latest version achieved top-1 accuracies of 0.22 and 0.26, respectively, which increased to 0.44 and 0.52 when expanding to top-5. When the specific histologic subtype of naevus was not considered in the diagnosis, accuracies increased to 0.85, 0.74, and 0.81, respectively, further improving as top-K was increased to top-5, reaching 0.89 and 0.93, respectively. Conclusions: The device's diagnostic capability in distinguishing malignant conditions is on par with that of expert dermatologists. This confirms its reliability as a tool for detecting skin malignant categories in ICD-11, assisting in prioritising patients based on urgency and directing them to the appropriate specialist or consultation.