ABSTRACT
Aims This study assessed an artificial intelligence (AI) model’s performance in predicting elevated brain natriuretic peptide (BNP) levels from chest radiograms and its effect on human diagnostic performance.
Methods and results Patients who underwent chest radiography and BNP testing on the same day were included. Data were sourced from two hospitals: one for model development, and the other for external testing. Two final ensemble models were developed to predict elevated BNP levels of >= 200 pg/mL and >= 100 pg/mL, respectively. Humans were evaluated to predict elevated BNP levels, followed by the same test, referring to the AI model’s predictions. The 8390 images from 1334 patients were collected for model creation, and 1713 images from 273 patients for tests. The AI model achieved an accuracy of 0.855, precision of 0.873, sensitivity of 0.827, specificity of 0.882, f1 score of 0.850, and receiver-operating-characteristics area-under-curve of 0.929. The accuracy of the testing with the 100 images by 35 participants significantly improved from 0.708±0.049 to 0.829±0.069 (P < 0.001) with the AI assistance (an accuracy of 0.920). Without the AI assistance, the accuracy of the experts was higher than that of non-experts (0.728±0.051 vs. 0.692±0.042, P = 0.030); however, with the AI assistance, the accuracy of the non-experts was rather higher than that of the experts (0.851±0.074 vs. 0.803±0.054, P = 0.033).
Conclusion The AI model can predict elevated BNP levels from chest radiograms and has the potential to improve human performance. The gap in utilizing new tools represents one of the emerging issues.
We developed AI models using an ensemble method to predict elevated BNP levels. The AI model achieved a higher accuracy rate than any individual participant. While the accuracy of experts was higher in the non-assisted test, with the AI assistance, the accuracy of non-experts surpassed that of the experts. AI, artificial intelligence; AUC, area-under-curve; BNP, brain natriuretic peptide; GPU, graphic processing unit; PR, precision-recall; ROC, receiver-operating-characteristics.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
There is no financial support for this study.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
IRB of Hiroshima City Asa Hospital gave ethical approval for this work.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
Funding: There is no financial support for this study.
Disclosures: The authors have reported that they have no relationships relevant to the contents of this paper to disclose.
This revision is for formatting adjustments for submission to a journal.
Data Availability
The source codes and weights of the models used in this study will be available on GitHub.
