Abstract
Background Cognitive impairment (CI), including Alzheimer’s disease (AD) and mild cognitive impairment (MCI), has been a major research focus for early diagnosis. Both speech assessment and artificial intelligence (AI) have started to be applied in this field, but faces challenges with limited language type assessment and ethical concerns due to the “black box” nature. Here, we explore a new stragety with patient led non-invasive observation for a novel cross-lingual digital language marker with both diagnostic accuracy, scalability and interpretability.
Methods Speech data was recorded from the cookie theft task in 3 cohorts. And automatic speech recognition (ASR), Networkx package, jieba library and other tools were used to extract visual, acoustic and language features. The SHAP model was used to screen features. Logistic regression and support vector machine and other methods were used to build the model, and an independent cohort was used for external verification. Finally, we used AIGC technology to further reproduce the entire task process.
Results In Chinese environment, we built 3 models of NC/aMCI, NC/AD, and NC/CI (aMCI+AD) through Cohort 1 (NC n=57, aMCI n=62, AD n=66), with accuracy rates of 0.83, 0.79, and 0.79 respectively. The accuracy was 0.75 in the external scalability verification of Cohort 3 (NC n=38, CI n=62). Finally, we built a cross-lingual (Chinese and English) model through Cohort 1 and 2, built a NC/aMCI diagnosis model, and the diagnostic accuracy rate was 0.76. Lastly, we successfully recreate the testing process through Text-to-Image’ and Animation Generation.
Discussion The visual features created by our research group and combines acoustic and linguistic features were used to build a model for early diagnosis of cognitive impairment, and a cross-lingual model covering English and Chinese, which performs well in external verification of independent cohorts. Finally, we innovatively used AI-generated videos to show the subject’s task process to the physician to assist in judging the patient’s diagnosis.
Keyword: Alzheimer’s disease, Amnestic mild cognitive impairment, speech test, Artificial Intelligence, interpretability
Competing Interest Statement
Junhui Gao, Rundong Tan, Yuyuan Jia, Xinjue Zhang, Chen Zhang, Dake Yang are current employees of Shanghai Nuanhe Brain Technology Co. Ltd., Shanghai, China. Other authors declare that they have no competing interests.
Clinical Trial
ChiCTR2000036718
Funding Statement
This study was funded by the Ministry of Science and Technology of the People's Republic of China (2021ZD0201804).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
This study has been approved by the Ethics Committee of Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present study are available upon reasonable request to the authors