We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Multilingual Prediction of Cognitive Impairment with Large Language Models and Speech Analysis.
- Authors
Agbavor, Felix; Liang, Hualou
- Abstract
Background: Cognitive impairment poses a significant global health challenge, emphasizing the critical need for early detection and intervention. Traditional diagnostics like neuroimaging and clinical evaluations are often subjective, costly, and inaccessible, especially in resource-poor settings. Previous research has focused on speech analysis primarily conducted using English data, leaving multilingual settings unexplored. Methods: In this study, we present our results from the INTERSPEECH 2024 TAUKADIAL Challenge, where we aimed to automatically detect mild cognitive impairment (MCI) and predict cognitive scores for English and Chinese speakers (169 in total). Our approach leverages Whisper, a speech foundation model, to extract language-agnostic speech embeddings. We then utilize ensemble models to incorporate task-specific information. Results: Our model achieved unweighted average recall of 81.83% in an MCI classification task, and root mean squared error of 1.196 in cognitive score prediction task, which placed the model at the second and the first position, respectively, in the ranking for each task. Comparison between language-agnostic and language-specific models reveals the importance of capturing language-specific nuances for accurate cognitive impairment prediction. Conclusions: This study demonstrates the effectiveness of language-specific ensemble modeling with Whisper embeddings in enabling scalable, non-invasive cognitive health assessments of Alzheimer's disease, achieving state-of-the-art results in multilingual settings.
- Subjects
LANGUAGE models; MILD cognitive impairment; STANDARD deviations; ALZHEIMER'S disease; COGNITION disorders; MULTILINGUALISM
- Publication
Brain Sciences (2076-3425), 2024, Vol 14, Issue 12, p1292
- ISSN
2076-3425
- Publication type
Academic Journal
- DOI
10.3390/brainsci14121292