We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
A Robust Framework for Multimodal Sentiment Analysis with Noisy Labels Generated from Distributed Data Annotation.
- Authors
Kai Jiang; Bin Cao; Jing Fan
- Abstract
Multimodal sentiment analysis utilizes multimodal data such as text, facial expressions and voice to detect people's attitudes. With the advent of distributed data collection and annotation, we can easily obtain and share such multimodal data. However, due to professional discrepancies among annotators and lax quality control, noisy labels might be introduced. Recent research suggests that deep neural networks (DNNs) will overfit noisy labels, leading to the poor performance of the DNNs. To address this challenging problem, we present a Multimodal Robust Meta Learning framework (MRML) for multimodal sentiment analysis to resist noisy labels and correlate distinct modalities simultaneously. Specifically, we propose a two-layer fusion net to deeply fuse different modalities and improve the quality of the multimodal data features for label correction and network training. Besides, a multiple meta-learner (label corrector) strategy is proposed to enhance the label correction approach and prevent models from overfitting to noisy labels. We conducted experiments on three popular multimodal datasets to verify the superiority of our method by comparing it with four baselines.
- Subjects
SENTIMENT analysis; ARTIFICIAL neural networks; FACIAL expression; TAGS (Metadata); MACHINE learning; ANNOTATIONS
- Publication
CMES-Computer Modeling in Engineering & Sciences, 2024, Vol 139, Issue 3, p2965
- ISSN
1526-1492
- Publication type
Article
- DOI
10.32604/cmes.2023.046348