We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
TEMM: text-enhanced multi-interactive attention and multitask learning network for multimodal sentiment analysis.
- Authors
Yu, Bengong; Shi, Zhongyu
- Abstract
Multimodal sentiment analysis is an important and active research field. Most methods construct fusion modules based on unimodal representations generated by pretrained models, which lack the deep interaction of multimodal information, especially the rich semantic-emotional information embedded in text. In addition, previous studies have focused on capturing modal coherence information and ignored differentiated information. We propose a text-enhanced multi-interactive attention and multitask learning network (TEMM). First, syntactic dependency graphs and sentiment graphs of the text are constructed, and additional graph embedding representations of the text are obtained using graph convolutional networks and graph attention networks. Then, self-attention and cross-modal attention are applied to explore intramodal and intermodal dynamic interactions, using text as the main cue. Finally, a multitask learning framework is constructed to exert control over the information flow by monitoring the mutual information between the unimodal and multimodal representations and exploiting the classification properties of the unimodal modality to achieve a more comprehensive focus on modal information. The experimental results on the CMU-MOSI, CMU-MOSEI, and CH-SIMS datasets show that the proposed model outperforms state-of-the-art models.
- Subjects
KNOWLEDGE graphs; SENTIMENT analysis; INFORMATION resources management; CLASSIFICATION
- Publication
Journal of Supercomputing, 2024, Vol 80, Issue 17, p25563
- ISSN
0920-8542
- Publication type
Article
- DOI
10.1007/s11227-024-06422-0