We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
A novel dual-modal emotion recognition algorithm with fusing hybrid features of audio signal and speech context.
- Authors
Xu, Yurui; Su, Hang; Ma, Guijin; Liu, Xiaorui
- Abstract
With regard to human–machine interaction, accurate emotion recognition is a challenging problem. In this paper, efforts were taken to explore the possibility to complete the feature abstraction and fusion by the homogeneous network component, and propose a dual-modal emotion recognition framework that is composed of a parallel convolution (Pconv) module and attention-based bidirectional long short-term memory (BLSTM) module. The Pconv module employs parallel methods to extract multidimensional social features and provides more effective representation capacity. Attention-based BLSTM module is utilized to strengthen key information extraction and maintain the relevance between information. Experiments conducted on the CH-SIMS dataset indicate that the recognition accuracy reaches 74.70% on audio data and 77.13% on text, while the accuracy of the dual-modal fusion model reaches 90.02%. Through experiments it proves the feasibility to process heterogeneous information within homogeneous network component, and demonstrates that attention-based BLSTM module would achieve best coordination with the feature fusion realized by Pconv module. This can give great flexibility for the modality expansion and architecture design.
- Subjects
EMOTION recognition; SPEECH; DATA mining; ARCHITECTURAL design; ALGORITHMS; TEXT recognition
- Publication
Complex & Intelligent Systems, 2023, Vol 9, Issue 1, p951
- ISSN
2199-4536
- Publication type
Article
- DOI
10.1007/s40747-022-00841-3