We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
Exploring hybrid spatio-temporal convolutional networks for human action recognition.
- Authors
Deng, Cheng; Wang, Hao; Yang, Yanhua; Yang, Erkun
- Abstract
Convolutional neural networks have achieved great success in many computer vision tasks. However, it is still challenging for action recognition in videos due to the intrinsically complicated space-time correlation and computational difficult of videos. Existing methods usually neglect the fusion of long term spatio-temporal information. In this paper, we propose a novel hybrid spatio-temporal convolutional network for action recognition. Specifically, we integrate three different type of streams into the network: (1) the image stream utilizes still images to learn the appearance information; (2) the optical stream captures the motion information from optical flow frames; (3) the dynamic image stream explores the appearance information and motion information simultaneously from generated dynamic images. Finally, a weighted fusion strategy at the softmax layer is utilized to make the class decision. With the help of these three streams, we can take full advantage of the spatio-temporal information of the videos. Extensive experiments on two popular human action recognition datasets demonstrate the superiority of our proposed method when compared with several state-of-the-art approaches.
- Subjects
PATTERN recognition systems; HUMAN mechanics; ARTIFICIAL neural networks; MATHEMATICAL convolutions; SPATIOTEMPORAL processes; COMPUTER vision; MATHEMATICAL models
- Publication
Multimedia Tools & Applications, 2017, Vol 76, Issue 13, p15065
- ISSN
1380-7501
- Publication type
Article
- DOI
10.1007/s11042-017-4514-3