ResLNet: deep residual LSTM network with longer input for action recognition - Université de technologie de Troyes Accéder directement au contenu
Article Dans Une Revue Frontiers of Computer Science Année : 2022

ResLNet: deep residual LSTM network with longer input for action recognition

Résumé

Action recognition is an important research topic in video analysis that remains very challenging. Effective recognition relies on learning a good representation of both spatial information (for appearance) and temporal information (for motion). These two kinds of information are highly correlated but have quite different properties, leading to unsatisfying results of both connecting independent models (e.g., CNN-LSTM) and direct unbiased co-modeling (e.g., 3DCNN). Besides, a long-lasting tradition on this task with deep learning models is to just use 8 or 16 consecutive frames as input, making it hard to extract discriminative motion features. In this work, we propose a novel network structure called ResLNet (Deep Residual LSTM network), which can take longer inputs (e.g., of 64 frames) and have convolutions collaborate with LSTM more effectively under the residual structure to learn better spatial-temporal representations than ever without the cost of extra computations with the proposed embedded variable stride convolution. The superiority of this proposal and its ablation study are shown on the three most popular benchmark datasets: Kinetics, HMDB51, and UCF101. The proposed network could be adopted for various features, such as RGB and optical flow. Due to the limitation of the computation power of our experiment equipment and the real-time requirement, the proposed network is tested on the RGB only and shows great performance.
Fichier non déposé

Dates et versions

hal-04453693 , version 1 (12-02-2024)

Identifiants

Citer

Tian Wang, Jiakun Li, Huai-Ning Wu, Ce Li, Yang Wu, et al.. ResLNet: deep residual LSTM network with longer input for action recognition. Frontiers of Computer Science, 2022, 16 (6), pp.166334. ⟨10.1007/s11704-021-0236-9⟩. ⟨hal-04453693⟩
7 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More