Contribution to Gestures Recognition Using Multimodal Signals through Deep Learning

Abstract

This work aims to advance gesture recognition by combining multimodal signals, including surface electromyography (sEMG), accelerometer, and eye-tracking data, using deep learning techniques. The proposed hybrid architecture incorporates Convolutional Neural Networks (CNNs) for feature extraction and Long Short-Term Memory (LSTM) networks for time series processing, aiming to enhance gesture recognition accuracy. The evaluation involves two stages: first, assessing performance with sEMG data alone, and second, evaluating a multimodal dataset with accelerometers and eye-tracking. The MeganePro dataset is used for training deep learning algorithms to improve hand gesture recognition and develop intuitive human-computer interaction control mechanisms. This research significantly contributes to the field of gesture recognition.