Hand Gesture Recognition Using Convolutional Neural Networks

Authors

  • Veluru Karthik Reddy, Vanapalli Durga Prasanth, R. Shiva Rama Krishna, Naidu Sri lekha, Jyothi N. M.

Keywords:

HPR,CNN,Segmentation,Background Clutter,Virtual Reality,Neural Network

Abstract

Hand gestures play a crucial role in communication and are essential in various scenarios where verbal communication is not possible. For instance, traffic policemen, newsreaders, airport staff, and gamers often rely on hand signals to communicate. Therefore, there is a growing need for robust hand pose recognition (HPR) methods that can identify hand gestures accurately. However, the current state-of-the-art HPR methods struggle with identifying hand gestures in the presence of cluttered backgrounds. To address this challenge, we propose a deep learning framework based on convolutional neural networks (CNNs) to identify hand postures regardless of hand size, location in the image, and background clutter. Our proposed CNN-based approach eliminates the need for feature extraction and learns to recognize hand poses without explicit foreground segmentation. This method effectively identifies hand poses, even in the presence of complex and varying backgrounds or poor lighting conditions. We have conducted several experiments, which demonstrate the superiority of our proposed method over state-of-the-art datasets. Our approach significantly improves the accuracy of hand pose recognition, making it more reliable and efficient for a wide range of applications. Therefore, our proposed method has significant potential for use in real-world scenarios, such as traffic management, sign language interpretation, and virtual reality gaming. Overall, our results suggest that deep neural networks can provide a robust and effective solution for hand gesture recognition tasks.

Downloads

Download data is not yet available.

References

Ke, W., Xing, Y., Di Caterina, G., Petropoulakis, L., & Soraghan, J. (2020). Deep Convolutional Spiking Neural Network Based Hand Gesture Recognition. 2020 International Joint Conference on Neural Networks (IJCNN).

Xu, J., & Jiang, T. (2017). Dynamic Hand Gesture Recognition Based on Parallel HMM Using Wireless Signals. Communications, Signal Processing, and Systems, 749–757.

Chen, Z., Kim, J.-T., Liang, J., Zhang, J., & Yuan, Y.-B. (2014). Real-Time Hand Gesture Recognition Using Finger Segmentation. The Scientific World Journal, 2014, 1–9.

Suk, H.-I., Sin, B.-K., & Lee, S.-W. (2010). Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recognition, 43(9), 3059–3072.

Zhang, C., & Tian, Y. (2013). Edge Enhanced Depth Motion Map for Dynamic Hand Gesture Recognition. 2013 IEEE Conference on Computer Vision and Pattern Recognition

Cheng, H., Luo, J., & Chen, X. (2014). A windowed dynamic time warping approach for 3D continuous hand gesture recognition. 2014 IEEE International Conference on Multimedia and Expo (ICME).

Li, Y.-T., & Wachs, J. P. (2014). HEGM: A hierarchical elastic graph matching for hand gesture recognition. Pattern Recognition, 47(1), 80–88.

Haria, A., Subramanian, A., Asokkumar, N., Poddar, S., & Nayak, J. S. (2017). Hand Gesture Recognition for Human Computer Interaction. Procedia Computer Science, 115, 367–374.

Mohanty, A., Rambhatla, S. S., & Sahay, R. R. (2016). Deep Gesture: Static Hand Gesture Recognition Using CNN. Proceedings of International Conference on Computer Vision and Image Processing, 449–461

Kopuklu, O., Kose, N., & Rigoll, G. (2018). Motion Fused Frames: Data Level Fusion Strategy for Hand Gesture Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

Abavisani, M., Joze, H. R. V., & Patel, V. M. (2019). Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition With Multimodal Training. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

Al-Hammadi, M., Muhammad, G., Abdul, W., Alsulaiman, M., Bencherif, M. A., Alrayes, T. S., Mathkour, H., & Mekhtiche, M. A. (2020). Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation. IEEE Access, 8, 192527–192542.

Li, C., Xie, C., Zhang, B., Chen, C., & Han, J. (2018). Deep Fisher discriminant learning for mobile hand gesture recognition. Pattern Recognition, 77, 276–288.

Mo, G. B., Dudley, J. J., & Kristensson, P. O. (2021). Gesture Knitter: A Hand Gesture Design Tool for Head-Mounted Mixed Reality Applications. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.

P. Parvathy, K. Subramaniam, G. K. D. Prasanna Venkatesan, P. Karthikaikumar, J. Varghese, and T. Jayasankar, “RETRACTED ARTICLE: Development of hand gesture recognition system using machine learning,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 6, pp. 6793–6800, Jul. 2020.

Hasan, H., & Abdul-Kareem, S. (2012). RETRACTED ARTICLE: Static hand gesture recognition using neural networks. Artificial Intelligence Review, 41(2), 147–181.

Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015). Hand gesture recognition with 3D convolutional neural networks. 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

Murthy, G. R. S., & Jadon, R. S. (2010). Hand gesture recognition using neural networks. 2010 IEEE 2nd International Advance Computing Conference (IACC). https://doi.org/10.1109/iadcc.2010.5423024

Neto, P., Pereira, D., Pires, J. N., & Moreira, A. P. (2013). Real-time and continuous hand gesture spotting: An approach based on artificial neural networks. 2013 IEEE International Conference on Robotics and Automation.

Alam, M. M., Islam, M. T., & Rahman, S. M. M. (2022). Unified learning approach for egocentric hand gesture recognition and fingertip detection. Pattern Recognition, 121, 108200.

Downloads

Published

24.03.2024

How to Cite

Vanapalli Durga Prasanth, R. Shiva Rama Krishna, Naidu Sri lekha, Jyothi N. M., V. K. R. (2024). Hand Gesture Recognition Using Convolutional Neural Networks. International Journal of Intelligent Systems and Applications in Engineering, 12(3), 2016–2020. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/5667

Issue

Section

Research Article