Hand Gesture Recognition Using Convolutional Neural Networks
Keywords:
HPR,CNN,Segmentation,Background Clutter,Virtual Reality,Neural NetworkAbstract
Hand gestures play a crucial role in communication and are essential in various scenarios where verbal communication is not possible. For instance, traffic policemen, newsreaders, airport staff, and gamers often rely on hand signals to communicate. Therefore, there is a growing need for robust hand pose recognition (HPR) methods that can identify hand gestures accurately. However, the current state-of-the-art HPR methods struggle with identifying hand gestures in the presence of cluttered backgrounds. To address this challenge, we propose a deep learning framework based on convolutional neural networks (CNNs) to identify hand postures regardless of hand size, location in the image, and background clutter. Our proposed CNN-based approach eliminates the need for feature extraction and learns to recognize hand poses without explicit foreground segmentation. This method effectively identifies hand poses, even in the presence of complex and varying backgrounds or poor lighting conditions. We have conducted several experiments, which demonstrate the superiority of our proposed method over state-of-the-art datasets. Our approach significantly improves the accuracy of hand pose recognition, making it more reliable and efficient for a wide range of applications. Therefore, our proposed method has significant potential for use in real-world scenarios, such as traffic management, sign language interpretation, and virtual reality gaming. Overall, our results suggest that deep neural networks can provide a robust and effective solution for hand gesture recognition tasks.
Downloads
References
Ke, W., Xing, Y., Di Caterina, G., Petropoulakis, L., & Soraghan, J. (2020). Deep Convolutional Spiking Neural Network Based Hand Gesture Recognition. 2020 International Joint Conference on Neural Networks (IJCNN).
Xu, J., & Jiang, T. (2017). Dynamic Hand Gesture Recognition Based on Parallel HMM Using Wireless Signals. Communications, Signal Processing, and Systems, 749–757.
Chen, Z., Kim, J.-T., Liang, J., Zhang, J., & Yuan, Y.-B. (2014). Real-Time Hand Gesture Recognition Using Finger Segmentation. The Scientific World Journal, 2014, 1–9.
Suk, H.-I., Sin, B.-K., & Lee, S.-W. (2010). Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recognition, 43(9), 3059–3072.
Zhang, C., & Tian, Y. (2013). Edge Enhanced Depth Motion Map for Dynamic Hand Gesture Recognition. 2013 IEEE Conference on Computer Vision and Pattern Recognition
Cheng, H., Luo, J., & Chen, X. (2014). A windowed dynamic time warping approach for 3D continuous hand gesture recognition. 2014 IEEE International Conference on Multimedia and Expo (ICME).
Li, Y.-T., & Wachs, J. P. (2014). HEGM: A hierarchical elastic graph matching for hand gesture recognition. Pattern Recognition, 47(1), 80–88.
Haria, A., Subramanian, A., Asokkumar, N., Poddar, S., & Nayak, J. S. (2017). Hand Gesture Recognition for Human Computer Interaction. Procedia Computer Science, 115, 367–374.
Mohanty, A., Rambhatla, S. S., & Sahay, R. R. (2016). Deep Gesture: Static Hand Gesture Recognition Using CNN. Proceedings of International Conference on Computer Vision and Image Processing, 449–461
Kopuklu, O., Kose, N., & Rigoll, G. (2018). Motion Fused Frames: Data Level Fusion Strategy for Hand Gesture Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
Abavisani, M., Joze, H. R. V., & Patel, V. M. (2019). Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition With Multimodal Training. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
Al-Hammadi, M., Muhammad, G., Abdul, W., Alsulaiman, M., Bencherif, M. A., Alrayes, T. S., Mathkour, H., & Mekhtiche, M. A. (2020). Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation. IEEE Access, 8, 192527–192542.
Li, C., Xie, C., Zhang, B., Chen, C., & Han, J. (2018). Deep Fisher discriminant learning for mobile hand gesture recognition. Pattern Recognition, 77, 276–288.
Mo, G. B., Dudley, J. J., & Kristensson, P. O. (2021). Gesture Knitter: A Hand Gesture Design Tool for Head-Mounted Mixed Reality Applications. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.
P. Parvathy, K. Subramaniam, G. K. D. Prasanna Venkatesan, P. Karthikaikumar, J. Varghese, and T. Jayasankar, “RETRACTED ARTICLE: Development of hand gesture recognition system using machine learning,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 6, pp. 6793–6800, Jul. 2020.
Hasan, H., & Abdul-Kareem, S. (2012). RETRACTED ARTICLE: Static hand gesture recognition using neural networks. Artificial Intelligence Review, 41(2), 147–181.
Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015). Hand gesture recognition with 3D convolutional neural networks. 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
Murthy, G. R. S., & Jadon, R. S. (2010). Hand gesture recognition using neural networks. 2010 IEEE 2nd International Advance Computing Conference (IACC). https://doi.org/10.1109/iadcc.2010.5423024
Neto, P., Pereira, D., Pires, J. N., & Moreira, A. P. (2013). Real-time and continuous hand gesture spotting: An approach based on artificial neural networks. 2013 IEEE International Conference on Robotics and Automation.
Alam, M. M., Islam, M. T., & Rahman, S. M. M. (2022). Unified learning approach for egocentric hand gesture recognition and fingertip detection. Pattern Recognition, 121, 108200.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.