Face and Hand Gesture Recognition Based Person Identification System using Convolutional Neural Network

Mysha Sarin Kabisha; Kazi Anisa Rahim; Md. Khaliluzzaman; Shahidul Islam Khan

doi:10.18201/ijisae.2022.273

Authors

Mysha Sarin Kabisha International Islamic University Chittagong (IIUC) http://orcid.org/0000-0003-1551-9908
Kazi Anisa Rahim International Islamic University Chittagong (IIUC) http://orcid.org/0000-0001-9224-566X
Md. Khaliluzzaman International Islamic University Chittagong (IIUC) https://orcid.org/0000-0001-6846-1610
Shahidul Islam Khan Dept. Of CSE, International Islamic University https://orcid.org/0000-0002-8740-2744

DOI:

https://doi.org/10.18201/ijisae.2022.273

Keywords:

Deep Learning, Convolutional Neural Network, Face Recognition, Hand Gesture Recognition, VGG-16

Abstract

Person identification system is now become the most hyped system for security purpose. It’s also gaining a lot of attention in the field of computer vision. For verification of human, facial recognition and hand gesture recognition are the most common topics of research. In the current days, various researchers focused on facial and hand gesture recognition using various shallow techniques and Deep Convolutional Neural Network (DCNN). However, using one feature of human for person identification is the most researched topic till now. In this paper, we proposed a Convolutional Neural Network (CNN) based system which will identify a person using two traits i.e., face and hand gesture of number sign of that person. For feature extraction and recognition Neural Network have shown immense good result. This proposed system works on two models, one is a VGG-16 architecture model for face recognition and another model is for hand gesture which is based on simple CNN with two convolutional layers. With two customized dataset our face model gained 98.00% accuracy and hand gesture (number sign) model gained an accuracy of 98.33%.

Downloads

Download data is not yet available.

References

B. Kranthikiran, and P. Pulicherla, “Face Detection and Recognition for use in Campus Surveillance,” International Journal of Innovative Technology and Exploring Engineering, vol. 9, no. 3, pp. 2908–2913, Jan. 2020, doi: 10.35940/ijitee.b7682.019320.

M. Coşkun, A. Uçar, Ö. Yildirim, and Y. Demir, “A FACE RECOGNITION ALGORITHM BASED ON CONVOLUTIONAL NEURAL NETWORK,” REVISTA ARGENTINA DE CLINICA PSICOLOGICA, 2020, doi: 10.24205/03276716.2020.339.

Y. Said, M. Barr, and H. E. Ahmed, “Design of a Face Recognition System based on Convolutional Neural Network (CNN),” Engineering, Technology & Applied Science Research, vol. 10, no. 3, pp. 5608–5612, Jun. 2020, doi: 10.48084/etasr.3490.

K. B. Pranav, and J. Manikandan, “Design and Evaluation of a Real-Time Face Recognition System using Convolutional Neural Networks,” Procedia Computer Science, vol. 171, pp. 1651–1659, Jan. 2020, doi: 10.1016/j.procs.2020.04.177.

G. Hu, Y. Yang, D. Yi, J. Kittler, W. Christmas, S. Z. Li, & T. Hospedales, “When face recognition meets with deep learning: an evaluation of convolutional neural networks for face recognition,” In Proceedings of the IEEE international conference on computer vision workshops, pp. 142-150, IEEE, 2015.

Z. Pei, H. Xu, Y. Zhang, M. Guo, and Y.-H. Yang, “Face Recognition via Deep Learning Using Data Augmentation Based on Orthogonal Experiments,” Electronics, vol. 8, no. 10, p. 1088, Sep. 2019, doi: 10.3390/electronics8101088.

P. Gupta, N. Saxena, M. Sharma, and J. Tripathi, “Deep learning model for group face recognition based on Convolution Neural Network,” Journal of Xidian University, vol. 14, no. 5, May 2020, doi: 10.37896/jxu14.5/415.

Z. Chen, J.-T. Kim, J. Liang, J. Zhang, and Y.-B. Yuan, “Real-Time Hand Gesture Recognition Using Finger Segmentation,” The Scientific World Journal, vol. 2014, p. e267872, Jun. 2014, doi: 10.1155/2014/267872.

R. F. Pinto, C. D. B. Borges, A. M. A. Almeida, and I. C. Paula, “Static Hand Gesture Recognition Based on Convolutional Neural Networks,” Journal of Electrical and Computer Engineering, vol. 2019, pp. 1–12, Oct. 2019, doi: 10.1155/2019/4167890.

J. Chen, S. Bi, G. Zhang, and G. Cao, “High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network,” Sensors, vol. 20, no. 4, p. 1201, Feb. 2020, doi: 10.3390/s20041201.

P. Molchanov, S. Gupta, K. Kim, & J. Kautz, “Hand gesture recognition with 3D convolutional neural network,” In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 1-7, IEEE, 2015.

L. Pigou, S, Dieleman, P. J. Kindermans, and B. Schrauwen, “Sign language recognition using convolutional neural networks,” In European Conference on Computer Vision, pp. 572-578, Springer, Chamm, 2014, September.

M.R. Islam, U.K. Mitu, R.A. Bhuiyan, and J. Shin, “Hand gesture feature extraction using deep convolutional neural network for recognizing American Sign Language,” In 2018 4th International Conference on Frontiers of Signal Processing (ICFSP), pp. 115-119, IEEE, 2018, September.

M. M. Islam, M. R. Islam, and M. S. Islam, “An Efficient Human Computer Interaction through Hand Gesture Using Deep Convolutional Neural Network,” SN Computer Science, vol. 1, no. 4, Jun. 2020, doi: 10.1007/s42979-020-00223-x.

Simonyan, K. and Zisserman, A., 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

D. P. Kingma, & J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

J. Duchi, E. Hazan, & Y. Singer, “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,” Journal of Machine Learning Research, Vol.12, 2121–2159, 2011.

S. Ruder, “An overview of gradient descent optimization algorithms,” ArXiv, abs/1609.04747, 2016.

https://www.kaggle.com/muhammadkhalid/sign-language-for-numbers (21 February, 2021).

I. Cinar, and M. Koklu, “Classification of rice varieties using artificial intelligence methods,” International Journal of Intelligent Systems and Applications in Engineering, 7(3), pp.188-194, 2019.

A. Krizhevsky, I. Sutskever, & G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, 25, 2012.

Y. S. Taspinar, I. Cinar, & M. Koklu, “Prediction of Computer Type Using Benchmark Scores of Hardware Units,” Selcuk University Journal of Engineering Sciences, 20(1), 11-17, 2021.