American Sign Language Recognition Based on Transfer Learning Algorithms
Keywords:
Gesture Recognition, American Sign Language (ASL), Deep Learning, Transfer LearningAbstract
This research focuses on recognizing American Sign Language (ASL) letters and numbers, addressing the evolving technology landscape and the growing demand for improved user experiences among those primarily using sign language for communication. Leveraging deep learning, particularly through transfer learning, this study aims to enhance ASL recognition technology. Various deep learning models, including VGG16, ResNet50, MobileNetV2, InceptionV3, and CNN, are evaluated using an ASL dataset sourced from the Modified National Institute of Standards and Technology (MNIST) database, featuring ASL alphabetic letters represented through hand gestures. InceptionV3 emerges as the top-performing model, achieving an accuracy of 0.96. Transfer learning, which fine-tunes pre-trained models with ASL data, significantly improves recognition accuracy, making it especially valuable when labeled ASL data is limited. While InceptionV3 stands out, other models like VGG16, MobileNetV2, and ResNet50 demonstrate acceptable performance, offering flexibility for model selection based on specific application needs and computational resources. These findings underscore the effectiveness of deep learning and transfer learning techniques, providing a foundation for intuitive sign language recognition systems and contributing to breaking down communication barriers for the deaf and mute community.
Downloads
References
Pleva, Matus & Liao, Yuan-fu & Bours, Patrick. (2022). Human–Computer Interaction for Intelligent Systems. Electronics. 12. 161. 10.3390/electronics12010161.
Lv, Zhihan & Poiesi, Fabio & Dong, Qi & Lloret, Jaime & Song, Houbing. (2022). Deep Learning for Intelligent Human–Computer Interaction. Applied Sciences. 12. 11457. 10.3390/app122211457.
Hela, Daassi-Gnaba & Krahe, Jaime. (2023). Combination of speech recognition, emotion recognition and talking head for deaf and hard of hearing people.
"International Day of Sign Languages," United Nations. [Online]. Available: https://www.un.org/en/observances/sign-languages-day. [Accessed: 24-Jan-2023].
I.A. Adeyanju, O.O. Bello, M.A. Adegboye. Machine learning methods for sign language recognition: a critical review and analysis. Intell. Syst. Appl., 12 (2021), Article 200056, 10.1016/j.iswa.2021.200056
Hekmat, Atyaf & Abbas, Hawraa & Shahadi, Ismael. (2022). Sign Language Recognition and Hand Gestures Review, Kerbala Journal for Engineering Science, 2(4), 192-316.
Manoharan, Madhiarasan & Roy, Prof. (2022). A Comprehensive Review of Sign Language Recognition: Different Types, Modalities, and Datasets, arXiv preprint arXiv:2204.03328.
CABRERA, MARIA & BOGADO, JUAN & Fermín, Leonardo & Acuña, Raul & RALEV, DIMITAR. (2012). GLOVE-BASED GESTURE RECOGNITION SYSTEM. 10.1142/9789814415958_0095.
Lv, Zhihan & Poiesi, Fabio & Dong, Qi & Lloret, Jaime & Song, Houbing. (2022). Deep Learning for Intelligent Human–Computer Interaction. Applied Sciences. 12. 11457. 10.3390/app122211457.
Xu, Yong-Jun & Wang, Qi & An, Zhulin & Wang, Fei & Zhang, Libo & Wu, Yanjun & Dong, Fengliang & Qiu, Cheng-Wei & Liu, Xin & Qiu, Junjun & Hua, Keqin & Su, Wentao & Xu, Huiyu & Han, Yong & Cao, Xin & Liu, Enke & Fu, Chenguang & Yin, Zhigang & Liu, Miao & Zhang, Jiabao. (2021). Artificial Intelligence: A Powerful Paradigm for Scientific Research. The Innovation. 2. 100179. 10.1016/j.xinn.2021.100179.
Wang, Ya-Hong, and Wen-Hao Su. 2022. "Convolutional Neural Networks in Computer Vision for Grain Crop Phenotyping: A Review" Agronomy 12, no. 11: 2659. https://doi.org/10.3390/agronomy12112659.
Pu, Junfu & Zhou, Wengang & Li, Houqiang. (2018). Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition. 885-891. 10.24963/ijcai.2018/123.
Aly, Walaa & Aly, Saleh & Almotairi, Sultan. (2019). User-Independent American Sign Language Alphabet Recognition Based on Depth Image and PCANet Features. IEEE Access. PP. 1-1. 10.1109/ACCESS.2019.2938829.
Oyedotun,Oyebade & Khashman, Adnan. (2017). Deep learning in vision-based static hand gesture recognition. Neural Computing and Applications. 28. 10.1007/s00521-016-2294-8.
Tolentino, Lean Karlo & Serfa Juan, Ronnie & Thio-ac, August & Pamahoy, Maria & Forteza, Joni & Garcia, Xavier. (2019). Static Sign Language Recognition Using Deep Learning. International Journal of Machine Learning and Computing. 9. 821-827. 10.18178/ijmlc.2019.9.6.879. International Journal of Intelligent Systems and Applications in EngineeringIJISAE, 2023, 11(6s), 232–245|243
Bendarkar, D., Somase, P., Rebari, P., Paturkar, R. & Khan, A. (2021).Web-Based Recognition and Translation of American Sign Language with CNN and RNN. International Association of Online Engineering.Retrieved November 9, 2022, fromhttps://www.learntechlib.org/p/218958/
Amrutha D , Bhumika M , Shivani Hosangadi , Shravya, Manoj H M, (2022), Real Time Static and Dynamic Hand Gesture Recognition using CNN, INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT) Volume 11, Issue 05 (May 2022),
Liu, Lei & Wan, Shaohua & Hui, Xiaozhe & Pei, Qingqi. (2022). Data Dissemination for Industry 4.0 Applications in Internet of Vehicles Based on Short-term Traffic Prediction. ACM Transactions on Internet Technology. 22. 1-18. 10.1145/3430505.
Mehreen Hurroo , Mohammad Elham, 2020, Sign Language Recognition System using Convolutional Neural Network and Computer Vision, INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT) Volume 09, Issue 12 (December 2020)
Taye, Mohammad. (2023). Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions. Computers. 12. 91. 10.3390/computers12050091.
Sarker, Iqbal. (2021). Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions. SN Computer Science. 2. 10.1007/s42979-021-00815-1.
Iman, Mohammadreza & Arabnia, Hamid & Rasheed, Khaled. (2023). A Review of Deep Transfer Learning and Recent Advancements. Technologies. 11. 40. 10.3390/technologies11020040.
Zhu, Zhuangdi & Lin, Kaixiang & Jain, Anil & Zhou, Jiayu. (2023). Transfer Learning in Deep Reinforcement Learning: A Survey. IEEE transactions on pattern analysis and machine intelligence. PP. 10.1109/TPAMI.2023.3292075.
Taye, Mohammad. (2023). Theoretical Understanding of Convolutional Neural Network: Concepts, Architectures, Applications, Future Directions. Computation. 11. 52. 10.3390/computation11030052.
Alzubaidi, Laith & Zhang, Jinglan & Humaidi, Amjad & Al-Dujaili, Ayad & Duan, Ye & Al-Shamma, Omran & Santamaría, J. & Fadhel, Mohammed & Al-Amidie, Muthana & Farhan, Laith. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data. 8. 10.1186/s40537-021-00444-8.
Tammina, Srikanth. (2019). Transfer learning using VGG-16 with Deep Convolutional Neural Network for Classifying Images. International Journal of Scientific and Research Publications (IJSRP). 9. p9420. 10.29322/IJSRP.9.10.2019.p9420.
Shafiq, Muhammad & Gu, Zhaoquan. (2022). Deep Residual Learning for Image Recognition: A Survey. Applied Sciences. 10.3390/app12188972.
Hong, Kiseong & Kim, Gyeong-hyeon & Kim, Eunwoo. (2023). GhostNeXt: Rethinking Module Configurations for Efficient Model Design. Applied Sciences. 13. 3301. 10.3390/app13053301.
Sandler, Mark & Howard, Andrew & Zhu, Menglong & Zhmoginov, Andrey & Chen, Liang-Chieh. (2018). MobileNetV2: Inverted Residuals and Linear Bottlenecks. 4510-4520. 10.1109/CVPR.2018.00474.
Shaheed, Kashif & Abbas, Qaisar & Hussain, Ayyaz & Qureshi, Imran. (2023). Optimized Xception Learning Model and XgBoost Classifier for Detection of Multiclass Chest Disease from X-ray Images. Diagnostics. 13. 2583. 10.3390/diagnostics13152583.
Barcic, Ena & Grd, Petra & Tomicic, Igor. (2023). Convolutional Neural Networks for Face Recognition: A Systematic Literature Review. 10.21203/rs.3.rs-3145839/v1.
Bragg, Danielle & Caselli, Naomi & Hochgesang, Julie & Huenerfauth, Matt & Katz-Hernandez, Leah & Koller, Oscar & Kushalnagar, Raja & Vogler, Christian & Ladner, R.E.. (2021). The FATE Landscape of Sign Language AI Datasets: An Interdisciplinary Perspective. ACM Transactions on Accessible Computing. 14. 1-45. 10.1145/3436996.
Baraa Wasfi Salim , Subhi R. M. Zeebaree(2023). Kurdish Sign Language Recognition Based on Transfer Learning. International Journal of Intelligent Systems and Applications in Engineering, international Journal of Intelligent Systems and Applications in Engineering, 11(6s), 232–245. Retrieved from
Subhi R. M. Zeebaree, & Hanan M. Shukur, & Lailan M. Haji, & Rizgar R. Zebari,& Karwan Jacksi, & Shakir M. Abas (2020), Characteristics and Analysis of Hadoop Distributed Systems, Technology Reports of Kansai University Journal, ISSN: 04532198 Volume 62, Issue 04, April, 2020.
Rajiv, A., Saxena, A.K., Singh, D., Awasthi, A., Dhabliya, D., Yadav, R.K., Gupta, A. IoT and machine learning on smart home-based data and a perspective on fog computing implementation (2023) Handbook of Research on Machine Learning-Enabled IoT for Smart Applications Across Industries, pp. 336-349.
Anand, R., Ahamad, S., Veeraiah, V., Janardan, S.K., Dhabliya, D., Sindhwani, N., Gupta, A. Optimizing 6G wireless network security for effective communication (2023) Innovative Smart Materials Used in Wireless Communication Technology, pp. 1-20.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
