Kurdish Sign Language Recognition Based on Transfer Learning
Keywords:
Kurdish Sign Language, CNN, Pre-trained, Transfer Learning, VGG19 and RESNET101Abstract
Sign language is used to communicate with deaf and dumb people; it is difficult for ordinary people to communicate with them. Hence, computer vision and automatic identification can reduce the difficulties of reaching them. Deep learning algorithms were used to distinguish sign language in different languages and styles. Convolutional Neural Networks (CNNs) are used in computer vision, particularly pre-trained algorithms. This research proposes using transfer and machine learning to distinguish Kurdish Sign Language (KSL). A KSL dataset was created to characterize the Kurdish language at the level of numbers and letters, using pre-trained algorithms for feature extraction and machine learning algorithms for classification. The proposed method was tested on two data sets; KSL and American Sign Language (ASL). The algorithms (VGG19 and RESNET101) are implemented in the feature extraction phase with pre-trained weights. The algorithms: Support Vector Machine (SVM), Decision Tree (DT), and Random Forest (RF), is dependent on the classification stage, and the CNN is designed for the KSL model. The efficiency of the proposed models is evaluated using (accuracy, recall, precision, and F1 score) metrics. The proposed model's outcomes illustrated that VGG19 is better than (RESNET101 and proposed CNN) algorithms in terms of feature extraction, and the random forest is the best classifier which achieved an accuracy rate of 95% at the numbers level and 97% at the level of the letter for KSL and ASL.
Downloads
References
Mertzani, Maria. (2022). SIGN LANGUAGE LITERACY IN THE SIGN LANGUAGE CURRICULUM. Momento - Diálogos em Educação. 31. 449-474. 10.14295/momento.v31i02.14504.
Patil, Prof & Bhagwat, Ruchir & Padale, Pratham & Shah, Yash & Surwade, Hrutik. (2022). Sign Language Recognition System. International Journal for Research in Applied Science and Engineering Technology. 10. 1772-1776. 10.22214/ijraset.2022.42626.
Sultan, A. (2022, March 8). International Day of Sign Languages. National Today. Retrieved November 15, 2022, from https://nationaltoday.com/international-day-of-sign-languages/
Alys Young, Jemina Napier, Rosemary Oram. (2019) The translated deaf self, ontological (in)security and deaf culture. The Translator 25:4, pages 349-368.
Lin, Hao. (2021). Early Development of Chinese Sign Language in Shanghai Schools for the Deaf. Frontiers in Psychology. 12. 702620. 10.3389/fpsyg.2021.702620.
Aggarwal, Karan & Mijwil, Maad & Garg, Sonia & Al-Mistarehi, Abdel-Hameed & Alomari, Safwan & Gök, Murat & Zein Alaabdin, Anas & Abdul Rahman, Safaa. (2022). Has the Future Started? The Current Growth of Artificial Intelligence, Machine Learning, and Deep Learning. Iraqi Journal for Computer Science and Mathematics. 3. 115-123. 10.52866/ijcsm.2022.01.01.013.
Li, Jiangong & Green-Miller, Angela & Hu, Xiaodan & Lucic, Ana & Mohan, M.R. & Dilger, Ryan & Condotta, Isabella & Aldridge, Brian & Hart, John & Ahuja, Narendra. (2022). Barriers to computer vision applications in pig production facilities. Computers and Electronics in Agriculture. 200. 107227. 10.1016/j.compag.2022.107227.
Gao, Yi. (2022). Research on the Application of Artificial Intelligence Technology in the Development of Computer Vision. Highlights in Science, Engineering and Technology. 9. 80-84. 10.54097/hset.v9i.1720.
Cui, Runpeng & Liu, Hu & Zhang, Changshui. (2019). A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training. IEEE Transactions on Multimedia. PP. 1-1. 10.1109/TMM.2018.2889563.
Stoll, S., Camgoz, N.C., Hadfield, S. et al. Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks. Int J Comput Vis 128, 891–908 (2020). https://doi.org/10.1007/s11263-019-01281-2
BANKUR, Fatih & KAYA, Mustafa. (2022). Deep Learning Based Recognition of Turkish Sign Language Letters with Unique Data Set. Turkish Journal of Science and Technology. 10.55525/tjst.1073116.
Bandyopadhyay, Shaon. (2020). A Study on Indian Sign Language Recognition using Deep Learning Approach. 10.13140/RG.2.2.25282.20168.
Reshna, S. & Sajeena, A. & Jayaraju, Madhavan. (2020). Recognition of static hand gestures of Indian sign language using CNN. AIP Conference Proceedings. 2222. 030012. 10.1063/5.0004485.
Alaria, Satish & Raj, Ashish & Sharma, Vivek & Kumar, Vijay. (2022). Simulation and Analysis of Hand Gesture Recognition for Indian Sign Language using CNN. International Journal on Recent and Innovation Trends in Computing and Communication. 10. 10-14. 10.17762/ijritcc.v10i4.5556.
Kumar, Hemant & Sharma, Mohit & Rohit, & Bisht, Kunal & Kumar, Ashish & Jain, Rachna & Nagrath, Preeti & Singh, Pritpal. (2022). Sign language detection and conversion to text using CNN and OpenCV. AIP Conference Proceedings. 2555. 040016. 10.1063/5.0108711.
Alleema, N. & Selvamani, Babeetha & Kumar, P. & chandrasekaran, Saravanan & Pandiaraj, S & Kumar, A. & Rajkumar, K.. (2022). Recognition of American Sign Language Using Modified Deep Residual CNN with Modified Canny Edge Segmentation. 10.21203/rs.3.rs-1521209/v1.
Jiang, Xianwei & Satapathy, Suresh & Yang, Longxiang & Wang, Shuihua & Zhang, Yu-Dong. (2020). A Survey on Artificial Intelligence in Chinese Sign Language Recognition. Arabian Journal for Science and Engineering. 45. 10.1007/s13369-020-04758-2.
Satiman, Supathep & Meesad, Phayung. (2022). Deep transfer learning base on sequenced edge grid image technique for sign language recognition. International journal of health sciences. 9982-9997. 10.53730/ijhs.v6nS5.12112.
Gaye, Sakshi. (2022). Sign Language Recognition and Interpretation System. INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT. 06. 10.55041/IJSREM13428.
Pu, Junfu & Zhou, Wengang & Li, Houqiang. (2018). Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition. 885-891. 10.24963/ijcai.2018/123.
Aly, Walaa & Aly, Saleh & Almotairi, Sultan. (2019). User-Independent American Sign Language Alphabet Recognition Based on Depth Image and PCANet Features. IEEE Access. PP. 1-1. 10.1109/ACCESS.2019.2938829.
Oyedotun, Oyebade & Khashman, Adnan. (2017). Deep learning in vision-based static hand gesture recognition. Neural Computing and Applications. 28. 10.1007/s00521-016-2294-8.
Tolentino, Lean Karlo & Serfa Juan, Ronnie & Thio-ac, August & Pamahoy, Maria & Forteza, Joni & Garcia, Xavier. (2019). Static Sign Language Recognition Using Deep Learning. International Journal of Machine Learning and Computing. 9. 821-827. 10.18178/ijmlc.2019.9.6.879.
Bendarkar, D., Somase, P., Rebari, P., Paturkar, R. & Khan, A. (2021). Web-Based Recognition and Translation of American Sign Language with CNN and RNN. International Association of Online Engineering. Retrieved November 9, 2022, from https://www.learntechlib.org/p/218958/.
Amrutha D , Bhumika M , Shivani Hosangadi , Shravya, Manoj H M, (2022), Real Time Static and Dynamic Hand Gesture Recognition using CNN, INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT) Volume 11, Issue 05 (May 2022),
Liu, Lei & Wan, Shaohua & Hui, Xiaozhe & Pei, Qingqi. (2022). Data Dissemination for Industry 4.0 Applications in Internet of Vehicles Based on Short-term Traffic Prediction. ACM Transactions on Internet Technology. 22. 1-18. 10.1145/3430505.
Mehreen Hurroo , Mohammad Elham, 2020, Sign Language Recognition System using Convolutional Neural Network and Computer Vision, INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH & TECHNOLOGY (IJERT) Volume 09, Issue 12 (December 2020).
"American sign language," National Institute of Deafness and Other Communication Disorders. [Online]. Available: https://www.nidcd.nih.gov/health/american-sign-language. [Accessed: 15-Nov-2022].
Lewis, Buckley & Secco, Emanuele. (2021). A CNN sign language recognition system with single & double-handed gestures. 10.1109/COMPSAC51774.2021.00173.
Ergen, Tolga & Sahiner, Arda & Ozturkler, Batu & Pauly, John & Mardani, Morteza & Pilanci, Mert. (2021). Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization.
Hassan, Nazmul. (2022). Bangla Sign Language Gesture Recognition System: Using CNN Model. 10.14293/S2199-1006.1.SOR-.PPUF56Q.v1.
Wattamwar, Aniket. (2021). Sign Language Recognition using CNN. International Journal for Research in Applied Science and Engineering Technology. 9. 826-830. 10.22214/ijraset.2021.38058.
Chen, I-Te & Shiang, Rung & Huang, Hung-Yuan. (2020). Sign Language Recognition System Using Cnn. Journal of Medicine and HealthCare. 1-6. 10.47363/JMHC/2020(2)138.
Alzubaidi, Laith & Zhang, Jinglan & Humaidi, Amjad & Al-Dujaili, Ayad & Duan, Ye & Al-Shamma, Omran & Santamaría, J. & Fadhel, Mohammed & Al-Amidie, Muthana & Farhan, Laith. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data. 8. 10.1186/s40537-021-00444-8.
Srivastava, Smriti & Khari, Manju & Gonzalez Crespo, Ruben & Chaudhary, Dr-Gopal & Narula, Parul. (2021). Concepts and Real-Time Applications of Deep Learning. 10.1007/978-3-030-76167-7.
Zhang, Qing & Huang, Huajie & Li, Jizuo & Zhang, Yuhang & Li, Yongfu. (2022). CmpCNN: CMP Modeling with Transfer Learning CNN Architecture. ACM Transactions on Design Automation of Electronic Systems. 10.1145/3569941.
Madanan, Mukesh & Sayed, B.. (2022). Designing a Deep Learning Hybrid Using CNN and Inception V3 Transfer Learning to Detect the Aggression Level of Deep Obsessive Compulsive Disorder in Children. International Journal of Biology and Biomedical Engineering. 16. 207-220. 10.46300/91011.2022.16.27.
Islam, Md & Uddin, Md & AKhtar, Md & Alam, K.M. Rafiqul. (2022). Recognizing multiclass Static Sign Language for deaf and dumb people of Bangladesh based on transfer learning techniques and the primary sign word dataset. Informatics in Medicine Unlocked. 33. 101077. 10.1016/j.imu.2022.101077.
Niswati, Za'Imatun & Hardatin, Rahayuning & Muslimah, Meia & Hasanah, Siti. (2021). Perbandingan Arsitektur ResNet50 dan ResNet101 dalam Klasifikasi Kanker Serviks pada Citra Pap Smear. Faktor Exacta. 14. 160. 10.30998/faktorexacta.v14i3.10010.
Zhang, Qi. (2022). A novel ResNet101 model based on dense dilated convolution for image classification. SN Applied Sciences. 4. 10.1007/s42452-021-04897-7.
Ji, Li & Mao, Rongzhi & Wu, Jian & Ge, Cheng & Xiao, Feng & Xu, Xiaojun & Xie, Liangxu & Gu, Xiaofeng. (2022). Deep Convolutional Neural Network for Nasopharyngeal Carcinoma Discrimination on MRI by Comparison of Hierarchical and Simple Layered Convolutional Neural Networks. Diagnostics. 12. 2478. 10.3390/diagnostics12102478.
Andrew, Andrew & Santoso, Handri. (2022). Compare VGG19, ResNet50, Inception-V3 for Review Food Rating. SinkrOn. 7. 845-494. 10.33395/sinkron.v7i2.11383.
Bouhsissin, Soukaina & Sael, Nawal & Benabbou, Faouzia. (2021). Enhanced VGG19 Model for Accident Detection and Classification from Video. 39-46. 10.1109/ICDATA52997.2021.00017.
Han, Baoru & Du, Jinglong & Jia, Yuanyuan & Zhu, Huazheng. (2021). Zero-Watermarking Algorithm for Medical Image Based on VGG19 Deep Convolution Neural Network. Journal of Healthcare Engineering. 2021. 1-12. 10.1155/2021/5551520.
Jo, Taeho. (2021). Decision Tree. 10.1007/978-3-030-65900-4_7.
Chistopolskaya, Anastasiya & Podolskii, Vladimir. (2022). On the Decision Tree Complexity of Threshold Functions. Theory of Computing Systems. 66. 10.1007/s00224-022-10084-x.
McTavish, Hayden & Zhong, Chudi & Achermann, Reto & Karimalis, Ilias & Chen, Jacques & Rudin, Cynthia & Seltzer, Margo. (2022). Fast Sparse Decision Tree Optimization via Reference Ensembles. Proceedings of the AAAI Conference on Artificial Intelligence. 36. 9604-9613. 10.1609/aaai.v36i9.21194.
Das, Sunanda & Imtiaz, Samir & Neom, Nieb & Siddique, Nazmul & Wang, Hui. (2022). A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier. Expert Systems with Applications. 213. 118914. 10.1016/j.eswa.2022.118914.
Thiruselvan, Dr. (2022). INDIAN SIGN LANGUAGE DETECTION USING ARTIFICIAL INTELLIGENCE. International Journal of Engineering and Artificial Intelligence. 3. 27-30. 10.55923/JO.IJEAL.3.3.104.
Ramani, B. & Lakshmi, T. & Durga, N. & Sana, Shaik & Sravya, T. & Jishitha, N. (2022). Recognition of Hand Gesture-Based Sign Language Using Transfer Learning. 10.1007/978-981-19-1976-3_12.
Tamiru, Nigus & Tekeba, Menore & Salau, Ayodeji. (2022). Recognition of Amharic sign language with Amharic alphabet signs using ANN and SVM. The Visual Computer. 38. 1-16. 10.1007/s00371-021-02099-1.
Zahid, Hira & Rashid, Sheikh Muhammad & Hussain, Samreen & Azim, Fahad & Syed, Engr. Dr.Sidra & Saad, Afshan. (2022). Recognition of Urdu sign language: a systematic review of the machine learning classification. PeerJ Computer Science. 8. e883. 10.7717/peerj-cs.883.
Myagila, Kasian & Kilavo, Hassan. (2021). A Comparative Study on Performance of SVM and CNN in Tanzania Sign Language Translation Using Image Recognition. Applied Artificial Intelligence. 36. 1-16. 10.1080/08839514.2021.2005297.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
