Unveiling Gesture Language: Advancements in Deep Backpropagation Neural Networks for Image-Based Sign Language Recognition
Keywords:
Sign Language Identification (SLI), pre- processing, segmentation, histogram, impaired people, and deep learning.Abstract
A defined set of languages called sign language (SL) uses both manual and visual methods to communicate information. It is frequently used to communicate with non-verbal individuals. The majority of people can only comprehend if they acquire knowledge of universal sign language. Thus, not everyone is unable to understand when we use sign language to communicate with hearing-impaired individuals. It is difficult to communicate with people without a translator. The prior approach to this problem concentrated on reading sign language. However, because it takes longer and provides incorrect precision performance, it is difficult to detect sign language in it. To solve this issue, this book offers an improved solution based on deep learning methods. For Sign Language Identification (SLI), we introduce a Deep Backpropagation Neural Network (DBNNet) method using a softmax activation function. First, collect the ASL dataset of SL images. Furthermore, we use the Gaussian Smoothing Histogram Filter (GMHF) method to improve contrast and image quality. Additionally, based on an enhanced sign image, the Intensity Gradients Sign Edge Detection (IGSED) method locates the edges. Next, we use a Cluster-Based Watershed Segmentation (CBWS) algorithm to examine the region of interest (ROI) for a sign. Subsequently, the DBNNet method with the softmax function is suggested to efficiently identify the sign. As a result, the suggested algorithm outperformed earlier techniques in terms of sign identification accuracy, sensitivity, specificity, and F-measure performance.
Downloads
References
Saiful, Md&Isam, Abdulla & Moon, Hamim&Tammana, Rifa& Das, Mitul &Alam, Md& Rahman, Ashifur. (2022). Real-Time Sign Language Detection Using CNN. 10.1109/ICDABI56818.2022.10041711.
Dabre, Kanchan; Dholay, Surekha (2014). The machine learning model for sign language interpretation using webcam images. [IEEE 2014 International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA) - Mumbai, Maharashtra, India (2014.4.4-2014.4.5)] 317–321.
Long Huynh;Viet Ngo; (2020). Recognize Vietnamese Sign Language Using Deep Neural Networks. 2020 7th NAFOSTED Conference on Information and Computer Science (NICS), doi:10.1109/nics51282.2020.9335904
S. B. Abdullahi and K. Chamnongthai, "American Sign Language Words Recognition Using Spatio-Temporal Prosodic and Angle Features: A Sequential Learning Approach," in IEEE Access, vol. 10, pp. 15911-15923, 2022, doi: 10.1109/ACCESS.2022.3148132.
S. Z. Gurbuz et al., "American Sign Language Recognition Using RF Sensing," in IEEE Sensors Journal, vol. 21, no. 3, pp. 3763-3775, 1 Feb.1, 2021, doi: 10.1109/JSEN.2020.3022376.
M. Lee and J. Bae, "Real-Time Gesture Recognition in the View of Repeating Characteristics of Sign Languages," in IEEE Transactions on Industrial Informatics, vol. 18, no. 12, pp. 8818-8828, Dec. 2022, doi: 10.1109/TII.2022.3152214.
B. Natarajan et al., "Development of an End-to-End Deep Learning Framework for Sign Language Recognition, Translation, and Video Generation," in IEEE Access, vol. 10, pp. 104358-104374, 2022, doi: 10.1109/ACCESS.2022.3210543.
J. Li, J. Meng, H. Gong, and Z. Fan, "Research on Continuous Dynamic Gesture, Recognition of Chinese Sign Language, Based on Multi-Model Fusion," in IEEE Access, vol. 10, pp. 106946-106957, 2022, doi: 10.1109/ACCESS.2022.3212064.
J. DelPreto, J. Hughes, M. D'Aria, M. de Fazio and D. Rus, "A Wearable Smart Glove and Its Application of Pose and Gesture Detection to Sign Language Classification," in IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10589-10596, Oct. 2022, doi: 10.1109/LRA.2022.3191232.
Y. Dong, J. Liu and W. Yan, "Dynamic Hand Gesture Recognition Based on Signals From Specialized Data Glove and Deep Learning Algorithms," in IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-14, 2021, Art no. 2509014, doi: 10.1109/TIM.2021.3077967.
K. Nimisha and A. Jacob, "A Brief Review of the Recent Trends in Sign Language Recognition," 2020 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, 2020, pp. 186-190, doi: 10.1109/ICCSP48568.2020.9182351.
S. S. Kakkoth and S. Gharge, "Survey on Real Time Hand Gesture Recognition," 2017 International Conference on Current Trends in Computer, Electrical, Electronics and Communication (CTCEEC), Mysore, India, 2017, pp. 948-954, doi: 10.1109/CTCEEC.2017.8455041.
X. Zhang, Z. Yang, T. Chen, D. Chen and M. -C. Huang, "Cooperative Sensing and Wearable Computing for Sequential Hand Gesture Recognition," in IEEE Sensors Journal, vol. 19, no. 14, pp. 5775-5783, 15 July15, 2019, doi: 10.1109/JSEN.2019.2904595.
B. G. Lee and S. M. Lee, "Smart Wearable Hand Device for Sign Language Interpretation System With Sensors Fusion," in IEEE Sensors Journal, vol. 18, no. 3, pp. 1224-1232, 1 Feb.1, 2018, doi: 10.1109/JSEN.2017.2779466.
T. Sagisaka, Y. Ohmura, Y. Kuniyoshi, A. Nagakubo and K. Ozaki, "High-density conformable tactile sensing glove," 2011 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia, 2011, pp. 537-542, doi: 10.1109/Humanoids.2011.6100898.
R. Galicia, O. Carranza, E. D. Jiménez and G. E. Rivera, "Mexican sign language recognition using movement sensor," 2015 IEEE 24th International Symposium on Industrial Electronics (ISIE), Buzios, Brazil, 2015, pp. 573-578, doi: 10.1109/ISIE.2015.7281531.
M. Al-Hammadi et al., "Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation," in IEEE Access, vol. 8, pp. 192527-192542, 2020, doi: 10.1109/ACCESS.2020.3032140.
Z. Chen, J. Yang and H. Xie, "Surface-Electromyography-Based Gesture Recognition Using a Multi-stream Fusion Strategy," in IEEE Access, vol. 9, pp. 50583-50592, 2021, doi: 10.1109/ACCESS.2021.3059499.
N. Mohamed, M. B. Mustafa and N. Jomhari, "A Review of the Hand Gesture Recognition System: Current Progress and Future Directions," in IEEE Access, vol. 9, pp. 157422-157436, 2021, doi: 10.1109/ACCESS.2021.3129650.
M. K. Burns, D. Pei and R. Vinjamuri, "Dynamic Control of Virtual Hand Grasp Using Spatiotemporal Synergies," in IEEE Access, vol. 7, pp. 112327-112338, 2019, doi: 10.1109/ACCESS.2019.2932956.
M. Diaz, M. A. Ferrer and J. J. Quintana, "Anthropomorphic Features for On-Line Signatures," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 12, pp. 2807-2819, 1 Dec. 2019, doi: 10.1109/TPAMI.2018.2869163.
W. Pan, X. Zhang and Z. Ye, "Attention-Based Sign Language Recognition Network Utilizing Keyframe Sampling and Skeletal Features," in IEEE Access, vol. 8, pp. 215592-215602, 2020, doi: 10.1109/ACCESS.2020.3041115.
M. A. Ferrer, M. Diaz, C. Carmona-Duarte and A. Morales, "A Behavioral Handwriting Model for Static and Dynamic Signature Synthesis," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1041-1053, 1 June 2017, doi: 10.1109/TPAMI.2016.2582167.
Fischer, R. Plamondon, C. O'Reilly and Y. Savaria, "Neuromuscular Representation and Synthetic Generation of Handwritten Whiteboard Notes," 2014 14th International Conference on Frontiers in Handwriting Recognition, Hersonissos, Greece, 2014, pp. 222-227, doi: 10.1109/ICFHR.2014.45.
YanglaiHuang , Jing Huang, “Dynamic Sign Language Recognition Based on CBAM with Autoencoder Time Series Neural Network”, Hindawi Mobile Information Systems, Volume 2022, 10 pages.
S. Huang and Z. Ye, "Boundary-Adaptive Encoder With Attention Method for Chinese Sign Language Recognition," in IEEE Access, vol. 9, pp. 70948-70960, 2021, doi: 10.1109/ACCESS.2021.3078638.
N. Sae-Bae, N. Memon, K. Isbister and K. Ahmed, "Multitouch Gesture-Based Authentication," in IEEE Transactions on Information Forensics and Security, vol. 9, no. 4, pp. 568-582, April 2014, doi: 10.1109/TIFS.2014.2302582
D. R. Kothadiya, C. M. Bhatt, T. Saba, A. Rehman and S. A. Bahaj, "SIGNFORMER: DeepVision Transformer for Sign Language Recognition," in IEEE Access, vol. 11, pp. 4730-4739, 2023, doi: 10.1109/ACCESS.2022.3231130.
D. Kothadiya, C. Bhatt, K. Sapariya, K. Patel, A.-B. Gil-González and J. M. Corchado, "Deepsign: Sign language detection and recognition using deep learning", Electronics, vol. 11, no. 11, pp. 1780, Jun. 2022.
A. Mannan, A. Abbasi, A. R. Javed, A. Ahsan, T. R. Gadekallu and Q. Xin, "Hypertuned deep convolutional neural network for sign language recognition", Comput. Intell. Neurosci., vol. 2022, pp. 1-10, Apr. 2022.
M. Zakariah, Y. A. Alotaibi, D. Koundal, Y. Guo and M. M. Elahi, "Sign language recognition for Arabic alphabets using transfer learning technique", Comput. Intell. Neurosci., vol. 2022, pp. 1-15, Apr. 2022.
P. Rathi, R. K. Gupta, S. Agarwal and A. Shukla, "Sign language recognition using ResNet50 deep neural network architecture", Proc. 5th Int. Conf. Next Gener. Comput. Technol. (NGCT), 2020.
H. Luqman, "An Efficient Two-Stream Network for Isolated Sign Language Recognition Using Accumulative Video Motion," in IEEE Access, vol. 10, pp. 93785-93798, 2022, doi: 10.1109/ACCESS.2022.3204110.
F. Shah, M. S. Shah, W. Akram, A. Manzoor, R. O. Mahmoud and D. S. Abdelminaam, "Sign Language Recognition Using Multiple Kernel Learning: A Case Study of Pakistan Sign Language," in IEEE Access, vol. 9, pp. 67548-67558, 2021, doi: 10.1109/ACCESS.2021.3077386.
E. Rajalakshmi et al., "Multi-Semantic Discriminative Feature Learning for Sign Gesture Recognition Using Hybrid Deep Neural Architecture," in IEEE Access, vol. 11, pp. 2226-2238, 2023, doi: 10.1109/ACCESS.2022.3233671.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
