Breaking the Silence: An innovative ASL to Text Conversion System Leveraging Computer Vision & Machine Learning for Enhanced Communication
Keywords:
American Sign Language (ASL), sign capture, sign-to-textAbstract
An innovative approach for converting American Sign Language (ASL) into text is proposed in this paper. The technology accurately recognises and instantly translates ASL signals into written text using cutting-edge computer vision and machine learning algorithms. A letter recognition model, a gesture recognition module, and a text generating module make up the suggested system. Then, using the recognised movements, the text production module produces text. The proposed technology may enhance hearing and deaf people's ability to communicate. To help deaf and mute people communicate with other people more successfully, the system can be used to translate ASL into text. Our study describes how ASL to text converters might be used in accessibility services, education, and ordinary communication.
Downloads
References
A.A. Abdulhussein, and F.A. Raheem, "Hand Gesture Recognition of Static Letters in American Sign Language (ASL) Using Deep Learning," Engineering and Technology Journal, vol. 38, no. 4, pp. 926–937, 2020.
J. Sahoo, A. Prakash, P. Pławiak, S. Samantray, "Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network," Sensors, vol. 22, no. [issue], pp. 706, 2022.
K.K.K. D. Bhavana, "Hand Sign Recognition using CNN," International Journal of Performability Engineering, vol. 17, pp. 314–321, 2022.
V. Adewale, A. Olamiti, "Conversion of Sign Language To Text And Speech Using Machine Learning Techniques,” JOURNAL of RESEARCH and REVIEW in SCIENCE, vol. 5, 2018.
KM Dafnis, E Chroni, C Neidle, D Metaxas, "Isolated Sign Recognition using ASL Datasets with Consistent Text-based Gloss Labeling and Curriculum Learning," in Seventh International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual (SLTAT 2022), LREC, Marseille, France, 2022.
P.M.N., P. A., S. R.S., A.S. Vadakkan, A. Khateeb, D. K.P, "Sign Language Converter Using Feature Extractor and PoseNet,” Webology, vol. 19, pp. 5476–5486, 2022.
DD. Sood, "Sign Language Recognition using Deep Learning," International Journal for Research in Applied Science and Engineering Technology, vol. 10, pp. 246–249, 2022.
J. Smith, J. Doe, "Real-time Sign Language Gesture Recognition,” 2021.
E. Johnson, D. Lee, "Enhancing Sign Language Recognition," 2019.
A Patel, L Wang, “Gesture Recognition in Challenging Environments,” 2022.
S. Thakar, S. Shah, B. Shah, A.V. Nimkar, "Sign Language to Text Conversion in Real Time using Transfer Learning,", Preprint ArXiv:2211.14446 [Cs], 2022.
A. Ojha, A. Pandey, S. Maurya, A. Thakur, D.D. P, "Sign Language to Text and Speech Translation in Real Time Using Convolutional Neural Network," 2020.
Y. Obi, K.S. Claudio, V.M. Budiman, S. Achmad, A. Kurniawan, "Sign Language Recognition System for Communicating to People with Disabilities," Procedia Computer Science, vol. 216, pp. 13–20, 2023.
F.R. Khan, H.F. Ong, N. Bahar, "A Sign Language to Text Converter Using Leap Motion," International Journal on Advanced Science, Engineering and Information Technology, vol. 6, pp. 1089, 2016.
S. Rajaganapathy, B. Aravind, B. Keerthana, M. Sivagami, "Conversion of Sign Language to Speech with Human Gestures," Procedia Computer Science, vol. 50, pp. 10–15, 2015.
T. Arsan, O. Ulgen, "Sign Language Converter," International Journal of Computer Science & Engineering, vol. 6, pp. 39–51, 2015.
R. Khan, “Sign Language Recognition from a webcam video stream,” Mediatum.ub.tum.de, 2022.
G.Z. de Castro, R.R. Guerra, F.G. Guimarães, "Automatic Translation of Sign Language with Multi-Stream 3D CNN and Generation of Artificial Depth Maps," Expert Systems with Applications, vol. 215, pp. 119394, 2023.
X. Chen, L. Su, J. Zhao, K. Qiu, N. Jiang, G. Zhai, "Sign Language Gesture Recognition and Classification Based on Event Camera with Spiking Neural Networks,”, Electronics, vol. 12, pp. 786, 2023.
M. Oudah, A. Al-Naji, J. Chahl, "Hand Gesture Recognition Based on Computer Vision: A Review of Techniques,” Journal of Imaging, vol. 6, pp. 73, 2020.
M. Al-Hammadi, G. Muhammad, W. Abdul, M. Alsulaiman, M.A. Bencherif, T.S. Alrayes, H. Mathkour, M.A. Mekhtiche, "Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation," IEEE Access, vol. 8, pp. 192527–192542, 2020.
A. Kamble, J. Musale, R. Chalavade, R. Dalvi, S. Shriyal, “Conversion of Sign Language to Text,” 2023.
P.B. Sonawane, A. Nikalje, “Text to Sign Language Conversion by Using Python and Data-base of Images and Videos,” 2018.
K. Brady, M.S. Brandstein, J.T. Melot, Y.L. Gwon, J. Williams, E. Salesky, M.T. Chan, P.R. Khorrami, N. Malyska, K. Brady, M.S. Brandstein, J.T. Melot, Y.L. Gwon, J. Williams, E. Salesky, M.T. Chan, P.R. Khorrami, and N. Malyska, “American Sign Language Recognition and Translation Feasibility Study,” Eprints.soton.ac.uk., 2018.
J. Shin, A. Matsuoka, Md.A.M. Hasan, and A.Y. Srizon, “American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation,” Sensors, vol. 21, pp. 5856, 2021.
Sunusi Bala Abdullahi, Kosin Chamnongthai, "American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven Multi-Stacked Deep LSTM," Sensors, vol. 22, pp. 1406–1406, 2022.
P.R. Sanmitra, V.V.S. Sowmya, K. Lalithanjana, "Machine Learning-Based Real-Time Sign Language Detection,” International Journal of Research in Engineering, Science and Management, vol. 4, pp. 137–141, 2021.
T.N. Abu-Jamie, S.S. Abu-Naser, "Classification of Sign-Language Using Deep Learning by ResNet," Philpapers.org, 2022.
C. Neidle, A. Opoku, and D.N. Metaxas, “ASL Video Corpora & Sign Bank: Resources Available through the American Sign Language Linguistic Research Project (ASLLRP),” ArXiv (Cornell University). 2022.
J. Mary, Samonte, J. Carl, R. Guingab, Relayo, J. Mark, J. Sheng, D. Ray, and Tamayo, “Using Deep Learning in Sign Language Translation to Text,” 2022.
P. Bagane, S. Vishal, R. Raj, T. Ganorkar, and Riya, “Facial Emotion Detection using Convolutional Neural Network,”IJACSA, 2022.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
