Sign Language Recognition Using Convolutional Neural Network

Abstract

Communication, the sharing of information, ideas, and feelings, is typically facilitated through a common language. However, for individuals who are deaf and mute, communication presents unique challenges due to the inability to hear or speak. Sign language emerges as a crucial medium for communication among the deaf and mute and with those who can hear and speak. Unfortunately, the broader population often underappreciates the significance of sign language, resulting in communication barriers.

To address this communication gap, we propose a machine learning solution—an innovative model designed to recognize various sign language gestures and translate them into English. Current Indian Sign Language Recognition systems, while employing machine learning algorithms, often lack real-time capabilities. In this paper, we introduce a method to construct an Indian Sign Language dataset using a webcam. We then leverage transfer learning to train a TensorFlow model, culminating in the development of a real-time Sign Language Recognition system. Notably, this system demonstrates commendable accuracy, even with a relatively modest dataset.

The creation of a real-time sign language detector marks a significant stride in improving communication between the deaf and the general population. We proudly present the implementation of a sign language recognition model, rooted in a Convolutional Neural Network (CNN) and utilizing a Pre-Trained SSD Mobile-Net V2 architecture. Applying transfer learning, we have achieved a robust model consistently classifying sign language gestures. This groundbreaking innovation not only facilitates effective communication but also serves as a valuable tool for individuals learning sign language, providing practical opportunities for practice.