Audio to Indian Sign Language Translator

The Audio to Indian Sign Language Translator is a web application that can translate audio or textual input into the Indian Sign Language and display it using gifs.

My Role

As the sole designer, I designed and developed the layout and design of the web app, tested its usability, and collaborated with other teammates to integrate the final product. I was responsible for creating a simple and interactive webpage that can be navigated around with ease.

Background

INDIAN SIGN LANGUAGE TRANSLATOR will help bridge the gap between specially-abled people (deaf/mute) and common people. It can be a medium between them by converting the audio into sign language. It can breakdown audio into words through JavaScript Web Speech Kit and Google API and can convert each word to sign language from its vast library.

Definition of Problem

Communication is one of the basic requirement for survival in society. Deaf and dumb people communicate among themselves using sign language but normal people find it difficult to understand their language. Extensive work has been done on American sign language recognition but Indian sign language differs significantly from American sign language.

Project Goals

The design needs to reflect the user needs and the ease of navigation and usability. At the same, the application should be robust and functional even when the internet connectivity is not present.

• Design a solution to help users communicate with specially-abled people with ease, across multiple devices
• Accurate translations of speech into vocabulary
• Create a good design for visual satisfaction keeping in mind the usability of the application

Solution

INDIAN SIGN LANGUAGE TRANSLATOR will help us to bridge the gap between speechless people and common people. It would help us to be medium between them by converting their audio into sign language. It can breakdown your audio into words through Google API and can convert each word to sign language from its vast library. Not only can this words which can be recognized be directly converted into sign language without breaking them into alphabets.

Procedure

1. Audio to Text Conversion: Audio input is taken using python PyAudio module. Conversion of audio to text using microphone Dependency parser is used for analyzing grammar of the sentence and obtaining relationship between words.
2. Text to Sign Language:
   • Speech recognition using Google Speech API.
   • Text Preprocessing using NLP.
   • Dictionary based Machine Translation.
   • ISL Generator: ISL of input sentence using ISL grammar rules.
   • Generation of Sign language with signing Avatar.

Send a quick Email or Connect on