Lingual

Login Page

Inspiration

Most of the people with hearing disabilities have problems in understanding the lectures in classrooms as they cannot completely comprehend the professor. Moreover, in a diverse university such as TAMU, everyone has a different accent and many times it becomes difficult for the students to understand the professor completely. This may be due to students not able to comprehend the difficult words in a language other than their native one or because of the accent. In addition, many professors feel more comfortable in giving lectures in thier native language. So our application is a step towards helping those and making the classrooms more accessible for them with the help of technology.

What it does

Our application translates the lecture in the classroom or meeting rooms into many different languages in real time. The application takes the input from the system's microphone which means a person, an audio file, or a video file can work as the input for the system. It converts the audio to text and the language of the audio file can be specified as well. Thus, the lecture doesn't have to be in English. Once the system has the text in the language of the professor, it sees what all students are currently listening to that course. The system then streams the text to individual users in the language of their choice.

How we built it

The application has been developed in Java using Javascript for front-end and MySQL as database. Spring MVC has been used for client-server communication. For speech to text conversion, 'web-speech-api' has been used. For inter language text conversion, 'google translate api' were used.

Challenges we ran into

The main challenge was to work with the live streaming. The http connections through which the students will receive live data has to be open until explicitly closed. Passing a chunk of data through this connection periodically was a complicated task. Another challenge was to break the speech into chunks for text conversion. If the speech is fast, the chunk of data that is converted in one go, becomes large which slows down the stream. If the speed is optimal, the streaming works quite well.

Accomplishments that we're proud of

It breaks the language barrier in learning and will make the world a better place with easy and efficient knowledge sharing. We can share freely even with differently abled persons who face difficulties in hearing. We can now interact with them without sign language, in the same manner as we interact with everyone. It gives us great joy in believing that it will make the life of many people so easy and will help them interact with everyone in a lot better way. It will provide easy access to everyone despite lingual and physical differences. In not only helps the differently abled people, it makes the lectures more understandable for everyone and also helps them to review the lecture afterwards.

What we learned

We learned about the challenges faced by the differently abled people in their day to day life and how important is to make all the facilities available to them so that they can interact with the people like they want to.

What's next for Lingual

-Lingual can be integrated online lectures as well. As online lectures have videos, Lingual can be used to create subtitles for the video on the go. -Lingual can be used for one-on-one conversation with people having hearing disabilities. This is a good alternative for those who do not know sign language. -It can be used for taking notes in a meeting in offices.