Optic Audio Communication Interface

Side View with Product Diagram
Front View
Alternate Side View

Inspiration

Only 1% of the deaf population and only 0.3% of the general global population knows sign language. Even with the ability to read lips, about 40% of the English language is misinterpreted—not to mention the fact that with the pandemic, reading lips has become near impossible with so many interactions now involving a mask. This has left many deaf people with few options to interact with people which in turn creates fewer and fewer conversations with deaf people. The Optic Audio Communication Interface will change that. However, this invention doesn't just help those who aren't able to hear. Many people have trouble communicating due to the majority of the population around them speaking a different language. For example, around 4 million Americans only speak Spanish and cannot understand English which is a large majority of the population's only language. Therefore, the Optic Audio Communication Interface can also help people to communicate with those speaking a language other than their own. Doctor Jennifer Rogoff, pediatric audiologist for Bergen County Special Services, shared deeply on the effect our invention could have on the education of deaf and minority students.

What it does

The Optic Audio Communication Interface provides visual live captioning services for deaf people which allows them to read what a person is saying to them as the conversation is happening. A Raspberry Pi Zero serves as the computer of the system, with a small microphone sending information to the speech-to-text software that allows the live subtitling to work. The captions are then displayed on a transparent LCD screen that the user can see through the glasses. In order to use the language translation software, the user chooses the language they would like to translate from and translate to, and the software will translate what the microphone picks up so that the captioning will be displayed in the user's native language.

How we built it

The software made in Python uses the module speech_recognition to recognize speech patterns in an audio file and reference a dictionary of words to make them into sentences. Then using numerous language dictionaries to translate the text into over a hundred languages. In order to build the glasses, a Raspberry Pi Zero is encased in a 3D printed case on the frame of the glasses with the microphone attached to the end of it. A rubber sound tube directs the sound from outside to the sound port on the Mouser Electronics SPU0414HR5H-SB microphone. A small DVI connector is led to the transparent LCD screen, which uses a transparent substrate or front of the screen to make white pixels appear clear. Making a see-through display until the subtitles would appear.

Challenges we ran into

We ran into some issues when it came to software to create the 3d model. It was software that we had never used before. When we were developing the software for the speech, we couldn't get the software to natively record audio, leading us to import audio files elsewhere.

Accomplishments that we're proud of

Accomplishments we are proud of is we are giving deaf people and people who speak other languages the opportunity to communicate with everyone. These glasses could give these people the opportunities to go on job interviews, to experience a better education, and be able to interact and connect with people better.

What we learned

Through this project, we learned how to handle 3D software such as Adobe Dimension, a program we hadn't used in the past. We learned multiple new modules, such as the "goslate" and "speech_recognition" modules. Our teamwork and communication skills greatly improved during this Hackathon as well.

What's next for Optic Audio Communication Interface

The OACI, for short, will next be turned into a functioning prototype. In a few years maybe even do trials for this invention and see the impact that we have created. We want the OACI to be a market product with a low-profit margin just to grow our invention and improve it. Features we would like to add include language recognition for better translation. Eventually, we'd turn it into a non-profit product.