ClearVoice

Deaf & Hard of Hearing: Accessible Communication Made Easy

Comment

Inspiration

Effective communication is a basic necessity in today's digital era. For the Deaf and HoH community, texting, videophone calls, and Relay Calls have become essential tools. However, these tools do not always provide the privacy needed when calling a doctor, lawyer, or a loved one. Unfortunately, many places, like doctor's offices, do not offer relay/TTY services, and most businesses lack phone systems that support SMS messaging.

To address these challenges, we have created ClearVoice. Our mission is to empower the Deaf and HoH community to communicate directly with the hearing community without relying on intermediaries or specialized equipment.

What it does

ClearVoice offers flexible calling options to its users, allowing them to make calls through their mobile phones or the web. Once they log into their ClearVoice account, which is linked to a Twilio number, they can choose the person they want to call.

When the user initiates a call from their Twilio number and the recipient answers, ClearVoice displays the conversation in text format on the user's WhatsApp bot interface using text-to-speech technology. The person receiving the call responds verbally, and ClearVoice converts their responses into text for the user via speech-to-text technology. This real-time communication allows for seamless exchanges between Deaf or HoH users and others. If the Deaf or HoH user takes time to type out their response, the person they call will receive a notification indicating that the user is composing a message. Similarly, if the person they're calling places them on hold, the user will be alerted. The call continues with this seamless exchange of communication until it concludes.

How we built it

Backend developed using Flask. Auth0 handles authentication—messaging, and calling integrated with Twilio—real-time speech-to-text conversion powered by Vosk.

Challenges we ran into

While developing the app, I had minimal experience with Twilio's call function, which made it challenging to integrate Vosk and send real-time messages on WhatsApp.

  • It was hard to come up with concepts that had real societal influence

Accomplishments that we're proud of

I developed a user-friendly interface for the deaf and hard of hearing. It required minimal text input and was based on extensive UI/UX research and analysis of client requirements.

What we learned

Over the weekend, I learned a lot:

  • Flask can be used to develop web applications that are fast, scalable, and user-friendly.
  • Twilio provides an easy way to create unique solutions.
  • Flask can help us to build backends more efficiently.
  • Vosk is useful for real-time transcription.
  • Twilio's call functionality is also very useful.
  • I learned how to display messages in real time on WhatsApp.
  • Additionally, I learned about frontend development and design.

Overall, I gained a lot from HappyHacks over the weekend.

What's next for ClearVoice

To improve emergency service accessibility, we plan on in-home emergencies. We aim to empower the deaf or hard of hearing to take action and plan to use cloud capabilities to translate text into different languages. Improving the user experience and chat functionality is also on the plan.

Share this project:

Updates