pHearless

The website actively transcribing audio
The list of currently active keywords
Some recommended keywords
Placeholder for a profile system
The Android phone UI
Android UI continued

Inspiration

Attending the Apple Swift workshop opened up our team's eyes to the possibilities of applications to help people. Although we didn't end up using Swift for our hack, this inspiration pushed us to making pHearless, a project we believe can help everyone.

What it does

pHearless runs in the background, constantly taking in a live transcription of any words it picks up. Users have the capability to select keywords, and when any of those keywords are detected, an alert will be raised.

How we built it

Originally wanting to make an IOS app, we wanted to use Xcode to develop our app, however this proved to be tedious and difficult to set up; we then used Figma to implement the UI/UX design we desired. This also was difficult to work with when trying to export this design elsewhere like Android Studio, so we eventually used our work as a framework to recreate the same experience in HTML.

Challenges we ran into

As a team with 3 first-time hackers and almost zero combined software development experience, creating an entirely software hack was a huge, huge challenge. Although our concept was computationally simple, it still took us many, many failed attempts and very little sleep to get our demo working. A more specific challenge we ran into while make our demo was the challenge of letting go. It felt awful to spend hours upon hours researching and attempting to make one approach work, but ultimately we had to make concessions to get our hack done. It was only due to all the help from others, and especially from our 4th member (who unfortunately stepped out) that we overcame these challenges.

Accomplishments that we're proud of

Drawing back to the challenges we mentioned earlier, we were all very proud of making it through this hackathon and having something to show for it. Every single one of us had to learn so, so much new information and learn to apply it in such a short period of time. Beyond our technical achievements, we are also proud of our final product. Not only are we happy with how it runs and how it looks, but we are very proud of the help that this service can provide to anybody! Although this is especially useful for the hearing-impaired, we've also realize that something like this is incredibly valuable for anybody, from people working in noisy environments, to people who are busy with other tasks. We have high hopes for what a product like this could potentially do, and that's something we're all very proud of.

What we learned

The correlation between the quality of the sound-to-text API and the difficulty of implementing it into our project was quite high. At first we wanted to use the Google Cloud sound-to-text API because it has a high degree of accuracy with picking up speech, however even after countless hours, we were unable to figure out how to use it for our project. We eventually used the Web API, which has a lower degree of accuracy, but it was significantly easier to implement into the project. We also learned a lot of UI/UX design skills as we used Figma to build a prototype of our original idea for an app. Beyond just the technical aspect of the project, we also learned a lot about teamwork and hacking in general. We learned about how to collaborate and spread our resources efficiently, we learned how to utilize generative AI to assist our efforts effectively, and we especially learned the value of teamwork and mentorship!

What's next for pHearless

For the future, our hope is to implement a more accurate speech-to-text API, as we had to settle for a cheaper one rather than the Google Cloud API we had originally wanted. We also really want to implement this application into a phone app rather than a website, since this increase in portability would increase the usefulness of the product, and the phone also opens up possibilities of alerts using vibrations. We also want to set up profile-retaining capabilities to allow users to store keywords and allow for real long-term use in that way, as well as possibly creating further personalization, such as recognizing certain voices or sounds. In short, we still have a ton of ideas and hopes for pHearless, and we would love to realize them someday!