Speak Now

Speak Now uses Natural Language Processing to help those who are verbally disabled communicate with others.

Comment

Inspiration

I was inspired by a relative who had suddenly started showing signs of uncontrollable motor movements (such as in Parkinson's Disease), which meant that soon she was not able to communicate using word of mouth as fluently. Most of the time, those who would take care of her, were not able to understand what she was trying to convey, and the situation quickly spiraled out of control. When I went to visit her, I had thought of a contraption to help communicate with her: using a simple word bank in which she could pick-and-choose what word or message she wanted to convey. This greatly improved her communication dynamics with those around her. Because this idea was an instant hit, I decided to digitize the process.

What it does

The app comes with a pre-made dictionary of words which would be common for people with her condition, however, this list can be edited. The first tab contains a grid of the alphabet. If a letter is clicked, it would show a drop-down list of all the words that start with that letter. The user can click on any word shown if requested by the patient. This system is like a physical dictionary; where all the words are sorted in their specific letter category (ie starts with A, or B, etc). This makes the process of choosing a word more convenient, organized, and efficient. The next tab is a "Suggested" tab that takes into account the most-clicked words and the time of the day and uses NLP to display the 12 most relevant words at that time. This serves as a quick-access panel. The 3rd and last panel simply serve to edit the dictionary; if the patient tries to convey a word that doesn't exist in the list, the person using the app to help communicate can easily add it for future use.

How I built it

I built this version using Java in Android Studio and Firebase to use my NLP model. I am currently working on a version that works cross-platform as well.

Challenges I ran into

I ran into issues into creating my NLP algorithm, but I ended up fixing all my issues. I also had trouble for a long time integrating it with the frontend, but I finally got it to work on time.

Accomplishments that I'm proud of

I'm proud that I have created a functioning app that is actually tested and is being used by several people. I am also excited that I was able to integrate NLP into this app.

What I learned

I learned how to use file storage for storing each user's custom settings, how to use NLP and integrate it into an app, and how to start beta-testing. I feel that getting Beta Testing is one of the most important aspects of this project because it proves that the app and the concept actually works well.

What's next for Speak Now

Next, I am planning to contact doctors and have even more people test it so I can have a greater understanding of how I can remove the communication barriers of disabled patients.

Share this project:

Updates