-
Snapshot of website homepage.
-
Snapshot of results sent to pediatrician.
Inspiration
Specific Language Impairment (SLI) is a prevalent disease among young children and it is important to identify the symptoms of SLI early in order to fix the problem when the child is young. Our team had recently gotten into natural language processing tasks and we were introduced into NLTK at our school, so we realized that we could use these skills to help prevent SLI symptoms from aggravating over time.
What it does
SLIagnosis is a web application that aims to quickly and accurately determine whether a user has Specific Language Impairment. The user records themselves describing the scenes in a short book, with the audio recording getting transcribed into a text transcript, and determining whether the user has SLI using machine learning. Based on the results of the model, we output the transcribed text transcript determined as well as the values of individual features calculated by the speech-to-text model, with the results getting automatically forwarded to a user’s doctor.
How we built it
We implemented machine learning algorithms to extract a text transcript from an audio recording using the Speech Recognition library, performed feature extraction using regular expressions and NLTK, and fed those features into a machine learning model developed using Scikit-Learn to determine whether a user has SLI. The front-end web application was built using HTML, CSS, and JavaScript with the back-end built using the Flask framework in Python. We implemented a texting feature that forwards a diagnosis to pediatricians/physicians using the Twilio API. Most of our development was done on Visual Studio Code, with small pieces of the application tested on Google Colaboratory integrated python notebooks. We used the Git protocol to integrate individual pieces into the overall system.
Challenges we ran into
We ran into challenges regarding how to record a user talking in order to analyze it.
Accomplishments that we're proud of
We spent a lot of time making the user interface compatible with the machine learning models trained separately. The website hosts the model and runs efficiently when the user passes in their input. This was our first time using an API such as Twilio and we were able to get that working effectively.
What we learned
We learned a lot about processing sound data and applying natural language processing using regular expressions and NLTK. We also improved our web development skills such as utilizing a device’s microphone to record audio of a user talking.
What's next for SLIagnosis
We are planning on using the Google Maps or Bing API to automatically suggest pediatrician offices to visit for speech therapy/treatment based on their proximity to the user and wait time.
Log in or sign up for Devpost to join the conversation.