Speech Analysis WebApp

Inspiration

An ESG centred web app which leverages Deepgram API and NLP to assist in communication through audio with visual and text cues

Environmental, Social and Governance

1. Our project aims to cultivate a culture of diversity and inclusion across its users by making allowing those on the spectrum and those that are hearing impaired to better understand speech in every form
2. This can be applied to lectures, interviews, video chats, and this analytic form of transcription has the capability of revolutionizing communication to make it more inclusive as well as more vivid
3. Microsoft Research papers show that taking notes and having other forms of information during a conversation or video allows those on the spectrum to parse and process information with far less difficulty
4. Furthermore, this project tackles the issue of the lack of tone/emotion in transcriptions of phone calls/similar for the hearing impaired through sentiment analysis and other methods

How we built it

Streamlit Framework for efficient live streaming React.js for aspects of the front end DeepGram API for the interactions with speech Wordcloud, matplotlib, TextBlob, SpaCy, NLTK and NumPy for analytics

We implement a range of cutting edge applied deep learning models, these include the use of: Python’s natural language toolkit for efficient and effective tokenization TextBlob and sentiment analysis through Naive Bayes Classification SpaCy for extractive summarization as well as keyword identification

We implemented both paradigms of usage for the DeepGram API to do speech to text conversion that are: Either real-time through a live recording feature in the web app with real-time analysis Or uploaded as a .wav and then processed with a more comprehensive suite of analytics

What we learned

Principles of software engineering, NLP, speech processing, real-time systems

What's next for Speech Analysis WebApp

We are aiming to deploy the app properly in the next week and then work gradually to enhance the UX

Built With

• deepgram
• nltk
• spacy
• streamlit
• textblob