SKIB | Devpost

Main page
Recording audio (once mic button is pressed)
Audio recording from frontend automatically gets saved to backend
Backend generates Amplitude vs Time graph based on audio recording
Also creates Tone vs Time graph

Inspiration

Now more than ever, we are seeing the importance of good conversation skills in everyday life. Conversations take place with anyone, from a family member to a teacher to a friend, acquaintance, or someone you’ve never met before -- but conversations don’t always go the way you’d expect. Needless to say, some appear to be more skilled at smooth conversations than others.

Our program aims to bridge that gap.

Through our program, people will be better able to connect with others, express themselves clearly, and navigate social and professional situations with confidence. We were inspired by the difficulties we faced when forced to do public speaking (such as at club meetings, or for school projects), and through this project we hoped to improve our own public speaking skills, and help others improve their skills too.

What it does

We use computer speech recognition, generative AI, and tone analysis to provide feedback for the user’s speech. We look at both the audio, as well as the transcript along with timestamps to check for the speaker’s use of filler words (like, um, etc.), number of pauses, vocabulary usage, pace of speech, and volume of speech to suggest the user potential points of improvement. In addition, we do fast foward fourier analysis to determine the fluctuation of the user’s tone, to make sure that the user’s speech is not too monotone.

How we built it

We used HTML+CSS and tailwind in the front end. For the backend we used Flask, OpenAI's API, Matplotlib, and Librosa to do graphing, analysis, and provide feedback/suggestions to the user.

Challenges we ran into

We had trouble converting the sound files from the browser (mp3) to a file type compatible with Computer Speech Recognition (PCM-wav files). In addition, we had a lot of trouble with integration between back-end and front-end, and so the front end only included the Computer Speech Recognition and audio recording UI. The backend contains points of improvement for the user's speech along with graph visuals.