speachu

using AI to provide feedback on presentations and speeches to improve public speaking skills

Comment

Inspiration

Ah well, during lockdown, everyone just gets used to texting and sometimes we forget how to speak. Maybe just me, but it's a problem. School is also starting soon and that means more presentations! I don't do very well on presentations so I wanted to create something that will help me and others like me. I wanted something like this because when I practicing by recording and listening to myself I usually just ignore my own voice or just completely too lazy to do anything. The other people in my household are too busy to listen to me present so well that inspired speachu !! ✨

What it does

It is a mobile app that analyzes and provides feedback/tips to improve on presentations and speeches using AI technology to help students, people in the workplace, or just anyone affordably improve their public speaking and presenting skills.

practice mode

  • Simulates an audience
    • currently by displaying a video
    • will use VR to simulate an audience in later versions
  • Allows you to record your presentation’s audio and listen to it again

(future versions will allow customization)

assessment mode

  • in addition to practice mode
  • audio will be analyzed by our AI to provide feedback, tips to improve, and an overall grade based on:
    • energy/tone
    • speed
    • amount of filler words

(future versions will grade you on other things such as pronunciation/enunciation)

How we built it

Challenges we ran into

  • keep getting errors when using the ai library
  • almost gave up and switched topics
  • everything went pretty smoothly (thanks to the help of mentors [tyty])
  • until we had two files, one python, one react and we had to use each other to make it work, sadly we didn't have time to figure it out especially because we were all new to this and a teammate was absent for most of the hackathon because she was sick)

Accomplishments that we're proud of

we all did something new!

  • chelsea: never made an app before
  • fatma: never touched ai stuff
  • annie: never made interactive figma prototypes
  • kailey: never attended a hackathon

What we learned

the above :" )

What's next for speachu

  • interview mode
    • can ask preset questions
    • uses watson ai chatbot + text to speech function to ask questions like in an interview
    • will also provide feedback on responses
  • vr
    • simulate real people in front of you
    • can also ask questions with interview mode
  • video mode (analyzes and provides feedback based on)
    • facial expressions (ex: smiling)
    • posture (ex: swaying, standing up straight)
  • classroom mode
    • allow student to share their score / results
    • maybe mini friendly competition against each other for better scores
    • but for their own good as well
    • maybe classroom codes
  • save the audio and feedback / show improvement
  • colour-coded slider for audio to respective timestamped feedback
  • profiles!
  • improved tech and analysis (mumbling/enunciation/pronunciation)
  • more customization
  • making a web version, compatibility across different devices (expo recording doesn't work on ios)
  • save past ones
  • be able to upload prerecorded audios to analyze

Links

please note: the app is not stylized yet because of time constraints and does not have ai analysis functionality yet, however to see the full stylized version you can see our mockup and interactive prototype with the FIGMA link and the working AI implementation in the GITHUB link; the app has fully functioning audio capture, playback, and audience simulation though

also please note: the expo snack link requires the Expo Go app to scan the QR Code. Also recording audio doesn't work on iOS as according to the documentation

(audience simulation video courtesy to pond5)

Built With

Share this project:

Updates