Inspiration
After experiencing multiple online meetings and courses, a constant ending question that arose from the meeting hosts was always a simple "How are we feeling?" or "Does everybody understand?" These questions are often followed by a simple nod from the audience despite their true comprehension of the information presented. Ultimately, the hosts will move on from the content as from what they know, the audience has understood the content. However, for many of us, this is not the case because of the intense Zoom fatigue that overcomes us and ends up hindering our full comprehension of all the material. It is extremely important to allow teachers to gain a better understanding of the more realistic "vibe" of the audience in the meeting, and thus improve the overall presentation method of future meetings.
What it does
Our website plays a role in allowing meeting hosts to analyze the audiences receptiveness to the content. The host would upload any meeting recording as a .mp4 file on our website. Our application will output a table with each individual’s name and the most occurring “emotion” for each individual during the meeting. Based on the results, the host would know how to acknowledge his/her group's concerns in the next meeting.
How we built it
We utilized the Hume.AI API to allow us to do an analysis on the emotions of the individuals in the meeting. Utilizing the data the Hume.AI provided us we ran an analysis on the average emotions each meeting participant felt throughout the meeting. That data was processed in Python and sent to our frontend using Flask. Our frontend was built using React.js. We stored the uploaded video in Google Cloud.
Challenges we ran into
From our team, two members had no experience in HTML, CSS, and JavaScript, so they spent a lot of time practicing web development. They faced issues along the way with the logic and implementation of the code for the user interface of our website. This was also our first time using the Hume.AI API and also our first time playing with Google Cloud.
Accomplishments that we're proud of
Every team member successfully learned from each and learned a great deal from this hackathon. We were a team that had fun hacking together and built a reasonable MVP. The highlight was definitely learning since for half the team it was their first hackathon and they had very little prior coding exposure.
What we learned
From our team, two of the members had very minimal experience with web development. By the end of the hackathon, they learned how to develop a website and eventually built our final website using ReactJS. The other 2 team members, relatively new to AI, explored and applied the HumeAI API for the first time, and learned how it can be applied to detect individual facial expressions and emotions from a video recording. We also were able to successfully connect frontend to backend for the first time using Flask and also used Google cloud storage for the first time. This hackathon marked a lot of firsts for the team!
What's next for BearVibeCheck
We hope to further improve upon the UI and make our algorithm faster and more scalable. Due to the nature of the hackathon a lot of pieces were hard-coded. Our goal is to provide this resource to Cal Professors who teach in a hybrid or fully online setting to allow them to gauge how students are feeling about certain content material.
Log in or sign up for Devpost to join the conversation.