Inspiration
StudyOwl was directly inspired by Quizlet, which presents study material in a game-like format. We wanted to make something similar to that, but we felt that current apps were too time-consuming and invariable once constructed. We wanted to add the functionality of simply uploading a huge chunk of text/audio — like a lecture recording, class transcript, or notes — and creating an entire game from just that input. Additionally, we were inspired by games like Typeracer, which allows you to compete against your friends in a PVP environment, which would foster more engagement and learning with the material.
What it does
StudyOwl accepts an uploaded audio or text input. If an audio file is uploaded, it first transcribes the input into text. We then use prompt engineering to turn the text into pairs of keywords and definitions and pairs of questions and answers. Next, it creates a table showing these keyword/definition pairs to let the user study these key terms. Once the user feels ready, they create a room to play the game with their friend, who uploaded the same file and studied the same material. Their friend can then join the room, and they'll play together! The room prompts both players with the question portion of the question/answer pair. Both players answer the question to the best of their ability. StudyOwl then assesses how close the user response is to the StudyOwl-generated answer. Depending on how close the answers are, StudyOwl marks the response as correct or incorrect. Users play as two owls at a campfire. The owls need to answer questions correctly to cook their marshmallow. If they answer incorrectly or run out of time, the marshmallow burns! The first player to cook 20 marshmallows wins.
How we built it
We used Flask, HTML, JavaScript, and Tailwind to display the website. We utilized prompt engineering with VertexAI with Gemini to first turn the .wav files into text, then turn the text into keyword/definition pairs and question/answer pairs. These question/answer pairs are given to the game. We utilized SocketIO to create a room for a friend to join. Our game itself includes assets made in Adobe Photoshop. With these assets, we created a 1v1 study game using spaCy to find the sentence similarity between the user's answer and the actual answer. The more similar the response, the more points the user recieves.
Challenges we ran into
Our initial similarity comparison library ran into compatibility issues, forcing us to look for alternative solutions. Thankfully, we were able to modify another sentence similarity model to suit our requirements. Our animated sprites also took longer to produce than anticipated, leading to backlogs in our workflow. Furthermore, as this was our first time integrating VertexAI with Gemini, we had troubles with setting up the API. VertexAI requires IAM connections to set up the project, making it different from the API keys we're used to.
Accomplishments that we're proud of
We're proud of our made-from-scratch pixel art and animations. Our user-friendly website works seamlessly and makes studying fun with friends. We believe that this website can actually help people study even if they miss a lecture.
What we learned
We learned how to create rooms with SocketIO to have multiple people play at once. In addition, we learned how to effectively utilize the Google Gemini API with VertexAI. Lastly, we learned how to effectively animate pixel art to create a cozy vibe.
What's next for StudyOwl
We plan to integrate support for more players, which means more owl animations and a more complex scoring system. In addition, we plan to create a server so people can play with each other from different computers.
Log in or sign up for Devpost to join the conversation.