StudySaber

What if studying became Beat Saber? We turned active recall into a high-energy rhythm game that keeps you locked in and focused. Think fast, move fast, learn faster and retain more. Study to the beat.

Comment 4

Inspiration

Traditional studying methods often feel monotonous and fail to engage students, leading to poor retention and recall. We were inspired to transform studying into an interactive, game-like experience that makes learning addictive rather than tedious. Our goal was to leverage gamification principles, immediate feedback, progress tracking, and competitive elements, to create a study tool that not only makes learning more engaging but significantly improves information recall through active recall testing.

The concept of combining rhythm game mechanics (inspired by Beat Saber) with educational content emerged from the idea that physical movement and rhythm can enhance memory formation. By requiring students to actively recall information in a time-pressured, game-like environment, we force deeper cognitive processing that leads to better long-term retention.

What it does

StudySaber is a comprehensive gamified study platform that transforms any PDF document into an interactive rhythm game quiz. The platform consists of two main components:

Chrome Extension: A Pomodoro timer that helps users structure their study sessions, with an integrated AI study assistant powered by ElevenLabs voice synthesis. Once study time is complete, users can seamlessly transition to the web application to test their knowledge.

Web Application: The core experience where users can upload PDF documents, which are automatically processed by AI (Google Gemini) to generate quiz questions. Each document gets a custom AI-generated music track (using Google's Lyria) with BPM matched to difficulty level (easy, medium, hard). Users slash blocks representing quiz answers in a rhythm game format, receiving immediate audio feedback via ElevenLabs for correct/incorrect answers.

The platform includes social features: an Explore page where users can discover and play quizzes from other students' notes, with an upvote system to highlight quality content. A comprehensive profile system tracks performance metrics, and a token-based store allows users to customize their game experience (saber colors, rail designs) by earning tokens through high scores. A global leaderboard adds competitive motivation, ranking users based on cumulative performance across all games.

How we built it

Frontend: Built with React and TypeScript, using Three.js for 3D game rendering and React Three Fiber for declarative scene management. The game engine implements custom beat detection algorithms, dynamic question spawning synchronized to music, and hand tracking for gesture-based controls. Framer Motion provides smooth UI animations, and Tailwind CSS creates the futuristic neon aesthetic.

Backend: FastAPI (Python) serves as the API layer, handling document uploads, quiz generation, and game data retrieval. We integrated multiple AI services: Google Gemini for PDF-to-quiz conversion with strict word limits for game compatibility, Gemini's Lyria API for music generation with BPM control, and ElevenLabs for natural voice synthesis in both the study assistant and game feedback.

Database: Supabase provides PostgreSQL for document storage, user profiles, game statistics, and social features. We implemented Row Level Security policies for data protection and used Supabase Storage for PDF and audio file management.

Chrome Extension: Built with TypeScript and Vite, featuring a Pomodoro timer with study session tracking and direct integration with the AI assistant API.

Game Mechanics: Custom audio analysis pipeline detects beats in generated music, synchronizes question blocks to rhythm, and implements a scoring system based on accuracy and combo multipliers. The difficulty system adjusts BPM, spawn rates, and question complexity.

Challenges we ran into

AI Integration Complexity: Integrating multiple AI services (Gemini, Lyria, ElevenLabs) required careful API management and error handling. We had to ensure quiz answers met strict token/word limits for game compatibility and handle API rate limits. Implementing parallel processing for quiz and music generation was crucial to reduce latency and improve user experience.

Avatar Facial Expressions: We attempted to implement facial expression tracking for the AI avatar to provide visual feedback during gameplay, but encountered significant challenges with real-time expression detection and synchronization. We pivoted to audio-based feedback using ElevenLabs, which proved more reliable and less computationally intensive.

Feature Prioritization: With limited time, we constantly had to balance adding new features versus refining core functionality. Decisions like implementing the social explore feature versus perfecting game mechanics required careful consideration of user value versus development time.

Performance Considerations: Parallel processing became essential—running quiz and music generation concurrently reduced latency significantly. We also had to optimize rendering and state management to maintain smooth gameplay performance.

Accomplishments that we're proud of

AI-powered quiz generation from PDFs: Automatically converting any PDF into structured quiz questions with validated answers in seconds.

Dynamic AI music generation: Generating unique, beat-detectable music tailored to each document and difficulty level.

Real-time speaking avatar: Building an interactive 3D avatar that speaks study content.

Year-end Wrapped feature: Creating a Spotify Wrapped-style analytics system with global percentile rankings and animated visualizations.

Customization store system: Implementing a token-based economy where users earn currency through gameplay achievements.

Browser extension integration: Extending the platform into a productivity tool with Pomodoro timers and quick document access.

Multi-API orchestration: Successfully integrating Google Lyra, Gemini AI, ElevenLabs, MediaPipe, Supabase, and TalkingHead into a cohesive system.

What we learned

This project pushed us to learn new technologies rapidly, Three.js for 3D rendering, multiple AI APIs, real-time audio synchronization, and complex state management. We discovered our ability to integrate complex systems and adapt when things didn't work as expected. The experience taught us that with clear goals and effective collaboration, we could build something ambitious in a short timeframe.

What's next for Studysaber

StudySaber is ready for use and we're excited for students to trial it! The platform is deployed and fully functional. We want students to have the best study experience possible, and we're eager to gather feedback from real users to continue improving the platform. Our goal is to make studying more engaging and effective for students everywhere.

Built With

Share this project:

Updates