This project, built during BoilerMake XII, is a web application that helps musicians analyze and refine their phrasing, timing, and expression using waveform visualization and AI-driven feedback.
- YouTube-to-Waveform Conversion β Extracts and visualizes waveforms from YouTube audio clips.
- Real-Time Recording & Waveform Generation β Users can record their playing and compare it visually.
- AI-Powered Phrasing Feedback β Uses Google Gemini AI to compare recordings and provide timing, rhythm, and expression insights with improvement suggestions.
- Waveform Alignment β Automatically synchronizes recordings for accurate side-by-side comparisons.
- Interactive Moving Cursor β Tracks playback across the waveform to help users visualize phrasing dynamics.
Frontend: React, Web Audio API
Backend: Flask, Express.js
AI & Processing: Google Gemini AI (GenAI), YouTube audio extraction, waveform analysis
π₯οΈ How It Works
- Upload a YouTube link or record your own playing πΌ
- The app extracts and visualizes the waveform π
- Google Gemini AI analyzes phrasing, timing, and expression π€
- Users receive AI-generated improvement suggestions πΆ
git clone https://github.com/your-repo-name.git
cd your-repo-name
pip install -r requirements.txt
npm install \# Install frontend dependencies
python3 app.py \# Start backend
npm run dev \# Start frontendπ Built by Bryan Yoo, Alex Liu, Christina Lee, Elaine Huang at BoilerMake XII