Crescendo

Your personal AI music coach — analyze your practice sessions, get instant feedback on tone and technique, and follow a smart, personalized plan to master your instrument before your next lesson.

Comment

🎵 Inspiration

As passionate musicians, we’ve all experienced the frustration of practicing alone—without timely feedback, it’s easy to reinforce bad habits or feel unsure about progress. Regular lessons can be costly, and many learners—especially beginners—lack consistent access to expert guidance. We built Crescendo to bridge this gap and democratize access to high-quality musical feedback using AI.

🤖 What it does

Crescendo brings the power of AI to your music practice sessions. Our platform, Crescendo, offers four core features:

  • 🎹 A home page with intuitive navigation and instrument selection.
  • 🧑‍🏫 An AI assistant music tutor tailored for each instrument.
  • 📚 Skill-based learning resources based on user experience.
  • 🎙 A voice-activated chatbot to interactively discuss feedback.

Here’s how it works:

  • Upload or record your music directly in the app.
  • Click “Analyze Performance” to receive AI-driven feedback.
  • Our fine-tuned Gemini prompt, acting as a virtual music teacher, gives ✅ 3 strengths, ⚠️ 3 areas for improvement, ⏱ time-stamped callouts and suggestions on what to replay and fix, and 🔗 linked resources (e.g., YouTube clips) to model proper performance.

You can also “Save Recording” to track your musical growth over time. Finally, our voice chatbot extends the learning experience—allowing users to speak naturally to the AI assistant to clarify feedback or explore additional musical tips.

🧰 How we built it

  • Google AI Studio
  • Gemini
  • Next.js
  • React
  • Tailwind CSS

🚧 Challenges we ran into

  • Integration Issues: Embedding our chatbot into the Next.js homepage proved tricky since the chatbot was built with TypeScript and React. Due to time constraints, we embedded it using an iframe instead of a full integration.
  • Audio Compatibility: Working across different audio formats and ensuring compatibility with LLM inputs required careful debugging and data handling.

📚 What we learned

Building Crescendo was a deep learning journey. Some highlights:

  • Multimodal AI: We explored how LLMs interpret audio inputs and produce insightful text responses. We handled different formats, including .wav and blob files—learning how to convert and prepare files so Gemini could process them effectively.
  • Prompt Engineering: We fine-tuned Gemini with a structured, music-teacher persona to provide relevant, digestible, and actionable feedback.
  • Front-End Collaboration: Many of us were new to Next.js. We worked together to quickly onboard and build with it, and special thanks to Victoria for helping others understand key concepts. It was a true team learning experience.
  • Design Thinking: We put ourselves in the shoes of real users—musicians looking for growth—and designed an experience that is intuitive, informative, and motivational.

🚀 What’s next?

We envision Crescendo expanding to support:

  • 🎻 More instruments, including orchestral and band ensembles.
  • 🤝 Group practice feedback (e.g., duets or quartets).
  • 🔁 Advanced prompt engineering with few-shot learning and chain-of-thought reasoning.
  • 📊 Personalized practice plans using user history and goals.

We believe Crescendo can democratize music education, offering personalized, high-quality instruction at scale—regardless of income or geography. By blending musical passion with technical innovation, we’ve created a tool that not only teaches but inspires.

Built With

Share this project:

Updates