Must Dance

Home Page
Tutorial or Google ADK Song
Log in Page

Inspiration

One of our groupmates wanted to work with OpenCV, and this led to discussions of applications that could use it effectively. We were throwing around ideas and landed on a Just Dance-esque dance coach app using OpenCV. Since we all played Just Dance as kids, it was a nostalgia trip discussing it and, ultimately, deciding upon recreating it.

What it does

Must Dance allows users to search for a dance, then compare themselves with a YouTube video in a rhythm game-like fashion (think Just Dance :D)!

How we built it

Utilizing Google ADK, a YouTube video is searched for, timestamped, and downloaded to be analyzed by Mediapipe. Then, Mediapipe is once again leveraged to compare the user's movement with the video's and scored for accuracy.

Challenges we ran into

Originally, we were using NextJS for the frontend of the project, but due to some complications with OpenCV, we opted to switch to a basic React with Vite app.

To find the YouTube videos, we originally utilized the google_search tool within Google ADK, but nearly every URL returned seemed to be either hallucinated or not publicly accessible. To combat this, we used a Python module to search YouTube directly to create a proper URL.

Downloading YouTube videos to feed them to Mediapipe also gave us quite a bit of trouble, mostly because of the added complication that the fudged URLs provided. This issue was resolved by using another Python module to download YouTube videos using a valid URL.

Accomplishments that we're proud of

Fully implementing the agentic process of searching for a video, finding what parts are most relevant to the dance queried, and downloading the video.

What we learned

How to search for and download YouTube videos using Python. Creating a sequential agent workflow to streamline the process of downloading videos.

What's next for Must Dance

Improving the efficiency of the agentic process, mostly by minimizing the time it takes to download YouTube videos using an AI agent. Adding a more detailed model for the tracking of the user, more akin to the Just Dance avatars.