MustDance

Inspiration

Learning to dance can be intimidating, especially for beginners. One of our teammates is a dancer, and but the rest of us want to learn how to dance. There can be a feeling of stigma around going to beginner dance lessons, especially when you feel out of place in that community and lack confidence in the area. We provide an alternative that lets you learn how to dance in the comfort of your own home, to songs that you actually enjoy!

What it does

The app takes any song, detects beats the tempo and creates a choreography video that is synced to the music. From there, you just follow along! Follow the moves as best as you can, and we'll give you a score so you can see your improvement over time!

How we built it

We used Next.js for the frontend, and a Python Flask server for the backend. Using librosa we were able to use algorithmic audio analysis to get beat positioning, and using moviepy we were able to sync our dance clips to any song. Once the user finishes dancing, we used opencv to perform pose estimation on both the user's video and our reference video. We find how different the poses are at certain time intervals using a variety of body landmarks to determine a score.

Challenges we ran into

We in fact ran into a lot of problems. The first big issue we ran into was how to calculate a score. Finding the difference between two videos was challenging, and testing it was even harder given the subjectiveness of the grader. There was a lot of tuning, but we got it working in the end!

Another challenge we ran into was syncing our dance clips to an arbitrary song. There was a lot of precise video editing involved with this, and making sure our clips meshed together nicely was something we had to keep in mind.

Accomplishments that we're proud of

We made a fully working dance app in under 18 hours! We used a lot of new technology that we haven't used before, and seeing it all come together was incredibly rewarding. First ever all-nighter for all of us was also notable.

What we learned

We learnt a lot about working full-stack, and how to develop under time pressure. These constraints led to us making a lot of tradeoffs, and learning how to make decisions efficiently was very important. We also learnt that McDonald's coffee is super great at 1am.