KaraokAI

Inspiration

Normally in a karaoke game, a song's lyrics and notes need to be manually input — a task known as mapping — before anyone can perform that song. In the end, only a small portion of the many songs that exist ever get mapped, leaving many people disappointed when they can't find specific songs. This is especially prominent in regards to obscure, dated, and/or foreign songs. Because we found this to be a recurring issue, we chose to make an app that could support any song for karaoke.

What it does

Our app turns any audio file into a fully mapped karaoke game, complete with scoring and multiplayer support.

How we built it

The project was developed in C# with a Windows Forms GUI. We used the Demucs API to separate the vocals from the instrumentals of a given audio file. The isolated vocal track was then processed by the Whisper API to transcribe the lyrics and generate timing information.

Challenges we ran into

The beats per minute (BPM) of a song is crucial for spawning notes and keeping time. Calculating the BPM for songs with tempo changes proved to be pretty hard so the program is sadly only optimized for songs with a static tempo.