Detec-Tune

Home Page - Light Mode
Home Page - Dark Mode
Upload Page - Light Mode
Upload Page - Dark Mode
Result Page - Light Mode
Result Page - Dark Mode

Inspiration

All of us are addicted to music and we always thought about what our favorite songs would sound like in another language (like Hindi). This would also help non-native English speakers better understand English songs by translating the lyrics of the song into their native language. This was the reason why we built this project.

What it does

Our website has three pages -

The Home Page - It displays the list of songs that have been recognized in the past and also has a search option that allows a user to search for a particular song by typing the lyrics of the song.
The Upload Page - This page allows the user to record or upload the songs that they would like to translate. There is also a dropdown that allows the user to select the target language.
The Result Page - This page displays the original lyrics of the recognized song along with the translated lyrics. Other details such as the artist's name and album art are also displayed.

How we built it

Frontend: We used Next.js and TailwindCSS to build the frontend of the application. Using TailwindCSS allowed us to easily implement a dark mode for the website and also allowed us to create a neat frontend for our application. The front end was deployed using Google App Engine.
Backend: The backend of our application uses the following Google Cloud APIs - Cloud Translation and Speech to Text. The backend runs on Google Cloud Functions and we also use Google Cloud Storage and Google Cloud Firestore to handle the data. When the recording or the song is uploaded from the front end to the cloud storage bucket, the cloud function is triggered and the speech to text API is used to convert the audio recording into text. We use the 'lyrics.ovh' API to get the song name and full lyrics. Once we get the original lyrics of the song, the Google Cloud Translation API is used to convert the lyrics to the target language. When the translation is done, the translated lyrics, original lyrics, and other metadata are written to the Cloud Firestore database and can then be written from the frontend.

Challenges we ran into

Some of the challenges that we ran into were as follows -

Speech to Text: We weren't able to find a free music recognition API and so we had to use the speech to text API to convert the incoming audio into text. Some of the drawbacks of this were that sometimes the speech-to-text does not work when the background music is loud and we had to use an acapella to manage this issue.
Cloud Firestore: We noticed that because of the size of the lyrics, it took a long time to write the data into cloud firestore and this increased the execution time of our cloud function.

Accomplishments that we're proud of

What we're most proud of is how the backend works. It was quite a challenge to make all these APIs work together and we did not know much about these APIs before. We are also proud of how the UI looks especially with Dark Mode.

What we learned

We were able to learn a lot about Google Cloud and its APIs through this project. We also learned more about frontend development and serverless cloud functions. We also learned more about teamwork and what it takes to build such a project in such a short timeframe.

What's next for Detec-Tune

We would like to improve the music recognition capabilities of the app by using a better API like Shazam or MusixMatch. Moreover, we could also make some changes to the front end of the application to make it more accessible and improve the overall user experience.