MeloMoods

MeloMoods

Inspiration

We've always had issues with the Spotify recommendation system. Sometimes, the songs don't match our mood. Other times, Spotify keeps recommending the same old songs, but never something new. That's why we decided to make MeloMoods, ensuring your Spotify melodies match your mood!

What it does

MeloMoods recommends Spotify songs based on your mood. You can either specify your emotions in Manual Mode or get AI to sense your mood using facial recognition, updating your playlist live with new recommendations according to your feelings every 10 seconds.

How we built it

Powered by 278k emotion-labelled songs from the Kaggle Moodify database, MeloMoods matches songs to your mood based on the energy level and valence (positivity) of the songs. Face detection uses pre-trained models from face-api.js while the custom detection model was trained on a database of 15k+ images using PyTorch.

Challenges we ran into

Deciding how to recommend songs for each emotion (e.g., which values for energy, valence, etc should be used)
Preprocessing the image from the camera to feed into the emotion detection: at first, we compressed the image dimensions which made it harder to detect sad / angry. It also output happy too often. This was because compressing the image horizontally ruined the aspect ratio and affected model classification. This was fixed by cropping the image instead.
Ensuring that songs are not repetitive: we fixed this by adding a set, map, and queue to track the last 10 played songs to prevent repeated songs.
The Moodify database has 278k songs on a .csv file which may be slow for bulk reading: we decided to transfer it to a PostgreSQL database (Neon) instead for faster reads and optimized performance.

Accomplishments that we're proud of

Having a user friendly experience that allows user control (Manual Mode) and also an automated experience (Auto Mode with AI)
Training our own AI face detection model
Integrating with Spotify for a non-stop music experience