oto hana.

home page
authenticate user's spotify account
input user's mood
select song preferences
generate playlist
happy listening :)

Inspiration

Both of us love listening to Spotify but sometimes its annoying to think long of all the songs you want to add to your playlist. Maybe you want to find music that matches your current mood, without having to search through countless tracks yourself. This inspired us to create a platform leveraging the Gemini 2.0 Flash AI to do that work for us.

What it does

You log into your Spotify account, describe how you're feeling, and the application generates a corresponding mood profile. It then presents you with ten pre-selected songs based on that mood, allowing you to click to either add a song to your playlist or skip it. Gemini then analyzes your selections along with your mood description to fill the playlist and the remainder of the playlist with compatible songs.

How we built it

"oto hana" integrates Spotify's Web API with Gemini API on a Next.js/React foundation. We used Gemini to create a sentiment analysis pipeline that maps a user's feeling/description into a single mood. The system generates initial recommendations, then fine tunes the remaining selections from the user feedback based on the their initial selection of songs.

Challenges we ran into

The environment setup proved quite tedious, and we encountered some difficulties perfecting the UI design. We faced significant compatibility issues between different Node.js versions and the Spotify Web API, resulting in hours of debugging. The Spotify Web API also required us to have an SSL certificate, so setting it up for Spotify user authentication made local setup more tedious. Additionally, optimizing the application's performance while maintaining responsive design across devices led to a lot of troubleshooting and refactoring of our initial implementation.

Accomplishments that we're proud of

We successfully fine-tuned Gemini to generate songs based on moods derived from user descriptions, and effectively connected Spotify and Gemini APIs to work in unison with each other. The simplistic user flow was also something we were proud of, as we managed to create a simple experience that guides users seamlessly from mood description to playlist generation. Our prompt engineering efforts also very impressive in matching emotional states to musical selections, creating a more personalized playlists that consistently align with the users' described feelings.

What we learned

We learned that setting up a development environment properly is more challenging than anticipated. We also learned that sometimes simple is better for product and implementation. A database could have been implemented but was not necessary in order to have a successful product. We also learned how to work with endpoints through ngrok for local development testing with external APIs. The importance of prompt engineering became evident as we refined our approach to natural language processing. User testing was also very important because for simple prompts it worked very well, but for more complex prompts it showed that refinement was needed in enhancing our API calls.

What's next for oto hana

We plan to incorporate users' Spotify listening activity to further enhance personalization. By importing users' listening history, we can identify favorite songs and artists to incorporate into future playlists.