Inspiration
Studying at a coffee shop has its perks - the good coffee, the ambiance, and the tranquility - but perhaps the most overlooked feature is the music. The relaxing instrumentals allow anyone to enjoy their stay or focus on their work. We wanted to emulate this vibe on our website while we read books, especially since the pandemic has restricted us from having frequent outdoor activities. But we also wanted to make our reading experience similar to watching a movie. As the scenes progress in any film, the appropriate background music accompanies the acting to deliver and invoke the right emotions among the viewers. We figured that if we can customize the music that is played according to what page or part of the book the user is currently reading, then we can create a similar theater-like atmosphere for the readers to enjoy their book.
What it does
From the backend, the text from the online books is extracted and compartmentalized into groups of two to three paragraphs. These groups of paragraphs are then inputted into an NLP-based model, where the emotions that they portray are determined. The five main emotions we selected are: anger, surprise, happiness, sadness, and fear.
In the front end, the pages of every book consist of two or three paragraphs depending on how the book was initially compartmentalized. The background instrumental music is randomly selected based on the emotion determined, using a database of pre-selected songs. This selected song would play while the user is on the particular page, and as they move back or forward to another page, the music changes.
How I built it
We used HTML, CSS, and JavaScript for the front end, mainly using the Bootstrap framework. The backend was implemented using the Django framework in Python. The NLP model we used was already pre-trained, and our main database for music and books was CockroachDB. We did low-fi prototyping using Figma to figure out the best designs for our website.
Challenges I ran into
Since this was a project for HackMIT, and it was my first time participating in a hackathon, I was overwhelmed by the pressure to produce output quickly. It has also been a while since I coded, so getting back on track was a little bit of a struggle. I also had to learn how to implement features I haven't encountered before, like playing music on a website using JavaScript.
Accomplishments that I'm proud of
As one of the two frontend developers in our team, I was proud of being able to produce a simple yet efficient design that can host our idea very well. I am also proud of our team since we managed to create something we could all be proud of in less than 24 hours.
What I learned
I realized my shortcomings as a front-end developer and have been improving day by day. I also came to understand the inner workings of a hackathon, which is essential for my preparations for my next prospective hackathons.
What's next for Musical Library
We plan to implement our own NLP model for the texts in the book, and a separate machine learning model for the selection of music. We also need to conduct user studies and analysis, especially on how to smoothen the transition of music change between pages. We would also love to find ways to operate this idea on a mobile app.
Log in or sign up for Devpost to join the conversation.