Collalbum

An app that solves the struggle of "What's your music taste?"

Comment 1

Inspiration

Two years ago, one of our members, Sam Rynerson, created a collage of his favorite albums. When he posted and shared it online, his followers and friends were interested in the software he used to create it. This inspired our group, as we created this application to allow people to create collages of their favorite albums, to better express their interests in art and music.

What it does

In short, Collalbum lets people share parts of themselves and things about their personality by packaging music taste in an easy and aesthetic way. A lot of the things that people like to do are all about social media, and its useful to have an image to screenshot and upload to an Instagram story, a Twitter post, a Threads post, etc. allowing you to have this otherwise intangible part of your personality captured in an image. Technically speaking, we use the Spotify API to gather a user's most listened to albums, and arranges these images, by using both their color data and semantic data, determined by using a generative LLM AI model, LLaMa on Groq, and the Google Vertex AI platform. This returns a collage with aesthetic and semantic significance to a given user.

How we built it

We use LLaMa, hosted by Groq, to analyze the sentiment of specific album images and generate sentences describing them. We analyze this data with the Google Vertex AI platform to combine this data with the initial image data to create rich, descriptive image objects. Using PCA (principal component analysis, a data science technique) we project these images onto a rough grid in 2-Dimensions, where relevant albums are grouped together on the image. Our frontend is written in React, which is hosted on an ExpressJS/NodeJS server. All of our AI, data science analysis, and image generation logic is written in Python, making heavy use of matplotlib, sklearn, and our API calls to LLM and sentence vector embedding services.

Challenges we ran into

One challenge we ran into was determining how to expand the scope of the project. We initially planned a simple, concise, and achievable project given the limited time constraints of the Hackathon. However, we ran into challenges trying to add additional features while remaining true to the initial goal of the project of recreating human-crafted art.

Additionally, we ran into the issue of generating the collages in a way that wasn't just random or color based. It was challenging trying to figure out clever and meaningful ways to group the images in an aesthetically pleasing way.

Another challenge we ran into was hosting the project so it could be accessible to everyone. We struggled to host the project in a way that was dependable and reliable. We also ran into issues with setting up a valid domain to point to our hosting instance, and relying on global domain propogation.

Accomplishments that we're proud of

We are proud of our ability to successfully see the project through and push a production version out that anybody can use! We met the initial objectives of the project, which was to allow a user to generate a collage of their favorite albums in an easy, programmatic method. We saw a solid rough draft of the project through to completion.

We are proud of our solution for how to organize the collages in an artistic way- using lots of cool math! We used a combination of using "visual" embeddings of the album art images to 784 dimensional vectors and "semantic" embeddings of the images using Grok to get sentence representations and Google Vertex AI to get additional 784 vector embeddings of the "semantic" meaning of each image. Then combining these vector representations for each image got us 1568 dimensional vectors, which we reduced down to 2 dimensions using PCA and then snapped to our messy-grid shape to produce a final result where images are grouped based on general color shade etc. but also by semantic meaning like "masculinity", "femininity", "human-ness" etc. (these are our own subjective interpretations- interpret the collages however you like! The closer any two images are in the collage, the closer they are semantically and visually. Placements in the collage are chosen by with very great care!

We're also proud of using caching, rate limiting, parallelism, batches, and other tricks to get api response times super low from the python collage creation server!

What we learned

We learned significantly more about how to see a project through to completion, from planning to the final product. We had to deal with the challenges of planning, integration, testing, hosting, and every step in between when completing this project.

What's next for Collalbum

For the next steps, we would continue to develop the image generation abilities and customizability. A user could potentially further develop their image, including the ability to select the number of images, the size of the final image, the quality of the image, and other configuration options so they can better control their final collage. Additionally, there is the potential to allow a user to choose a background image that they could use to manipulate the layout of their collage to fit a specific scene, instead of developing a best fit gradient.

Share this project:

Updates