GPT Rewind

The landing page where users submit their conversation history
Front slide
Time spent slide
Monthly activity slide
"Hardest Question" slide

Inspiration

As AI/LLM usage has increased over the past few years, we think that many people might be interested in how much they actually utilize LLMs. We wanted to present this in a fun and concise form, taking inspiration from Spotify Wrapped for people listening to music on their platform.

What it does

The application first asks you to upload your conversations which you can export in your LLM setting (we have steps to do this for ChatGPT). And that's all the user has to do! We then pass it through an custom built data compression and unsupervised learning pipeline to analyze the information to answer key questions such as usage per month, total hours, and the easiest/hardest questions., WITHOUT calling any external APIs. This is then displayed on the frontend, with a special narrator guiding you along.

How we built it

Using parallel processing, we stripped the unnecessary metadata from conversations.json, reformat the data (grouped by conversation and month), and remove all stopwords.
We then generated embedding for each message using GTE-Large model, optimizing GPU processing speed using large batch size, mixed precision, fast inference mode, and CPU-GPU desynchronization.
Next, we made various queries to the embeddings to reveal data insights using unsupervised learning algorithms such as HDBSCAN for clustering and JAKE! for keyword mining, again optimized for CPU parallelism using shared memory and thread synchronization.
Once we got the data from our model, we used React/NextJS on the frontend and FastAPI on the backend to display the information to the client, using Fish Audio API for generating the narration and asynchronous loading to provide a seamless user experience.

Challenges we ran into

Going through a large dataset (millions of tokens) efficiently. We were able to address by doing things in a smart order, designing efficient data structures, implementing state of the art algorithms such as HDBSCAN, and parallelism in both CPU & GPU.
Memory management for large dataset and parallelism. We were able to overcome this by again, ordering our calculations smartly and manual garbage collection within Python.
Integration between our various pipelines

Accomplishments that we're proud of

In the end, we were able to cut down our processing speed from 20mins to 2mins.
We did not use any external API or calling a LLM -> Everything was done locally, in-house.
Overcame all the problems we have came across and achieved a high level of accuracy for our data insights, according to the person who let us demoed on their data.

What we learned

We need to manage our expectations because unexpected problems occur all the time and we need to be able to ditch parts of our plan when needed. However, for the most important parts, we must persevere and find creative solutions.
We all rely on LLMs too much in our lives, without first considering alternatives. I (Austin) learned a lot about myself and my AI usage, and I've become much more cognizant of how much I use AI.