reSolve

Entering your own resolutions
Generating resolutions similiar to the ones you enter, with GPT-J
Example of generated next steps, with GPT-3 175B

⭐ Inspiration

After 1 month, 35% of New Year’s resolutions fail. After 6 months, 54% of them do. (source)

Why? We (and many psychologists) think it’s because resolutions aren’t specific enough. People make resolutions without clear-cut and realistic goals, which often leads to failure. We created reSolve to help others effectively create and maintain their resolutions.

🎆 What it does

reSolve is a web application that uses deep learning to assist with creating and maintaining your New Year's resolutions. You input a few non-specific resolutions and reSolve will generate other similar resolutions to take on. Then, our deep learning algorithm plans specific next steps for each resolution, making it easier than ever to achieve your goal!

🛠️ How we built it

Large neural networks (transformer models) power the core functionality of our application.

When creating similar resolutions, our web app sends a request to our serverless NextJS backend. The backend queries GPT-J (a 6-billion parameter neural network) and then uses natural language processing to remove similar entries. To generate next steps, we process completions from OpenAI’s 175-billion parameter GPT-3 model.

We used few-shot learning paired with Top-p sampling to make the generated results relevant. We implemented the frontend using Typescript, NextJs, and React.

Flowchart showing the architecture of our app

🚧 Challenges we ran into

The GPT-J model was generating a lot of similar resolutions, so we had to use natural language processing (which neither of us have experience doing!) to filter those out
Fine-tuning the GPT-J model to generate New Year’s resolutions didn't work on our machine with 4 Nvidia 1080 Ti's and 64gb of RAM, so we had to use few-shot learning instead
- The 6-billion parameter model proved too big to train on, even with 8-bit weights (instead of the regular 32-bit)
- We were able to fine-tune the 125 million parameter model, but unfortunately that didn't give us the results we were looking for with our limited data

😊 Accomplishments that we're proud of

We are incredibly proud of having an app that can provide a coherent and insightful list of next-steps for your resolutions—generated by a computer!

🧐 What we learned

How a transformer model actually works
How to use the natural language processing to remove similar phrases
How to use the various machine learning as a service (MLaaS) services, such as OpenAI API and Neuro AI
How to fine-tune the GPT-J model with DeepSpeed

⏭️ What's next for reSolve

Fine tune GPT-J with data from Twitter users’ New Year's resolutions, enabling the app to generate resolutions from your Twitter posts
Implement a sign-in system so you can track your progress across devices