dev-gpt

Inspiration

The primary inspiration came from the Generative Agents paper. https://arxiv.org/abs/2304.03442

TLDR: It's a simulation world where agents plan, interact, and have character traits.

I wanted to create a software developer Generative Agent that grows to understand you and be a coding buddy. Theoretically, it can develop code, be a maintainer and talk to coworkers.

I wanted it to be an organization with multiple interesting, diverse Agents that you can drop anywhere.

And then, recently, gpt-engineer (https://github.com/AntonOsika/gpt-engineer) was released. I was really surprised at how well it did. And it convinced me that this is not a farfetched idea.

In my opinion, UX is a key issue for AI adoption. And I think, with gpt-engineers, it'd make the most sense if there was not a UI. It should integrate seemlessly, as if it was another (human) developer.

What it does

So I had to settle on a simpler project to create:

• A conversational agent to gather requirements.
• A Github Issues plan and execute agent
• A coder plan and execute agent that submits a merge request

There are humans in the loop with this approach (I can create a ticket and give it to an agent), and the agents can get dropped onto any project.

How I built it

A lot of Langchain, and ChatGPT Functions Agent. I also used Pinecone to backup summaries of conversations, and allow agents to search the history to create more meaningful tickets. Also, the repository that the agents work on is indexed with pinecone and using HyDE you can search for code files semantically (https://arxiv.org/abs/2212.10496). This lets agents understand what to write to.

Why is Pinecone needed?

Pinecone backs up the conversation agent's memory. Pinecone also enables code file search on a repository with hypothetical document embeddings. It could also allow for agents to read documentation in the future. It could store the whole universe of agents interactions.

Challenges we ran into

Initially, getting familiar with Langchain was a challenge (there's still so much to learn!). I got lost exploring all the capabilities.

A really difficult challenge was getting ChatGPT-Functions to use the Tools provided. Especially the write file tool. It gave a scripted response saying that it was not allowed to write to disk. Possibly a result of its training.

The solution was to rename the tool and description.

Accomplishments that we're proud of

It works pretty consistently end-to-end. I'm also incredibly happy to take part in this competition and finish, and to have learned so much in the past few days.

What we learned

Prompt Engineering is incredibly difficult. AGI might be possible with GPT-4 and some clever systems around it, like memory and etc. We should be careful with our creations.

What's next for dev-gpt

Lots of UX work. First of all, probably a voice interface with the conversation agent. I do not want the user to interact with the repo. I think the ultimate goal is to have this agent be as present as a remote software developer, and exist in the cloud, messaging github issues, reviewing pull requests, etc.

Related to UX work, I found the Generative Agent in langchain's api docs under experimental, but at that point I was committed to what I already had. It could give the agent a lot more personality. Maybe it can brighten the day of many software devs.

Hopefully, this can become a full time job. Otherwise it's a nice hobby.

Built With

• chatgpt
• github
• langchain
• pinecone