Inspiration
We saw how developers spend countless hours debugging, often combing through test cases to find where things went wrong. The idea of BugBasher was born from the desire to streamline this process, harnessing the power of AI to transform debugging from a daunting task into a more manageable, even enjoyable, part of development.
What it does
BugBasher uses LLMs to analyze your test cases, identify patterns in code failures, and rewrite the code to hopefully remove the bugs through an iterative debug and test process. It's not perfect, but it provides a starting point for debugging.
How we built it
We integrated large language models with a code analysis framework. Our LLM is CodeLlama 34B Q4 running locally on a self-hosted remote server securely located 100km away. Our LLM runs in the text-generation-ui hosting layer with an api endpoint connecting to our python script. The most important part is to seamlessly blend mixed code and text input/output tokens from an LLM with executable code which runs via the test suite.
Challenges we ran into
Originally we aimed much higher. We wanted an LLM which used a vector database to store information about the codebase, and could self build a database which helps it learn how to debug that codebase most effectively. We were going to get the LLM to self-create bugs then learn to fix them as a way of training. Ultimately the mixed input led to it becoming far too complex and we had to go much simpler. Additionally the LLM itself was very challenging to get running efficiently on a local server, when it's 34 billion parameters!
Accomplishments that we're proud of
Getting the 34 billion CodeLlama model to run well, and for the test cases to run at all.
What we learned
LLMs, Codellama, Vram, executing python code internally.
What's next for BugBasher
Introduce the parts we had to let go. We want it to learn to debug not just debug!
Built With
- llama
- python
Log in or sign up for Devpost to join the conversation.