Vicuna Mafia

Inspiration

LLM security is an emerging field. People are trusting LLMs more and more with private data. Security is more important than ever.

What it does

It's a research teaser for the possibilities on chat models, understanding hallucinations, and instruction-based secrets.

How we built it

We used vllm as an openai API server for inference and some code to glue a bunch of vicunas together.

Challenges we ran into

Inference is a pain when managing multiple contexts in a multithreaded environment. We have to save each Vicuna's context to prevent information from bleeding into the context of the other vicunas.

Accomplishments that we're proud of

The Vicunas start a blame war and vote each other out.

What we learned

Vicuna 7B seems to have a good time playing mafia

What's next for Vicuna Mafia

More roles. Bigger models, RL training for agents that win and lose. Once the model equals the random chance of losing by being voted out, you know the probability of it exposing secrets is decreased.

Built With

• huggingface
• openai
• vllm