Bare Bones D&D

Final Product
Final Product
Product before outer aesthetics
Attaching animatronic skull
Product Internals
Individual system components

A dungeon master in their realm

Inspiration

Dungeons and Dragons (D&D) is a fantasy roleplaying game where players take up larger-than-life characters to delve into dark dungeons and best legendary foes. It has fostered an extremely devoted and diverse community of players in its many years of existence. The dungeon master (DM) runs the show, piloting all the enemies and describing how the characters interact with the world around them. However, the DM can sometimes be a thankless role as they devote hours of prep for each session and do not get to play their own characters. This motivated the idea of a product that could supplement the role of DM for dedicated D&D players who want a break from DMing.

What it does

Bare Bones uses AI (Arcane Intelligence) to understand what players are saying and keep track of character, enemy health, and stats in addition to planning out the individual encounters that compose full-scale adventures. Players take turns saying what they want their character to do and the skull narrates what happens. If a skill check is required the skull will ask the player to roll a dice and then explain what happens as a result. Behind the magic, the project converts user speech into text, which is then fed into an OpenAI API call along with other information to direct the response. The response is then received and processed before it is converted to audio and told back to the participants.

A bunch of dwarves working in a forge

How we forged it

We started with the software concepts at the core of the project and tackled text-to-speech, speech-to-text, and AI narration generation first. From there we created the necessary hardware features to give this magical software a physical form by adding a speaker and a microphone. Finally, the skull was crafted with laser cutting and 3D printing before being coaxed to life with LEDs and a servo.

A monster disguised as a chest

Constitution checks and challenges

The core microcontroller in our system is a Raspberry Pi, which does not have an ADC. We tried many different methods to communicate an audio microphone signal to the Pi, from using an ADC IC to using a Teensy microcontroller and finally using a Raspberry Pi Pico. The Pico simulates a USB microphone, which is easy to integrate with the Raspberry Pi. We also encountered issues trying to juggle all the animation tasks of the skull while running the actual game. This was overcome by multithreading, allowing the product to perform many functions seemingly simultaneously.

I won at DnD!

Our natural 20s and accomplishments

As soon as the skull jumped off of the print bed it lept into our hearts (and maybe nightmares). It brought an amazing amount of animation to the project and we love the result. The quality of the microphone signal is another point of pride due in part to the difficulty with which it was achieved. This high-quality signal is key in making sure that the skull understands what the players are saying using speech-to-text.

How we leveled up

We learned that before diving head first into solving an issue, it may be beneficial to ask others for ideas and try and look at alternative routes. We spent a lot of time on adding the microphone, and a lot of it ended up not being applicable as we went a different route with the actual application. We also learned a lot about using different types of microcontrollers like the Raspberry Pi and Raspberry Pico, neither of which we had great familiarity with.

This is also the first time that many of us tried to program a microcontroller in Python and we were excited with how quickly we were able to implement features. Things like internet connectivity and text-to-speech would be extremely difficult to implement in C++, but they were much more manageable in Python.

The fellowship of the ring marching on

Onto the next adventure!

While the AI we used can create engaging scenarios, it is not capable of creating campaigns that would span multiple sessions to tell a story of truly epic proportions. We want to expand its ability to create larger stories in this way because we believe this is part of what makes D&D such an engaging and community-building activity.