baton / mace

Inspiration

I once got the opportunity to conduct a band, and that experience stuck with me. The feeling of controlling tempo, dynamics, and expression through simple gestures was fun and exciting. We wanted to recreate this feeling by turning physical motion into a digital conducting tool that anyone could use, even without a live orchestra in front of them.

What it does

Baton / Mace is a gesture-based digital conducting system. By moving a baton-like controller, the user can control tempo, cues, and expressive elements of music. The system tracks motion and orientation and translates those gestures into signals that can drive sound effects and change how the music is played.

How we built it

We built our project using three ESP32 microcontrollers:

One ESP32 embedded in the baton to capture motion data using onboard sensors (accelerometer and gyroscope)
A second ESP32 that controls the other baton that can produce special sound effects
A third ESP32 to interface with the output system (sound, software, or visual feedback)

The ESP32s communicate wirelessly through bluetooth, allowing low-latency gesture tracking and real-time responsiveness. We wrote firmware to process sensor data, filter noise, and interpret conducting patterns such as beats, pauses, and accents.

Challenges we ran into

Accurately detecting conducting gestures without false positives
Reducing latency in wireless communication between ESP32s
Implementing a BLE_GATT communication protocol.
Playback speed and audio file formats were different from expected frequency range
Proper wiring to the corresponding pins would change constantly, causing confusion.

Accomplishments that we're proud of

Successfully translating real-world conducting gestures into digital control signals
Achieving stable, low-latency communication between three ESP32 devices
Turning a personal inspiration into a functional prototype
Getting all the C++ code to function with our system and build properly.

What we learned

How to design and debug multi-device embedded systems
Practical experience with ESP32 networking and real-time data handling
The importance of filtering and smoothing sensor data
The power of friendship can lead to anything possible and that it's not about the destination, it's about the journey