Inspiration
Northstar C2 came from a simple idea: voice should feel like a natural control surface for drone operations, not a gimmick layered on top of a keyboard app. The challenge was compelling because it combined real-time speech, safety-critical decision making, simulation control, and product design in one system. We wanted to build something that felt less like a hackathon demo and more like an operator-facing tool you could imagine in a real control room.
What it does
Northstar C2 is a voice-driven drone command system for the Voice-Driven UxS challenge. It takes spoken or typed commands, converts them into structured flight actions, validates them against the compound layout and no-go zones, and then executes them in the provided drone simulation stack. At the same time, it gives the operator a live tactical map, telemetry, command history, and clear feedback on whether a command is executing, blocked, or needs clarification.
How we built it
We built Northstar as a full-stack system with a React and Next.js frontend and a FastAPI backend. On the backend, we created a challenge-native battlespace model based on the actual brief and challenge/config.py, including landmarks like the towers, rooftop, motor pool, barracks, and restricted zones around the fuel depot and communications tower. We wrote a deterministic parser that understands takeoff, altitude changes, landmark navigation, and relative movement, plus a safety layer that blocks invalid or unsafe actions.
For live simulation control, we integrated with the provided ArduPilot and Gazebo stack through pymavlink. That meant handling telemetry, GPS readiness, takeoff, landing, local position targeting, and reconnect behavior. On the frontend, we updated the UI to match the challenge instead of the earlier Alpha/Bravo demo framing, added challenge-specific sample commands, rendered restricted zones on the map, and exposed the drone’s command and telemetry state in a way that feels operational and easy to follow.
Challenges we ran into
The biggest challenge was bridging product polish with low-level sim reliability. Early on, the UI could appear to move correctly while the actual drone in Gazebo did not, which turned out to be a coordinate-frame mismatch between the challenge geometry, the sim world origin, and MAVLink local positioning. We also ran into classic simulation setup issues: stale Gazebo and SITL processes, port collisions, missing plugins, missing Python dependencies, and dev-server cache corruption in Next.js.
Another challenge was language handling. Human phrasing is messy, but drone control needs to be precise. We had to find the right balance between accepting natural commands and refusing to guess when a request was vague, unsupported, or unsafe.
Accomplishments that we're proud of
We’re proud that Northstar ended up being more than a voice-to-text demo. It became a complete operator loop: listen, interpret, validate, execute, and explain. We successfully pivoted the entire system away from the older Alpha/Bravo waypoint model into a challenge-native environment, aligned it with the actual brief, and made it work with the provided drone sim stack.
We’re also proud of the safety behavior. Northstar doesn’t just execute commands; it understands when to reject them. It blocks routes through the fuel depot, enforces altitude restrictions around the communications tower, rejects invalid actions like self-destruct or weapons requests, and asks for clarification instead of making unsafe assumptions.
What we learned
We learned that building a voice interface for robotics is as much about system design as it is about speech recognition. The hardest problems were not just “did the transcript sound right,” but “did the system understand intent correctly,” “is this safe,” and “does the simulation interpret the same coordinates we think it does.” We also learned how important observability is: clear telemetry, readable status messages, and visible command traces made debugging and operator trust much easier.
On the product side, we learned that judges and users respond strongly to polish. The map, the command lifecycle, the rejection messaging, and the overall presentation matter just as much as raw technical capability.
What's next for Northstar C2
The next step for Northstar C2 is making the voice interface even more natural and robust. We want to support more conversational phrasing, better recovery from ambiguous commands, and richer multi-step missions without sacrificing safety. We also want to improve path planning so the system can choose safer and smarter routes automatically instead of only validating direct ones.
Beyond the challenge, Northstar could grow into a broader voice-first mission control platform: multi-vehicle coordination, smarter autonomous behaviors, spoken status readbacks, and more advanced mission planning over real or simulated environments.
Log in or sign up for Devpost to join the conversation.