IntegratedFlight

AI-assisted mission control for autonomous drone operations in contested environments

IntegratedFlight is a safety-first drone command and mission control platform that turns natural language into structured flight actions, validates those actions against mission constraints, and provides operator feedback through a real-time UI and voice assistant workflow.

What It Does

IntegratedFlight lets an operator issue plain-English commands like:

• “Alpha take off to 10 meters”
• “Fly to the northwest watch tower”
• “Drop to 3 meters and fly through the west gate”
• “Return to the landing pad and land” The system then:
• Parses command intent with an LLM-powered command brain (groq-integration)
• Maps natural language locations to canonical mission waypoints
• Applies hard safety checks (IFF + no-go zones + altitude constraints)
• Returns a validated execution result with confidence/risk context

5. Displays mission state in a live frontend and supports voice I/O

Why We Built It

Modern autonomous systems need an interface that is:

• fast for operators under pressure,
• understandable (natural language),
• and safe by design. We built IntegratedFlight to bridge natural command UX with deterministic mission constraints, so operators can command quickly without bypassing safety rules. --- ## Core Features
• Natural Language Command Parsing
  Converts spoken/text commands into structured drone actions.
• Challenge-Aware Waypoint Mapping
  Resolves terms like “northwest watch tower” to canonical IDs (e.g., TOWER_NW).
• Hard-Block Safety Enforcement
  Rejects commands violating no-go zones, invalid waypoint constraints, unsafe targets, or prohibited actions.
• Operator Confirmation Flow
  Handles ambiguity/high-risk commands with explicit confirm/cancel flow.
• Session Memory + Follow-Ups
  Supports context-aware follow-up commands (“do that again”, “same target”, etc.).
• Multi-Waypoint Tasking
  Supports ordered waypoint routes in one command for multi-step mission flows.
• Coordinate Verification (Telemetry-Ready)
  Includes execution verification hooks for live MAVLink telemetry and retry-aware status reporting. --- ## Architecture
• frontend/ – Operator UI (command log, mission status, interaction panels)
• whisper-backend/ – FastAPI backend for STT + command execution bridge
• groq-integration/ – LLM parser + validator + mission command brain
• challenge/ – Mission waypoints, no-go zones, scoring config

- mavsdk-app/ / sim stack – SITL/Gazebo integration context

Safety Model

IntegratedFlight enforces safety before execution:

• Unknown or invalid mission waypoints are blocked
• No-go zone incursions are blocked based on configured constraints
• Altitude violations at constrained waypoints are blocked
• Unsafe hostile/friendly targeting patterns are blocked or confirmation-gated

- Unsupported/invalid weaponized intent can be hard-rejected

How We Tested

• Unit-style command tests for parser + validator behaviors
• Manual scenario testing with challenge command sets
• End-to-end API/UX smoke tests (/api/execute-command)
• Error-path validation (missing telemetry, API-key failures, fallback messaging)

- Branch-based integration tests for UI/brain combinations

Tech Stack

• Python (FastAPI, validation pipeline, telemetry hooks)
• Groq API / LLM parsing
• JavaScript / React / Vite
• Whisper-based transcription backend

- MAVLink / SITL / Gazebo challenge environment

Challenges We Ran Into

• Keeping parser flexibility while enforcing deterministic safety
• Branch integration across evolving UI/brain variants
• Resolving path/runtime mismatches between backend launch contexts
• Handling telemetry-available vs telemetry-unavailable verification states

- Maintaining robust confirm/cancel behavior with natural spoken variants

Accomplishments

• Built a full NL-to-safety-validated command pipeline
• Added challenge-specific hard-block enforcement
• Implemented robust waypoint aliasing and route support
• Integrated branch workflows for modular UI/brain mixing

- Shipped operator-facing command logging and voice response flow

What’s Next

• Full live mission execution against SITL controls (not only validation)
• Richer mission planner for multi-drone concurrent tasking
• Better automatic recovery/replanning on verification failures
• Expanded observability (timeline replay + mission audit report)

- Production-grade auth and role-based command permissions

Team

IntegratedFlight was built by our team as a mission-focused, safety-aware autonomous operations platform for hackathon challenge environments.

Built With

• ardupilot
• groq
• mavsdk
• python
• react
• tailwind