Pyro

Fast crisis response using artificially intelligent drones to detect fire and guide first responders in real time.

Comment

Inspiration

The wail of sirens often comes too late. Fires spread silently, smoke suffocates before it can be seen, and by the time first responders arrive, precious minutes have already been lost. This is especially the case for rural areas in the world, the countryside, and third world countries. Delayed emergency response is not just an inconvenience; it is the difference between life and death, between a home saved and a community devastated. Innovation is needed. We were inspired by the urgency of this problem: how can we harness technology to give responders an edge, cutting down their arrival times and providing them with the knowledge they need before they even set foot on the scene? The future looks dim...

What it does

We are proud to present… Pyro. Pyro is an intelligent drone-based emergency response system that accelerates crisis detection and response. By utilizing drones capable of detecting smoke, fire, and people in real time, Pyro provides first responders with instant aerial insights, allowing them to arrive prepared and significantly faster than human scouts. With advanced AI analysis, Pyro generates structured reports highlighting the presence and location of hazards, empowering responders to act with clarity and confidence when every second counts, proving the light at the end of the tunnel.

How we built it

Communication: Using the Twilio API, users can call a phone number to trigger drone deployment. The call sends an HTTP request to our server, which initiates the drone’s scanning routine.

Drone Deployment: For demo purposes, Pyro scans a small square grid, but the system is highly scalable to cover larger zones.

Object Detection: At the core is a YOLOv8 model. While pretrained for person and object detection, we expanded its abilities by finetuning on the D-Fire dataset (21k labeled images of smoke and fire). This enables highly reliable hazard recognition.

Image Analysis: Drone frames, with bounding boxes of detected hazards, are parsed by the LLaVA model, which generates human-readable summaries and detailed scene reports.

Frontend Integration: Built with Next.js and Tailwind CSS for fast, clean UI. We used the Leaflet API to display real-time drone geolocation on an interactive map. Reports from LLaVA appear directly in the frontend, giving responders a clear overview of the crisis in progress.

Challenges we ran into

We encountered challenges in:

Optimizing the drone’s scanning path for maximum coverage while maintaining low response times.

Achieving a balance between YOLOv8’s accuracy and real-time inference speed.

Seamlessly integrating Twilio’s phone-triggered API with drone commands.

Making sure LLaVA’s reports were concise and structured enough for emergency responders.

Handling synchronization between the drone’s telemetry and the Leaflet map without lag.

Accomplishments that we're proud of

Successfully integrating real-time drone control, object detection, and AI reporting into one system.

Finetuning YOLOv8 on a massive smoke/fire dataset to expand its arsenal.

Creating an intuitive frontend that shows both the live drone location and LLaVA’s interpreted reports side by side.

Proving that Pyro can realistically reduce emergency response times by up to 80%.

What we learned

How to fine-tune state-of-the-art computer vision models for specialized domains.

Best practices for integrating multi-model AI pipelines (YOLOv8 → LLaVA → frontend reports).

How to leverage the Leaflet API for smooth real-time drone geolocation tracking.

New technical skills in Next.js, Tailwind, and real-time API orchestration.

The importance of designing outputs not just for accuracy, but for clarity and usability in high-stakes scenarios.

What's next for Pyro

Scaling scanning patterns to handle large crisis zones like wildfires or industrial disasters.

Adding more detection capabilities, including hazardous materials, collapsed structures, and injured individuals.

Integrating with emergency dispatch systems for instant deployment without a manual call trigger.

Adding thermal imaging for night-time or low-visibility detection.

Partnering with fire departments and first responders to take Pyro from hackathon prototype to field-ready deployment.

Built With

Share this project:

Updates