ArcRescue

ArcRescue is a comprehensive workflow that enables natural disaster rescue missions by leveraging drone imagery and Esri technology to detect and locate people in real time.

Comment 1

Inspiration

Nearly 50,000 people lose their lives each year due to natural disasters, and one of the leading causes is a lack of timely response. Search and rescue teams refer to the first hour after a disaster as the "golden hour" a critical window where rapid intervention can mean the difference between life and death. At the same time, the drone technology company DJI has recorded more than 1,000 people rescued by drones globally. After the recent floods in Kerr County, Texas and the wildfires in Los Angeles, California, we asked ourselves the question of wether the combination of drone and Esri technology can be leveraged to aid first responders. ArcRescue was born from this question, with the goal of leveraging modern technology to speed up rescue efforts and save lives.

What it does

ArcRescue is a full end to end emergency response workflow that detects and locates missing people during natural disasters using real-time drone imagery and telemetry data, as well as Esri’s geospatial technology, all in real time. It is made up of three parts: ArcRescue-Flight,ArcRescue-Server, ArcRescue-FeatureLayer.

ArcRescue-Flight is a custom application that connects to the done. The application features a live FPV view of the drone, a map that shows where the drone is, a sidebar showing live telemetry data, and other critical drone information. When the user starts the stream, the drone sends a continuous stream of images and the corresponding telemetry data to ArcRescue-Server.

ArcRescue-Server The server first receives the payload from the drone. The server runs a deep learning model on the image in the payload in order to detect people. If the model detects a person, it then uses the telemetry data from the payload in order to calculate an approximate location of where that person is. It then sends a request to our ArcRescue-FeatureLayer.

ArcRescue-FeatureLayer The ArcRescue-FeatureLayer is a feature layer hosted in the ArcGIS environment. It plots points of people detected to a feature layer where rescuers can see the coordinates, number of people found in each image, as well as the image corresponding to the point, with bounding boxes.

How we built it

Flight- DJI IOS SDK, Swift Server- Python, YOLOv8x, Esri Python API Feature Layer- Esri

Challenges we ran into

Real time processing: Achieving processing of drone video while preserving accuracy was technically demanding. Data integration: Combining live drone feeds with Esri’s mapping tools required careful architecture and API handling. Telemetry syncing: Ensuring synchronized interpretation of telemetry and image data required significant troubleshooting.

DJI SDK: Using the DJI SDK to create a custom application with outdated documentation.

Accomplishments that we're proud of

Getting the computer vision model on the server to identify people, do the calculation, and send a request to the feature layer in real time.

Uploading the identified points and images to the feature layer in real time.

Setting up the custom DJI drone application, implementing functionality found in the SDK.

What we learned

The power of real time geospatial analysis for emergency response.

How to handle streaming data pipelines from IoT devices like drones.

Deepened our understanding of Esri’s Living Atlas and how it can enhance decision-making in critical moments.

What's next for ArcRescue

Model improvement: Enhance the accuracy and reliability of person detection under varying weather and terrain conditions.

Scaling infrastructure: Deploy to multiple drones and ensure scalability for large-scale disasters.

Partner with agencies: Work with emergency response teams to test ArcRescue in field conditions and gather feedback.

Offline capability: Add functionality for low-connectivity or offline environments.

Built With

Share this project:

Updates