Chase Baxter, Calvin Sringam, Spencer Bush

At the start of HackGT we set out with the goal of creating a way for first responders to safely and accurately assess the threat level of hazardous interior enviroments, and by and large we achieved this. Our project consists of two major parts: a simulated robot exploring a potentially dangerous path through a room, and a convolution neural network model which analyzes the risk of said path through classification of images taken by the robot in the course of its exploration.

We first utilized MATLAB ROS and navigation toolboxes in conjunction with Webots to create a model enviroment that resembles a partially collapsed, inflamed, or otherwise deadly warehouse with an entry point and waypoint- in our case a person waiting for assistance. Webots allowed us to build a complex scene with many obstacles that the robot must be able to navigate, which was then simulated in order to generate a map, exported as a pgm file for MATLAB. This output fed into a MATLAB script that models a theoretical robot moving through the obstacles, and uses a rapidly expanding random tree (RRT) model to find a path to the end goal, in this case a disaster victim. Finally, this path is used for the final simulation seen in the video to guide the robot through potential hazards. Additionally, the robot takes pictures at each branch for a hazard analysis.

Using the images collected by the robot, we then determined if the robot passed any hazardous items, and if so how many and which types. We did this by feeding the image into a convolution neural network which classified each image to one of six different categories. Using a dataset of 500 images scraped mainly from facebook marketplace, we were able to achieve a greater than 60% accuracy in identifying the correct category of each image.

Because first responders will likely need relevant info in an easily accessible form at the scene, we also created a webapp which displays a map of the route the robot took, hazards it encountered along the way as classified by the neural network, and the location the robot was in when it captured each hazard. Our hope is that this will allow first responders to better assess the safest paths when navigating the building and reduce the danger of life-saving efforts in such an event.