Inspiration
With a sharp increase in gun violence in educational institutions across the United States over the past 2 decades, we were deeply interested in creating an emergency response solution that is both highly functional and economically viable.
What it does
We are an end-to-end emergency response system for gun violence in schools, that requires only the mobile phones of students and teachers, and the existing CCTV camera systems to provide accurate emergency alerts and routing. There are 3 core components to our system.
1. Location mapping of mobile phones and cameras
Our mobile app asks every student of the school to input their weekly schedule. This means that we know which room every student is in at a given time. Moreover, the room number of each CCTV camera is stored in our database. Every device in our network is accurately located.
2. Machine learning to detect gun violence
Our ultra-light audio classification model which detects gunshot audio in the surroundings using the phone's microphone along with our object detection model that uses CCTV footage, we alert the system on the perpetrator's location based on which room the given phone/CCTV is located in.
3. Responsive mobile dashboard and SMS alerts
We use MapKit and a custom floor plan of the VCU Engineering building to highlight rooms that have potential threats based on our detection system. Everyone in the school gets instant alerts that contain accurate locations as soon as a threat is detected. SMS alerts to students and emergency responders add a layer of safety.
How we built it
- Swift UI and MapKit for the iOS app
- SQLite, Flask, Python, and Twilio API for the server
- Apple Create ML and Core ML for the custom audio classification model
- Roboflow and Python for the object detection model
Challenges we ran into
- Getting all our platforms integrated in < 24 hours
- Problems with real-time data rendering and MapKit
Accomplishments that we're proud of
- Integrating our custom audio classification model into the Swift app
- Building a custom object detection model for CCTV camera footage
- Creating an accurate floor plan of the VCU Engineering building
What we learned
- Creating custom CoreML models using Apple Create ML
- Creating custom object detection models with Roboflow
- Using custom annotations with SwiftUI and Apple Maps
What's next for Realert
- Support for Android devices
- More sophisticated authentication for accounts
Log in or sign up for Devpost to join the conversation.