Empowerment Track
Inspiration
We were inspired to create this app inspired by many experiences we have had ourselves when we leave to complete an important task and completely forget to lock the door. Many of us can purchase a surveillance systems but the same cannot be said about those who are less fortunate, as these systems can cost hundreds of dollars. We instead wanted a quick, low-cost solution that could simply reuse a webcam and electronic device already being utilized in the house and provide similar functionalities. In addition, our system produces less e-waste and is a more eco-friendly alternative to Ring as it doesn't require batteries, whose chemical waste can wreak havoc on the environment.
What it does
Our application leverages state-of-the-art machine learning and computer vision models to provide a cheap yet effective home surveillance system for those who cannot afford expensive systems such as Ring, Nest, etc. The system consists of three parts: the webcam, web application, and the user's phone/device. The user first logs in and creates an account with our website, providing information such as their email, username, and phone number. The user is then prompted to set the facial recognition system up by uploading images if any friends and family, which are then used to train the model and stored in our backend to be used in the detection systems. The webcam is placed at the front of the house, where it constantly takes in video and deploys it as a stream on our Flask web application. We have three main streams, one for facial recognition, one for suspicious activity/weapon detection, and one for motion detection. If the webcam gets significant activity in any one of these streams, it then alerts the user through text messaging or email about whether there is suspicious activity, someone they know is at the door, or if motion was detected for a significant period of time. If the face at the door is unrecognized and the model is unable to recognize it for a certain time period, the web app takes a screenshot and sends it to the person via text/email.
How we built it
Python, specifically OpenCV and Tensorflow for machine learning and computer vision model development: all of the models are trained with 100s of images resulting in fast and accurate detection Flask for back-end development: used for streaming the data to the internet where it can be accessed by the accessory devices Ngrok for hosting out website and accessing it from phone JavaScript, HTML, CSS for front-end development MySQL for data storage: names, email The Twilio API for notifications and alerts: send alerts when the movement is detected
Challenges we ran into
Some technical difficulties we ran into programming our app included training the computer vision models, as these require extensive datasets and GPU cores to do quickly. We bypassed this by using Google Colab, which provides free GPUs. In addition, we had some difficulty running the streams in a fast manner, as the webcam captures lots of data every second. We tried improving this by making our models slightly more efficient and using flask ngrok, which hosts our streams. In addition, our stream was slowed down in some places due to wifi bandwidth being overloaded, as well as the fact that our computers lacked the power to display at a quicker FPS
Accomplishments that we're proud of
Learn how to use ngrok, and build multiple ensemble computer vision models that are able to accomplish different tasks.
What we learned
Learned how to use Twilio API, developed full-stack development skills
What's next for SmartBell
If we were to create a 2.0 version of the app, we would improve it by hosting the application on a faster cloud network, such as google cloud or Microsoft azure, improving our ensemble models by training with more data, and improving the relaying speed between models and phones. We would also move the web framework from flask to javascript, which is faster and more scalable. We would also make a Swift/Flutter correspondent for our application that would be used as a complement to our web application.
Log in or sign up for Devpost to join the conversation.