HI-Security

This is a advanced camera system which utilizes machine vision and machine learning to track and identify faces. The handy ui also enables users to track the system remotely and in real time.

Comment

ACS-HTN2019

HI-Security is an accessible, yet powerful firmware that provides tracking of intruders through machine learning and machine vision. Our algorithms are able to differentiate between familiar and unrecognized faces and adjust the camera direction accordingly. The accompanying website provides real-time tracking of the camera system. It is able to stream audio, view suspicious snapshots, as well as the transcript from the camera's microphone.

Inspiration

Often, security is taken as a granted. We don't want to think of the possibility of our house being robbed or our car stolen, yet there lies real risk in our negligence. Our project attempts to bridge that gap, utilizing the latest IOT, machine learning, and machine vision algorithms to protect people's privacy while being cheap and simple to the end-user in the process.

Technical details

The program uses the OpenCV DNN library to accomplish facial detection and recognition in real-time using a custom-built recognizer trained using standard ML languages including Caffe and Sklearn. To track the person on camera, the application uses body center estimation to best guess where a person's geometric center is.

One of the more powerful aspects of the image recognition software is that it is able to detect the center of the suspect in the screen. Due to the limitations of some camera positions, our camera mount is able to adjust for this. It consists of two stepper motors controlled by Arduino, which attempt to center the suspect according to the center detected by the algorithms.

In the front end, we have a fully functional website, which users can sign in and view their security system live. It was built from the JavaScript library React. We have various features available to the end-user. Firstly, is a video stream from the camera, enabled through Flask. This is where users can view the recognized faces and names in action. Frames with unfamiliar faces are saved and uploaded to Azure cloud storage, where we could potentially do more image processing with multiple users. Users can also view these frames on their login session. Lastly, the camera uses its built-in camera to utilize Google speech recognition API to display a live transcript of recorded audio on the website.

Challenges and Lessons Learned

Even though everyone in our group was already experienced with Hackathons and coding, we still lacked some experience in dealing with more technical projects. The goal we set for ourselves was a difficult challenge for all of our group members, and each of us learned a great deal in the end. Firstly, we divided our work into relatively independent chunks, so we had to communicate effectively with each other, to make sure that no one was going off track. It was also a challenge to put our parts together, and we had to decide which compromises to make, as well as seriously test our VCS skills.

Individually, we each encountered quite a few challenges. One of the most significant was the implementation of motor camera tracking. Our initial implementation was in Arduino, where we had multiple problems. It was extremely hard to debug due to there being no interface, so we had to build hardware-based debugging such as LEDs. The libraries we used also lacked support, which compounded the problem. In the end, we realized that we had too many sources of error, so we used the Py-Serial library and used serial communication instead. What we learned was that it was okay to start over if things are getting difficult, and the ease of finding errors is a vastly underrated value.

Share this project:

Updates