C.H.E.R. - Comprehensive Human Emotion Recognition

The form parents fill out when using C.H.E.R.
Logo for our project
How the form data is stored with Firebase

Inspiration

One of the brightest minds we know (who shall remain unnamed), has lived through a plethora of experiences that we can only dream of. One of these experiences, however, is raising a child with diabetes. We took inspiration from how he circumvents the stress of asking his child to read his sugar levels every few minutes. He has software that will do the monitoring itself and will send him an alert the instance a low blood-sugar level is detected. Seeing the potential and using this methodology of a real-time monitoring and alert system, we set out to play our parts in the solution to yet another problem society faces: Suicide.

What it does

C.H.E.R. is a cutting-edge Artificial Intelligence Model that has been trained using over 35,000 images of human emotion. Integrating C.H.E.R. with Computer Vision, we equipped her with the ability to detect the real-time emotions of her subject. This alone, is just the tip of the iceberg. Utilizing data storage techniques and our knowledge of front-end development, we created the ability for the guardians of C.H.E.R.’s subjects to receive updates on their child’s mental health in real time. By acting as an intelligent intermediary, C.H.E.R. eliminates parental stress by minimizing consequences caused by a lack of communication between the parents and the child, bolstering the relationships children have with their parents.

How we built it

Starting this project, we wanted to work more with AI. Our previous experiences led us to choose Python as our primary programming language. Since we didn’t have the capabilities to make our data, we found datasets from Kaggle to train our model. The actual training itself was done with the help of TensorFlow’s Keras library. Once we were satisfied with our model, we needed a way to use it with our cameras. To do so, we integrated Computer Vision into our project, making an integral connection. Now we needed a way for others to use C.H.E.R. We found that making a UI with react served as the best method. Inputs from the application would be sent to our database hosted by Firebase, to configure C.H.E.R., tailoring it to be applicable for every use.

Challenges we ran into

When it came to working as a team, we never realized how important it is to properly communicate with each other and plan accordingly. This became apparent when countless hours were wasted working on the same task or trying to fix merge conflicts on our GitHub, all of which could have been avoided with better coordination. Once we took time to sit down and organize in advance, our GitHub issue died, but new technical problems arose. We soon realized that integrating multiple technologies into one project requires a lot more finesse and trial and error than we expected. These challenges, once again, could be fixed with some planning.

Accomplishments that we're proud of

To this date, every AI model that we have created has either used numerical data or text inputs – we have never built one to work with images. This in itself was an enormous accomplishment as we used our previous experience to set foot into a completely unknown realm. However, this was just the beginning. Soon after, we needed to find a way to test this model on completely new data: ourselves. To do so we successfully learned our way around a completely new technology, Open CV, within a day. Most importantly, however, we are the most proud about being able to finally make a commit without any errors.

What we learned

As mentioned above, the entirety of our experience with AI has always been through text-based models. This was the first time we had ever attempted to work with images, let alone a live camera feed that successfully tracks our emotions. Not only did this hackathon help us learn how to use Computer Vision, but we also learned to do it while effectively connecting multiple technologies. This experience has significantly broadened our understanding of AI, laying the groundwork for future projects like this one that have the potential to make an impact on lives worldwide.

What's next for C.H.E.R. - Comprehensive Human Emotion Recognition

Even though we are done with the hackathon, we aren’t even close to finishing C.H.E.R. There is still so much room for improvement, and we intend to make our model as accurate as possible. We’ll start by improving our training program, as well as increasing the quantity and variety of our dataset. After all this, we still won’t be satisfied. Rather than sticking to solely emotional recognition, we plan to make C.H.E.R. even more powerful by implementing human, as well as object detection in real-time. Our vision is for C.H.E.R. to become an efficient, reliable, AI, one that will be used by professionals in the real world.