Emotional Damage Helper (EDH)

Inspiration

Inspired by the conditions faced by many students (also the students currently participating in the hackathon). This hopes to improve their mood and their environment so they are better able to focus.

What it does

EDH uses a camera, temperature sensor, buzzer, and LCD display integrated all together. The camera records the user's facial expressions to determine their current mental state. Concurrently, the temperature/humidity sensor gathers data about the current environment of the user as well. Based on these two pieces of data, some suggestions may be provided to help the user improve their environment mood, or both. These suggestions would be shown on the LCD for the user to see.

How we built it

The individual components were integrated through the use of an Arduino. In addition, the Arduino is connected locally to a computer that has a webcam. Using computer vision and neural networks, the live video from the webcam was analyzed for information about the user's mental information. For the computer vision aspect, we utilized a media pipe model for face detection. As for neural networks, we found a pretrained model which we fine-tuned to match with our requirements to provide more accurate results.

Challenges we ran into

When integrating the components using the Arduino with the computer running the face detection progression, there were a lot of challenges faced when it came to timing the transmission of data between the devices properly.

Accomplishments that we're proud of

We were able to achieve a respectable level of accuracy and performance for the face detection software. In addition, integrating all of that with hardware as well was not a simple feat either and required significant time and effort.

What we learned

On the software side, we learned a lot about face detection and neural network fine tuning. In addition, on the hardware side, we learned about how to manage the timings of data transmission between the Arduino with the sensors and the computer with the web camera and face detection software to ensure accurate sending and receiving of data.

What's next for Emotional Damage Helper (EDH)

We would likely to be able to add more sensors relating to the environmental conditions so that there are a wider range of suggestions that are possible to help the user. For example, we want to add a light sensor that can inform the user to maybe let in some light to their room to improve their energy levels. In addition, we want to add more emotions to the emotion detector such as anger, exhaustion, and so on so that the suggestions can be more helpful.