CovidEye

AboutUs Page
Face Touching
Pose Detection Base

DEMO WITHOUT PRESENTATION

this app would typically be running in a public space

demo without presentation (judges please watch the demo with the presentation)

Inspiration

We spent hours thinking about what to create for our hackathon submission. Every idea that we had already existed. These first hours went by quickly and our hopes of finding an idea that we loved were dwindling. The idea that eventually became CovidEye started as an app that would run in the background of your phone and track the type and amount of coughs throughout the day, however we discovered a successful app that already does this. About an hour after this idea was pitched @Green-Robot-Dev-Studios (Nick) pitched a variation of this app that would run on a security camera or in the web and track the coughs of people in stores (anonymously). A light bulb immediately lit over all of our heads as this would help prevent covid-19 outbreaks, collect data, and is accessible to everyone (it can run on your laptop as opposed to a security camera).

What it does

CovidEye tracks a tally of coughs and face touches live and graphs it for you.CovidEye allows you to pass in any video feed to monitor for COVID-19 symptoms within the area covered by the camera. The app monitors the feed for anyone that coughs or touches their face. _For demoing purposes, we are using a webcam, but this could easily be replaced with a security camera. Our logic can even handle multiple events by different people simultaneously. _

How we built it

We used an AI called PoseNet built by Tensorflow. The data outputted by this AI is passed through through some clever detection logic. Also, this data can be passed on to the government as an indicator of where symptomatic people are going. We used Firebase as the backend to persist the tally count. We created a simple A.P.I. to connect Firebase and our ReactJS frontend.

Challenges we ran into

We spent about 3 hours connecting the AI count to Firebase and patching it into the react state.
Tweaking the pose detection logic took a lot of trial and error
Deploying a built react app (we had never done that before and had a lot of difficulty resulting in the need to change code within our application)
Optimizing the A.I. garbage collection (chrome would freeze)
Optimizing the graph (Too much for chrome to handle with the local A.I.)

Accomplishments that we're proud of

All 3 of us We are very proud that we thought of and built something that could really make a difference in this time of COVID-19, directly and with statistics. We are also proud that this app is accessible to everyone as many small businesses are not able to afford security cameras.
@Alex-Walsh (Alex) I've never touched any form of A.I/M.L. before so this was a massive learning experience for me. I'm also proud to have competed in my first hackathon.
@Green-Robot-Dev-Studios (Nick) I'm very proud that we were able to create an A.I. as accurate as it in is the time frame
@Khalid Filali (Khalid) I'm proud to have pushed my ReactJS skills to the next level and competed in my first hackathon.

What we learned

Posenet
ChartJS
A.I. basics
ReactJS Hooks

What's next for CovidEye

-Refining : with a more enhanced dataset our accuracy would greatly increase

Solace PubSub, we didn't have enough time but we wanted to create live notifications that would go to multiple people when there is excessive coughing.
Individual Tally's instead of 1 tally for each person (we didn't have enough time)
Accounts (we didn't have enough time)