deltawash

deltawash
picam rig
poster
poster guts

Inspiration

One time my dentist handled the tools from the previous patient, then only quickly rinsed their hands before operating on me. I was worried whether their hands were completely clean. Turns out a vast majority of people including healthcare workers do not wash their hands in the correct manner, so we decided to bring change to how people wash their hands. In this project, we use computer vision to check if all areas of the hand are thoroughly cleaned, according to the procedure recommended by the WHO.

What it does

Our product checks that the complete WHO protocol is followed to ensure a clean environment in healthcare locations. As you wash your hands, there will be visual feedback if you are following the steps correctly. A blinking green LED will signify which step it detects and a solid green LED signifies that the step has been completed. A blinking LED will only become solid after the step has been done for a sufficient duration. After each wash, the overall performance is uploaded to the website deltawash.tech. deltawash.tech serves as a dashboard to evaluate compliance across different units and to identify commonly missed steps.

How we built it

On the hardware side, a Pi cam is used to take in video of the person washing their hands, and the feed is then sent to the Pi. Once the Pi processes what step it identifies, it sends the information through Wifi to an esp8266 which controls the respective LED. Once the whole hand washing process is finished the Pi sends the information what steps were successfully completed to the website via Wifi.

On the software side, we built a model ensemble with a CNN and LSTM trained on labeled videos of the hand washing steps. This model is run locally on the Pi and all data is processed offline.

Challenges we ran into

Each model we built and tested had its own advantages and disadvantages, and as a team, we needed to decide what model to settle on. Each model was individually best at specific steps, so singling out what we prioritized most was difficult. Through using model ensembles and iterating, we combined as many good traits as possible to arrive at our final product.

Accomplishments that we're proud of

This is the first time our team has worked with a Raspberry Pi. It had a much larger learning curve than microprocessors such as esps and Arduinos. Despite this, we managed to get it working with minimal issues.