Communi Care

Opening Whatsapp
Tap controlled Commands
Automated Messaging

Inspiration

A close friend of ours suffered a severe injury that left him immobilized for a long period of time. During his recovery, even simple tasks like texting became a challenge for him. Seeing his struggles and feeling helpless, we were motivated to create a solution that could assist others in similar situations. This led us to develop an application designed to empower patients to perform basic tasks—such as messaging, streaming videos, navigating user interfaces—using subtle gestures like finger tapping, eye movements, and other minimal physical actions.

What it does

CommuniCare is a comprehensive assistive tool that tracks subtle physical movements like eye blinks, eye movements, and hand gestures to interact with digital systems. Users can:

Control a virtual keyboard or mouse.
Perform actions like making calls or sending messages via WhatsApp, streaming videos etc.
Navigate tutorials and access essential communication tools through an intuitive interface.

How we built it

We developed CommuniCare using:

OpenCV and MediaPipe to track real-time hand and face movements.
A custom-trained YOLO v8 model for hand-tapping recognition.
Tkinter for a user-friendly interface.
Selenium to integrate WhatsApp messaging, youtube video streamig functionality seamlessly and more.

Challenges we ran into

We faced multiple challenges involving the calibration of the application to be able to track and detect the various gestures and movements for the camera positioned at different angles and for the application to be used by different people since their eye movement speeds and distances had to be taken into account.

Accomplishments that we're proud of

Successfully implementing precise tracking of eye and hand movements.
Developing an accessible and user-friendly GUI tailored to the needs of individuals with mobility challenges.
Creating a robust system that bridges the gap between assistive technologies and everyday digital tools.

What we learned

We learned the importance of balancing accuracy and simplicity in assistive technologies and the hardships faced in collaborating to integrate various functions of codes or even files of codes.

What's next for Communi Care

We soon intend to work on this project to improve the accuracy and adaptibility of movement tracking using more advanced models. Another feature we thought of implementing but were not able to given the time constraints was to combine the gestures with voice-based commands.