BlinkBot

Created by Shiv Narayan Rai and Sonit Sahoo

Check out our slideshow for more: https://docs.google.com/presentation/d/1FTwStcEBoFJFtj2pEcHFB_8EXBNv53sVOYeXm62PuAw/edit?usp=sharing!

Inspiration

Our aim was to broaden the coverage of digital education. We thought of adding AI in aiding students with motor function disabilities, as students using standard interfaces oftentimes have a difficult experience.

What it does

With BlinkBot, one can perform mouse operations, such as movement and clicking, by simply blinking their eyes while voice commands are detected through sophisticated language algorithms. The tool allows for complete hands-free interaction with the computer.

How we built it

We used OpenCV and MediaPipe to handle real-time facial landmark tracking through a webcam. Eye movement and blinking control the mouse, while mouth opening triggers a voice input session via speech recognition. The GUI was built using Tkinter to provide sensitivity controls and live voice feedback, integrating all components into one seamless user experience.

Challenges we ran into

Fine-tuning the blink detection algorithm to avoid accidental clicks was tricky, especially under different lighting conditions. We also had to design a reliable way to detect mouth movement without confusing it with other facial motions. Ensuring smooth and responsive voice input without using heavy models or external triggers was another key challenge.

Accomplishments that we're proud of

We created a fully functional, hands-free computer control interface using only a webcam and microphone. The app combines eye tracking, blinking, and voice typing into an intuitive experience. It's lightweight, works in real time, and could significantly improve digital access for users with physical limitations.

What we learned

We learned how to combine computer vision, voice processing, and AI into a single workflow. We gained deeper insights into accessibility challenges. We also learned the importance of designing with real users in mind.

What's next for BlinkBot

We're planning to add support for gesture-based scrolling, better multi-monitor tracking, and more customizable commands. Integration with platforms like Google Docs and Classroom is also on the roadmap. Eventually, we aim to make this toolkit usable across platforms, including mobile and tablet environments.