gitcommitters

Inspiration

Our inspiration stemmed from the idea of making web navigation more accessible and intuitive. We wanted to create a hands-free browsing experience that could be beneficial not just for convenience but also for accessibility. Whether someone has limited mobility, is multitasking, or simply wants a futuristic way to interact with the web, we saw an opportunity to build something innovative.

What it does

Our Chrome extension allows users to navigate the web using facial movements. By detecting nose position as a reference point, the extension identifies head movements—turning left or right to navigate between pages and tilting up or down to scroll. This hands-free interaction provides a new way to explore websites, making browsing seamless and interactive.

How we built it

We built the extension using face-api.js for facial landmark detection and manifest.json to integrate the necessary models. The core functionality relies on detecting facial movements using key points on the nose. By tracking the nose’s position, we determine directional movement and trigger corresponding keyboard events. This allows users to scroll and navigate pages without using their hands.

Challenges we ran into

Since this was our first hackathon, we faced several challenges along the way. Figuring out how to use APIs and integrate FaceAPI.js into a Chrome extension was a learning curve. Fine-tuning the thresholds for movement detection to ensure smooth and accurate navigation also required multiple iterations.

Another key challenge was working as a team—coordinating tasks, merging code efficiently, and managing different ideas in a limited time frame. Despite these challenges, we adapted, communicated, and pushed through, making this a valuable learning experience.

Accomplishments that we're proud of

A major next step is tracking eye movement to control the cursor—allowing users to move the pointer simply by looking in a direction. We also want to implement eye blinking as a selection mechanism, so users can click on elements without needing a mouse or touchpad. These improvements would bring us even closer to a fully hands-free browsing experience.

Future improvements will also focus on refining the movement detection algorithm and enhancing user experience with a polished interface. This is just the beginning, and we’re excited to see how far we can take this idea! 🚀

What we learned

Throughout this project, we gained hands-on experience with JavaScript, Chrome extension development, and face-api.js. We learned how to process facial landmarks, implement real-time detection, and fine-tune movement recognition for optimal performance. Additionally, we deepened our understanding of how to work with browser permissions and API integrations.

What's next for gitcommitters

We plan to expand this project beyond web navigation by integrating it with eBooks and PDFs, allowing users to turn pages hands-free. Additionally, we aim to incorporate speech recognition for an even more interactive experience. Future improvements will also focus on refining the movement detection algorithm and enhancing user experience with a polished interface.