Inspiration
Virtual formats show no sign of stopping in this post-pandemic era. Whether it be restlessness from prolonged screen-time, or difficulty using mice and keyboards due to mobility impairments, reDirect is an adaptive technology software to create a no-keyboard or mouse toggle system to minimize the sedentary handling of technology and also allows you to navigate through your applications. You deserve to be comfortable at your desk, so let's reDirect.
What it does
reDirect accesses the user's default webcam and allows the user to pair numerical hand gestures with a designated desktop application for a navigation system that spares you the headache of clicking through the ever-growing array of tabs and applications already open.
How we built it
The reDirect prototype was created with a Python backend and a CSS + HTML frontend that is deployed into the browser with Flask. To evaluate the shape and motion of hands, MediaPipe Hand and OpenCV were used in conjunction to implement machine learning to extrapolate likely landmarks of the hand and match them to the conditional statements to open instances of various desktop applications.
Challenges we ran into
Deploying machine learning while trying to accustom our inputs for a prompt engineering approach continues to be a work in progress. We want to create a seamless web application with no delays, and when implementing conditions to reinforce the hand landmarks detected, these oftentimes slowed down the delivery of the output to the user-side. Reversely, the absence of conditions sometimes leads to too many instances of an app created at once.
Accomplishments that we're proud of
We knew with time, Python would be the best approach to create a responsive web application, however, when we arrived at fostering the interactions between the frontend and backend, storage solutions became rather tedious trying to ensure our web application would successfully deploy. As a team we successfully deployed a local host prototype of reDirect in 36 hours, which is also our first Python project.
What we learned
It was interesting to come across all the different ways we could have implemented our design ranging from using a Raspberry Pi to working directly with TensorFlow.
We also appreciate the importance of pursuing a project with the idea of data sealing at the forefront of decision making to create an easily deployable app.
What's next for reDirect
This first ideation of reDirect emphasizes the barriers that when an individual lacks the dexterity to handle a mouse and keyboard for extended periods of time. The future of our project is to take it a step further and develop a hub for accessibility. For example, reDirect could eventually implement an eye tracker to accomplish the same task or grow our current set of commands beyond toggling. We would also like to create a more compatible and customizable software so users can connect more applications on more devices. Currently reDirect allows for front end customization of apps to demonstrate our design concept, so in the future we would like to implement allowing user input to edit the backend server so they are able to change the numerical hand gesture controls along with the link and display edits you can handle on the user-interface. We would also like to incorporate the already existing option to create an instance of an application (i.e. opening multiple windows of Google Chrome) in tandem with the option to navigate back to previous opened instances.
Log in or sign up for Devpost to join the conversation.