Inspiration
Using a computer can be a hassle sometimes. If you're tired of reaching over to your mouse every time you want to switch a window or move your mouse one line up, our solution is for you.
What it does
It uses the standard webcam on a computer to track hand movement and special commands. This can allow users to move the mouse and click links, scrolling over long documents, navigating browsers and programs, and typing through accurate speech API's such as rev.ai.
How we built it
We ran the application on a flask backend to create clean and elegant menu and control options. The hand detection uses OpenCV and mathematical models to track and predict hand movement. In addition, the most important part of our program, dictating text and navigating the computer using speech was made possible with rev.ai and speech API's.
Challenges we ran into
There was many issues in reducing jitter with the webcam. In addition getting the audio to recognize properly proved a challenge as well.
Accomplishments that we're proud of
We made a very good prototype that we feel comfortable going forward with in the future.
What we learned
What's next for Iron Hand
More advanced interpolation and jitter control. We will reduce errors and hope to make this a marketable product that many will want to use.
Log in or sign up for Devpost to join the conversation.