Hand Terminal

Hand Terminal adds new layers of user experience while watching videos on the web. You can control your videos by just moving your hand in front of your web camera. (It's almost magical!)

Comment

Inspiration

We were inspired to make our lives easier during these tough times. All we can do now is to relax and watch videos and connect with others. Hand Terminal adds that extra bit of fun and convenience to our leisure and productive times.

What it does

Our Chrome Extension tracks the specific gestures you make with your hand to control youtube videos and soon it will encompass a lot more video streaming services! It is able to play, pause, forward and rewind (Awesome!)

How we built it

We used TensorFlow to build our ML model and trained it with some helpful databases found online to recognize hand gestures and mapped them to HTML5 video player commands. We find tuned the model with our own recognition procedures and added fault tolerance to enhance the experience.

Challenges we ran into

We ran into various fault tolerance situations with the dynamics of the gestures and timing issues. We also found it very challenging to read up on the slim documentation with some of our technologies. We also ran into some classic deadlock and thread synchronization issues when we were dealing with the chrome extension back ground and foreground pages (That was a headache).

Accomplishments that we're proud of

We are very proud of the smooth usages of our app. We tested it with our friends and they sounded very excited to use it in their daily routines. That was our goal and it was very satisfying. Not only we managed to produce a product that people will love using, we overcame so many technical roadblocks such as memory optimization, synchronization, data persistence and browser compatibility.

What we learned

  • We learned how to compile backend java script to static browser friendly java script.
  • We learned how to make chrome extensions.
  • We learned how to deal with AI models.
  • We designed suitable methods of fault tolerance.

What's next for Hand Terminal

We aim to add more features like switching to full-screen, muting and picking the next video to watch! Also we want Hand Terminal to span not jut the youtube domain, but many other video streaming networks.

Built With

Share this project:

Updates