E-Motion

Controlling car using a cardboard steering wheel (using CV)
Controlling a document using hand gestures
Controlling music setting (here volume) using hand gestures

Inspiration

It is nearly a year since the start of the pandemic and going back to normal still feels like a distant dream. As students, most of our time is spent attending online lectures, reading e-books, listening to music, and playing online games. This forces us to spend immense amounts of time in front of a large monitor, clicking the same monotonous buttons.

Many surveys suggest that this has increased the anxiety levels in the youth.

Basically, we are losing the physical stimulus of reading an actual book in a library, going to an arcade to enjoy games, or play table tennis with our friends.

What it does

It does three things: 1) Controls any game using hand/a steering wheel (non-digital) such as Asphalt9

2) Helps you zoom-in, zoom-out, scroll-up, scroll-down only using hand gestures.

3) Helps you browse any music of your choice using voice commands and gesture controls for volume, pause/play, skip, etc.

How we built it

The three main technologies used in this project are:

1) Python 3 The software suite is built using Python 3 and was initially developed in the Jupyter Notebook IDE.

2) OpenCV The software uses the OpenCV library in Python to implement most of its gesture recognition and motion analysis tasks.

3) Selenium Selenium is a web driver that was extensively used to control the web interface interaction component of the software.

Challenges we ran into

1) Selenium only works with google version 81 and is very hard to debug :(

2) Finding the perfect HSV ranges corresponding to different colours was a tedious task and required me to make a special script to make the task easier.

3) Pulling an all-nighter (A coffee does NOT help!)

Accomplishments that we're proud of

1) Successfully amalgamated computer vision, speech recognition and web automation to make a suite of software and not just a single software!

What we learned

1) How to debug selenium efficiently 2) How to use angle geometry for steering a car using computer vision 3) Stabilizing errors in object detection

What's next for E-Motion

I plan to implement more components in E-Motion that will help to browse the entire computer and make the voice commands more precise by ignoring background noise.