AirBoard

Logo
start menu

Inspiration

We had the idea to make a no touch keyboard mouse set-up from our experience at restaurant dining during the covid pandemic where many restaurants made customers sign in their informaiton onto an iPad or laptop to keep contact tracing data. With new health and safety procedures we thought that the check in model was becoming out dated as our devices tend to be the least clean parts of our lives. With our application we can avoid the dangers of touching bacteria heavy devices while maintaining the convenience of a digital form of contact tracing.

What it does

Our application allows a user to have mouse and keyboard functionality without the need to actually touch a mouse or keyboard. We used OpenCV and different python libraries to facilitate a way for the camera to recognize hand movements as cursor movements and hand gestures to be associated with different functions. For example to left click with the mouse you are able to tap you index finger and your thumb together to get the computer to recognize a click. With the addition of a gesture to open and close an onscreen keyboard we are able to perform all the functionality needed from a device without having to touch anything and maintaining a safe distance.

Functionalities

Right click: Middle finger and Thumb pinch Left click: Index finger and Thumb pinch Double click: Pinky and Thumb pinch Access Keyboard: Ring finger and Thumb pinch Cursor Movements: Controlled by the position of the base of your Index finger Scroll Down: Swipe index finger upwards (quickly) Scroll Up: Swipe index finger downwards (quickly)

Challenges we ran into

Our biggest challenge was making it a good experience to use for typing and browsing. The initial set up we had with the pointer corresponding with to the tip of the index finger caused it to be a little glitchy and very jumpy so it was hard to type. We circumvented this issue by connecting it to a section of the palm instead which is also tracked by OpenCV. This combined with the use of a different mouse library and decreasing delays inside the libraries let us make a very stable and responsive system.