UseAbylity

For controlling media
Body recognition
Browser controls
Cursor movement via hand
Logo

Inspiration

Health-related issues(Paralysis, Spinal injuries, Back problems) inspired us the most for this project. So here goes the story!

Everybody is locked in. Online classes are the norm. Many have developed some sort of back problems It's affecting your productivity? Think of Stephen hawkings using this. Like him, many individuals have the potential to serve humanity but are bounded by their immobility.

Mostly the root cause of back problems -chill out? Sitting or laze around in awkward positions.

Sorry to be a spoilsport, but that affects your spine and muscles!!

So what did we do about it? Find out!!

What it does

It's software that uses existing tech (webcams ) on laptops and integrates them with hand gestures. So now you can control the media using only your hands.

And not just media, we also made one for the internet.

So now you can swish around as much as you like and feel like Professor X from x-men ( the guy who can move stuff using his finger alone)

How we built it

We chose python, for its diversity in library selections. We mainly used MediaPipe. We used the pre-trained hand and body recognizer from mediapipe and used a bit of math to figure out the proper co-ordinate mappings.

Challenges we ran into

We ran into a few problems on our way.

Configuration of co-ordinates for web scrolling (a bit of math)
Media Player adaptation (different stuff works on the different players (vlc, quick time, etc) )
Getting things to work the way we want them too

Accomplishments that we're proud of

We were able to browse the web and consume media in a better manner using our program. We were also able to work through our differences and code errors. We worked as a team and here is the finished product. We hope it can bring a little change to the world.

What we learned

We learned a lot about mediapipe and how computer vision works. The mapping of our hands to different coordinates and the precision of the system really amazed us. We also learned about various back issues and the inability of people with half paralysis or spinal injuries.

What's next for UseAbylity

We aim to package this software into an easy-to-install, working piece of software that can hopefully correct people's posture and also be convenient to them. Also, we hope a huge subset of people who were unable to their computer before due to unfortunate circumstances can feel like the general public again.