AirSync

Inspiration

Our biggest inspiration came from attending Purdue classes. While explaining concepts or answering questions, our professors had a hard time scrolling through websites which he displayed to the class. Our idea was to help professors multitask and make it easier to navigate websites handsfree. We wanted to make an application which can also help in a variety of things - even if it meant helping students navigate websites handsfree while eating a bag of cheetos. One other inspiration which led to our application is to help those who have a hard time navigating through browsers(like youtube, or even blogs)- since one our team members had a broken hand, we wanted to help him navigate websites better.

What it does

With the help of pre-trained AI models, we are able to control browser functionality including scroll, volume, and other essential functionalities by using signing gestures. We also implemented a Speech Recognition library to make these functions more accurate when using signing gestures(basically not taking in unwanted hand gestures).

How we built it.

We used Python for Computer Vision and Speech Recognition(used 2 AI models), and JavaScript for building a Chrome extension such that every user would be able to use it in just 1 click.

Challenges we ran into

Like almost evey hackathon project, we ran into several challenges along the way. It was hard to figure out how to build extensions and connect our AI model to the chrome extension. Additionally, sometimes async functions were messing up our multithreading. However, we got through most of the challenges and our team is glad we were able to learn so much from this.

Accomplishments that we're proud of

We managed to get it working at the end, with bunch of very cool functionalities build in and we are happy at how interactive our project was. We tested it with several people and they all loved the applications of this project.

What we learned

We learnt how to build Chrome extensions, how to utilize computer vision and voice recognition, and even how to integrate Python and JavaScript together.

What's next for AirSync

We plan to add more features and gestures that users can utilize to manage their hands-free browsing experience including adjusting brightness and even a more comprehensive click functionality.

Built With

• chrome
• css
• html
• javascript
• machine-learning
• opencv
• python
• speech-recognition