Voice-Controlled Browser Assistant

Inspiration

Our inspiration for this project was originally to increase our academic productivity (we can talk faster than we can type!), but as the program progressed, we discovered that it requires minimal physical interaction to use. This technology can be applied to aid those living with physical disabilities in navigating the Internet.

What it does

When the user clicks on the Chrome Extension widget, they have the option to start voice control. After clicking a button, they can give their instructions out loud and their computer will comply if it recognizes the command.

How I built it

Development started with just a 'dummy' Chrome extension, which was then updated to contain a speech recognition client, and was finally updated to respond to commands.

Challenges I ran into

Chrome's extension development policies dictate that a strict procedure must be followed in order to comply with their security measures. As such, it was difficult to gain microphone permissions, and the ability to edit the object model of the webpage is crippled by Chrome's DOM policy.

Accomplishments that I'm proud of

It works :)

What I learned

How to make a Chrome extension, how to develop speech-to-text and text-to-speech clients

What's next for Voice-Controlled Browser Assistant

It would be helpful to the user if they could upload their own commands and keywords. In order to increase usability for those living with vision disabilities, the assistant could also read results aloud by scraping the page for information.

Built With

• aanyang
• chrome
• javascript
• jquery