VerbAIise 93

Inspiration

In recent years, many applications have been fine-tuned to make the user experience easy as possible. As a Chrome user, I as well as many other people in the community have found that the Google virtual assistant, while offering some tools, lacks many large features that have become standard in today's time. For example, the google assistant does not offer the ability to text others while on a computer, an ability that would simplify communications between devices exponentially; Not only this, but we also found that by utilizing the popular AI, Chat GPT, you could generate accurate and original responses to questions like, 'write me a poem.' All of these reasons lead us to create VerbAIlize, the best alternative to Google Assistant.

How we built it

To recognize speech, we first read the raw audio data using PyAudio. Then through a WebSocket connection, we used Deepgram’s API to perform live transcription of the raw audio data. We had a commands framework to handle custom commands such as sending texts which defaulted to feeding chat GPT a prompt. Finally, we would use Google’s text-to-speech API to generate an audio file which we would play with pygame so that it seemed like the program was giving a live “response” to the user.

Challenges we ran into

We faced many challenges while creating this project. A major roadblock that we hit was having the program recognize speech in time. We solved this problem by reading audio as a stream and analyzing the raw audio data with an API that provided transcription through a WebSocket which made the program run almost 2-3x as fast. Another challenge that we faced was getting chat GPT to respond in a sufficient time span. We were previously using the GPT-3.5 Turbo model to receive responses but we eventually switched over to text-DaVinci-003 because it provided quicker responses while also giving proficient detail.

Accomplishments that we're proud of

We're proud that we got to finish our project early, which gave us time to improve on the project more and to build a better presentation/website. We are also very proud that we managed to create an awesome working project despite some of our members joining late. Our team is also proud that we were able to get all members of our team to be involved with the project and contribute as much as possible. Although we had different ideas for how to approach this project, we were still able to cooperate well together to make sure everything was functional and satisfactory.

What we learned

The most important thing is that we all learned how to work together, communicate with each other, and talk about our ideas and thoughts with one another. We also learned how to be more confident in ourselves by sharing our ideas. We were able to learn from each other's strengths and weaknesses throughout this project. We have been working so hard on this project for the last 24 hours and it feels great to see the end product of what we've created. We did a really good job of communicating with one another and supporting each other without getting frustrated. Overall, we are very proud of ourselves and we can't wait to continue on our journey to becoming good coders.

What's next for VerbAIise

Due to numerous time constraints and the obstacles we faced, there is so much to improve and add to VerbAIise. For starters, we only programmed some functions and voice commands for the project. There are many more functions that we could add to the virtual voice assistant, some ideas we have are relatively new and different while others are enhancing existing functions from other virtual assistants like Alexa and Siri. Furthermore, we want to take bigger steps to improve the Chat GPT function, allowing it to maintain longer conversations and build off ideas previously talked about. This would make the experience and intelligence of the virtual assistant even greater. Finally, we hope to bring VerbAIise to the app store to provide better and stronger alternatives to the existing virtual assistants that have many flaws.