VisionAid CV

Personal Agent for Daily Convenience

Inspiration

Every year, we have witnessed how new technology has impacted senior citizens in their daily lives. As technology becomes more and more advanced, we began to realize that it is becoming much harder for senior citizens to use the technology that is put in front of them. Recognizing this pressing issue, we set out to create a project aimed at simplifying technology for users of all ages through an intuitive platform designed to streamline access to the digital world.

What it does

Our project is a personal agent which can do daily tasks which might be technologically complex for senior citizens. We wanted to make it as easy as possible for users to do daily tasks such as sending a text message, or an email, through a personal agent which takes voice requests, and prompts a vocal response back. The agent processes the request and sends the request to the python tools that we developed through index. Once the request is through, the agent is able to do a variety of tasks. It can recommend music, youtube videos, send texts and emails, and answer any potential questions the user has.

How we built it

Our project was built using python for the backend and HTML/CSS for the frontend. The code first takes a voice request and converts the speech to text. Then, the code uses artificial intelligence to interpret the user's request and determine which tool should be used to respond and support the user. Once the tool is declared, the tool completes the user request and then prompts the next request. The frontend and UI was coded entirely using HTML and CSS.

Challenges we ran into

The main challenge we faced when developing the program was getting the live camera to display on the website. Some other challenges we faced included some code errors, finding API's which were free of cost, and setting up the vector database. Despite these challenges, we were able to overcome our problems and develop our project.

Accomplishments that we're proud of

Our team is proud of the fact that we finished our project faster than normal. We were able to have our backend done in 7 hours, and the rest of the time was spent working on the frontend. Additionally, we were proud that we were able to display the video on a front end. Typically it is a challenge but we were able to get it done. Both our time managment and the video display were something that we were proud of.

What we learned

We learned how to send messages via SMS API's and email API's. We also learned how to implement a live video in to an HTML website, since it was our first time ever doing so. Finally, we learned how to divide work and code efficiently when working on projects.

What's next for VisionAid CV

In the future we would want to implement location services so that the personal agent could help senior citizens locate places near them just with one request. We also would want to implement more AR features in order to make the features more accessible.