Talk With Case

Frontend

Team Name: 2 AM

Team Members:

Anand (Student Number: 47018775)

Mrunal Mustapure (Student Number: 84358100)

Inspiration

This project is inspired from a deep concern for the mental well-being of people living in the isolated, harsh environments of space stations. Mental Health issues are already a major concern on Earth; in space, with absence of natural surroundings such as grass and sun, these issues are likely to become an extremely important aspect to consider. Recognizing the unique challenges in the black and quiet space, we were driven to develop a solution that could provide vital emotional support.

Inspired by the film "Interstellar" and its portrayal of the AI companion CASE, we aimed to create a conversational AI with a human touch, capable of engaging in meaningful dialogue and offering empathetic support.

Our goal was to harness the power of technology to address the psychological challenges inherent in space exploration, empowering people to navigate their emotions and maintain their mental well-being while embarking on the extraordinary journey of living among the stars.

What it does

Our software, "Talk with CASE" is an AI powered TalkBot programmed to talk like a human with emotions. With the help of this software, a user can engage in a human-like talking conversation. Sometimes, people don't have anyone to talk with and sometimes people don't want to talk about their hurdles with any real human. The software provides a non-judgemental friend someone can always talk with to reduce their stress or just have entertainment. With this software, one can talk to CASE in almost any language of their choice.

How we built it

The project has a frontend, built with React.JS and Three.JS, and a backend, built with FastAPI (background tasks) and openAI APIs. On the frontend, we implemented a 3D model of CASE to provide the feeling of presence of the bot along with API connections to the backend. The backend takes requests from the frontend and then initiates a conversation. The human voice is first converted to text through machine learning models and then this text is fed to GPT 3.5 model along with memory tokens and the model generates a "human-like" response. Finally, this response is converted into speech and is read in a humanly way; and then the process loops until the end of conversation.

Challenges we ran into

Setting up the 3D .gltf model of CASE on the frontend with ThreeJS and handling the lighting and camera movement was very complicated and took a lot of time to figure it out.

Accomplishments that we're proud of

We came up with the idea of Reminder Tokens which we continuously feed to the GPT model to talk like a human. This was very successful and we are very proud of this idea. Unlike an AI model, our model has preferences and does not give diplomatic/generic responses. For example, if you ask CASE about his favourite color, it will give you a color instead of saying something generic.

What we learned

We had very little knowledge on ThreeJS and 3D model rendering on webpages before this project. While working at this project, we not only learnt on how to implement these large glft models on webpages with optimized performance, but also learnt using blender for generating these models on our own. For example, we created a prototype of realistic grass just for sake of learning and fun.

What's next for Talk With Case

We really look forward to improve on the response time of CASE. At the moment, CASE takes an average of 5-7 seconds to respond to our responses due to the complicated implementation and the API lags. Hopefully, we will be able to make this efficiency better by implementing the machine learning model in the backend server itself.