AIDU

• 

  Landing Page

• 

  App Page

• 

  Loading Status

Inspiration

The inspiration for developing this therapy chatbot using AI stems from the growing need for accessible mental health support. The aim is to provide individuals with a convenient and custom platform to seek guidance and assistance in their emotional well-being.

What it does

AIDU utilizes AI to provide a conversational interface for users. It can understand and respond to user queries, offering empathetic and supportive therapy-related suggestions. Users can engage with the chatbot by typing or speaking, with Whisper API converting voice input into text. DALL-E generates a relevant visual content with the goal of relaxing the user. GPT-3.5 powers the chatbot's conversational abilities. Pinecone aids in storing relevant vectors to enhance the chatbot's capabilities. The user's video feed is also showed and using Hume's API we detect and output his real-time emotions.

How we built it

We used Next.js for the frontend of our application. We retrieved transcripts from multiple therapy sessions that we used as embeddings and after converting them to vectors stored in Pinecone. We use DALL-E to generate the image, and Whisper API was integrated to enable speech recognition and transcription. GPT-3.5 powers the chatbot's language understanding and generation capabilities.

Challenges we ran into

First, getting to understand how to collect audio input from the user's microphone and sending that to Whisper proved to be quite challenging and time consuming. We also ran into several other challenges regarding the Hume API. It was also difficult to understand at first, but we managed. Finally, utilizing Pinecone and fetching in a Next.js application proved the most difficult.

Accomplishments that we're proud of

Successfully integrating multiple AI technologies and creating a functional chatbot for therapy.
Implementing voice recognition capabilities using Whisper API, making the chatbot accessible to users who prefer voice input.
Generating personalized visual content using DALL·E to enhance the chatbot's user experience.
Training and fine-tuning GPT-3.5 to provide empathetic and relevant therapy-related responses.
Utilizing Pinecone effectively for efficient vector storage.

What we learned

We learned how to use Whisper API, Pinecone, in addition to the use embeddings for GPT-3.5

What's next for AIDU

We hope to add more functionalities and make it more personalized for each user. The goal for this is to become a valuable and reliable companion for each.

Built With

• hume
• nextjs
• openai
• pinecone
• python
• react