Inspiration
My initial idea was to create an AI Advisor bot that assists clients with their mortgage applications. After having worked in the banking industry, specifically on the retail side for five years, I have always had the ambition to build a project capable of automating my tasks. However, to achieve this goal, I needed to implement the core feature: a bot that can be activated using voice and respond with its own voice."
What it does
With that in mind, I'd like to introduce you to ConverseAI, an innovative web application designed to revolutionize the way we interact with technology. ConverseAI seamlessly integrates voice commands, offering an intuitive and effortless way to communicate with artificial intelligence. This cutting-edge platform not only accepts voice prompts but also responds with its own natural-sounding voices, bringing a human touch to your digital interactions.
How we built it
Backend Development: I am using FastAPI, a modern web framework for building APIs with Python. CORS (Cross-Origin Resource Sharing) middleware is added to handle cross-origin requests, allowing communication with my frontend.
Voice Processing: I use the elevenlabs library to generate audio responses. I interact with an API provided by Eleven Labs, which generates audio based on text input.
Chatbot Integration: I use OpenAI's GPT-3.5 Turbo model for chatbot interactions. Users can provide text queries, and the chatbot responds with text.
Frontend Development: I am using React with Material-UI components to build the user interface. I use the react-speech-recognition library to handle voice recognition on the frontend.
Challenges we ran into
Due to a lack of time, I attended too many seminars (which taught me a lot). As a result, I was only able to produce a simple communicative AI. Unfortunately, I was not able to implement my initial idea.
The second issue is latency; when the prompt is too lengthy, it takes some time for the audio to be generated.
What's next for ConverseAI
Work on completing the initial idea would be the first steps. After that, implementing different LLM models, that allows more customization like fine tuning, lora, etc.
Log in or sign up for Devpost to join the conversation.