AGV DOCtor

Inspiration

Our team was inspired by the challenge given by Continental and the opportunity to tackle the complexities of AI for the first time. We were driven by the desire to make complex industrial processes more accessible and understandable.

What It Does

AGV DOCtor is an AI-powered chatbot expert that simplifies the AGV (Automated Guided Vehicle) User Manual, a guide for an industrially used robot in transportation and manufacturing. The chatbot is designed to be accessible to a wide range of users, from beginners with no prior knowledge to experts familiar with technical terms. For enhanced accessibility, we’ve incorporated simple prompts and speech input/output capabilities.

How We Built It

We structured the project into independent modules that communicate over APIs, with each subgroup responsible for a specific part. In the AI module, we preprocessed the PDF into smaller, categorized pieces of information. This refined data serves as the source for user prompts, enabling the creation of accurate responses.

Challenges We Ran Into

On the backend, we faced challenges in refining content and parsing a PDF with over 300 pages of images, tables, and text explanations. Implementing a semantic search with an AI embedding model running on a local system, prompt engineering, and function calling implementation with LLM for better results were also significant tasks. On the front end, implementing speech recognition posed its own set of challenges.

Accomplishments That We’re Proud Of

We successfully built a functioning architecture using the state-of-the-art LLM model and embeddings to refine and deliver the perfect answer to a user’s query. We’re proud to present a clean, easily accessible chatbot that assists users in finding, troubleshooting, and understanding errors when operating an AGV robot, leveraging the existing user manual.

What We Learned

Our journey led us to learn frontend animation using Remix and React, understand and implement RAG (Retrieval Augmented Generation), and parse PDFs with images, tables, and text explanations along with content refining for the AI model.

• Frontend web layout, understanding color schemes, and designing vector graphics
• API development in Python
• Teamwork and coordination

What’s Next for AGV DOCtor

We envision AGV DOCtor evolving to support multiple users and developing multi-modal capabilities. We’re excited about the potential of AGV DOCtor to revolutionize the way users interact with complex industrial manuals.

Built With

• javascript
• python
• remix
• tailwind
• typescript