TAVASCAN Rush

Start your Cupra journey before the engine does. Explore your future Cupra TAVASCAN with AI-driven interactions, 3D models, and AR. Learn features, ask questions, and get ready for the road ahead!

Comment

Inspiration

We want to make learning easier for everyone. Newer techlonogies like live voice conversations with LLMs and AR give us the tools to simplify and make learning more interesting. In addion to this we liked the benefits of digital twins and their capability to simulate many processes in the automotive sector. In this project we are exited to use a digital twin in a produt made for customers of CUPRA.

What it does

Learning about new technologies is challening due to the number of availabe features of a product. Buyers of CUPRA Tavascan should read the manual of almost 400 pages to know abaut the innovative features of the car. Our project facilitates learning of the Tavascan's features through a voice and AR based interactive guide that explain the various features and highlighs or animates the relevant parts during the tutorials. This visual and audio modality facilitates leaning and guides the buyer to know all the features of their new car even before receiving it.

How we built it

We used Python and PyMuPDF to parse the Manual and extract images and audio, then saved those images in a Pinecone db in order to retrive them with RAG. We setup a Gemini Live API with function calling to access tools and execute actions on the 3d model during the tutorials. We worked on an AR and Web implementation to have a large range of supported devices. We connected the Gemini function to this actions and other tools like custom made tutorials and a n8n webhook to unlock advanced agentic and RAG capabilities. We fused the interactive AR visualizzation and the low latency voice chat with function calling in our Lavarel code to give useful and precise tutorials to the users.

Challenges we ran into

All of our team members had to lean unfamiliar languages for them like Javascript, Laravel or specific knowlede on glb models and 3d animations. The other challengs was to to integrate as many features as we wanted.

Accomplishments that we're proud of

In addition to the AR interface we make also a 3d interface to use the app from a computer or a non-compatibe device. This interface also uses a slightly different model configuration and has image retrival capabilities to create a more immersive learning experience.

What we learned

Each one of us learned new technologies and how to organize the work in this fun team experience.

What's next for TAVASCAN Rush

We would like to make the project more usefult by adding procedural animations and more complex tutorials that will aid and stimulate the user. Additionally we would like to implement a reward system to motivate the users as they walk along their learning journery. We think that the same methodology can be useful to accellerate learning in various fields and would like to see more projects like ours.

Share this project:

Updates