Feel the CUPRA

Using the Muse Headband, the EEG signal was classified by the ML model as indicating focus, which triggered the car’s brakes or acceleration

Our project consists on a groundbreaking digital twin representation of the CUPRA Tavascan car so that the customers of the model are able to learn all its cutting-edge functionalities without reading the manual. This allows them to enjoy a full exploitation of the car features before they even get the car.

The project consists on a digital twin that includes many state of the art functionalities, such as a 3D model representation of the car integrated with a user-friendly tutorial. In addition to the tutorial, voice interactive Generative AI agent is able to answer any user question based on the oficial car user manual using a RAG strategy. Moreover, we have implemented a electroencephalograpic (controlled with the mind) driving simulator for the Tavascan.

The key features of this Project are:

Interactive user-friendly digital twin to showcase all car features. When we first open the application, we are presented with a realtime 3D animation of the Cupra Tavascan welcoming the software. This 3D model is the main focus in the application, since it is the digital-twin that will approach the car to the user before even having it, noticing no differences once you see the physical car. A "Know more" button makes the user eager to feel what is like having a Cupra Tavascan after that cinematic introduction, wich when pressed, leads us to a new scene where we can discover every single one of the features that the car offers to us. A "Learn about..." menu pops up, revealing the possibility to go through 6 different visually and voice assisted tours that guide the user through all the amazing features, without needing to read any manual. Some of this tours have the possibility of you to try out a feature, for example, the car acceleration and braking system, in a physically accurate simulation. In every moment, you have the possibility to ask the virtual assistant any question you might have about the car by pressing a button and start talking. The technologies used for this are Unity3D and C#, using every single feature about Unity: Animations, Lighting, Physics, Interactive UI and integration with all the other project parts.
- Voice interactive AI assistant: it knows all about the user manual and answer the questions the customer has about the car using:
- Gemini 2.0 Flash
- Embeddings (RAG strategy)
- Voice processing
- Pipeline: Record user audio -> precess audio and answer based on the manual -> output the response audio and transcript to text.
- Mind controlled physically accurate driving simulation to showcase the acceleration and brakes of the car.
- The EEG component is implemented using a Muse EEG headband. Although this device is primarily marketed to help users meditate by monitoring their brainwaves, it also provides real-time measurements of alpha and beta wave activity. Since higher beta wave levels generally correspond to increased mental activity, we can infer whether a person is focused or relaxed.

The headband uses four electrodes positioned near the frontal cortex to capture raw EEG signals. We acquired these signals in Python, then filtered out noise via a Fast Fourier Transform (FFT) to isolate the relevant frequency bands.

Next, we assembled a dataset from the cleaned EEG readings and manually labeled each sampling period as “focused” or “relaxed.” Using this labeled data, we trained a Random Forest classifier to predict mental state from EEG features.

Finally, we integrated the classifier into a simple visualization tool: an orange cube whose behavior—accelerating or braking a simulated car—is controlled by the model’s predictions. We exposed the classifier via a Flask API, allowing the Unity-based simulation to update the car’s speed in real time based on the user’s mental state.

Kinematic equations with the real values of power, torque, acceleration curves, mass etc are used to build an unprecedented simulator of the acceleraion and braking phases of the car in the most realistic way in Unity3D using the C# programming language.

When you buy a Tavascan you simply do not look for a regular car, you want the best confort and performance. This simulation aims to make even the most strict enthusiast fall in love with the car by being the car and feeling it.

Challenges we ran into

We had the AI voice assistant ready in Linux but when using it in Windows we struggled to integrate it adequately, as some libraries where not compatible. We surpassed it, making use of Docker.

Another challenge was to correctly process the voice input sended to the Gemini API. It was hard to make it work and transcript it later to text. We managed to make it work and create a good pipeline and have a working AI assistant.

What we learned

Above all, our teamwork and task division skills have been greately increased, specially in what it comes to working under high pressure in a project of this dimensions. Furthermore, Inside this team planning that guided us to completing this challenge, every member had to learn and use technologies that were out of their confort zone, such as having to learn and implement the simulation in Unity or using for the first time the Gemini API.

What's next for Feel the CUPRA

This project goes far beyond than just a hackathon submission, it introduces a new step between physical user manuals and digital interaction and assistance in Industry 4.0. Additionally, it is intended to be an inspiration for future professional projects inside CUPRA for this purpose, such as introducing this new technologies and paradigms to digitize the process of learning a new car, in such a new way that not only enhances the user experience with the car itself, but also makes them enthusiastic about it to the point of being curious of learning more about it.