RoadBook

Road Book VR is a VR truck-driving HUD that gives players real-time speed, fuel, alerts, and navigation in an immersive way.

Comment

Inspiration Truck drivers deal with a huge amount of information at once—weather, warnings, fuel, navigation, and unexpected hazards. Most VR driving experiences barely show any useful info, so players don't understand the challenges real drivers face. RoadBook was inspired by the idea of giving players a realistic, helpful, and immersive HUD that shows what truckers really experience on the road.

What It Does RoadBook VR is a prototype heads-up display for VR truck simulations. It shows speed, fuel, weather, road warnings, navigation, and AI-generated assistance. Everything is displayed in a clean, readable VR interface that updates dynamically using mock data. Hand interaction support makes the HUD feel natural and easy to use inside VR.

How We Built It RoadBook was built in Unity using the Meta XR SDK All-in-One package. We created multiple panels for weather, warnings, chat, and navigation—each using mock data to simulate real truck conditions. We also implemented ray-interaction buttons and future-proofed the UI for passthrough and AI upgrades. A custom VR typing keyboard was added for interacting with the AI assistant.

Challenges We Ran Into

  • VR UI panels behaved differently than standard Unity UI, causing issues with ray interactions, scrolling, and button clicks.
  • The VR keyboard didn't always open on the first attempt due to object activation timing.
  • Integrating AI responses (Ollama → Hugging Face) required rewriting the backend messaging system.
  • Designing a HUD that stayed readable inside VR without taking over the player's view was harder than expected.

Accomplishments We're Proud Of

  • A fully working HUD prototype with multiple interactive panels
  • Weather and road warning systems using dynamic mock data
  • A VR-ready AI assistant with an on-screen keyboard
  • Clean, immersive visuals with animations and easy interaction
  • A strong foundation that already feels like a real trucker tool in VR

What We Learned

  • VR UI design is very different from 2D UI, and spacing, text size, and distance matter a lot
  • Interaction timing in VR needs careful control (especially for keyboards and toggles)
  • Modular systems (like weather, warnings, chat) make expanding the HUD way easier
  • AI can enhance immersion when used for contextual feedback in simulations

What's Next for RoadBook

  • Adding real-time data instead of mock data (AI vision, location, weather APIs)
  • Passthrough-based mixed reality driving mode
  • Advanced hand-tracking gestures for interaction
  • Full truck cockpit simulation including steering wheel, buttons, and gauges
  • A mission system with deliveries, hazards, and progression
  • A complete AI co-pilot that can warn, guide, and assist the driver

Built With

Share this project:

Updates