AR Geocaching

Geocaching hot-cold game in Augmented Reality utilizing Oculus Quest 2 passthrough technology

Comment

Inspiration

Geocaching and the hot-cold game

What it does

Play a geocaching hot-cold game with Augmented Reality!

Search for the hidden object (connected to the Quest 2 headset via Bluetooth). The Bluetooth RSSI (Received Signal Strength Indicator) is transmitted to the headset and used as an approximate measure of distance from the user. This signal strength is used to display in the user's head-up display whether the hidden object is close (hot) or far (cold) from the user.

AR passthrough allows the user to see their real-world environment while hunting for the hidden object.

How we built it

  • The Oculus Quest 2 connects to the ESP-32 microcontroller via Bluetooth.
  • Unity game engine
  • C# scripting
  • The Singularity, a VRatMIT SDK for connecting custom hardware to Unity VR project with Bluetooth
  • Oculus Integration SDK (for AR passthrough)
  • XR Interaction Toolkit (for UI interaction with Quest controller)
  • Arduino IDE
  • C++
  • Send reading of Bluetooth classic RSSI to the headset
  • SideQuest for recording video from headset including passthrough
  • AI generation for project logo and hot and cold icons (these were also modified)

Challenges we ran into

  • Getting the Bluetooth signal strength from the ESP-32
  • Getting Singularity working to let ESP-32 send messages to Quest 2
  • Enabling passthrough and disabling Guardian on the Quest 2
  • Getting scene objects to show up with passthrough
  • Deciding how to show the distance from the hidden object to the user
  • Getting UI to show/hide
  • Automatically grabbing MAC address of device to connect to
  • Recording video including passthrough

Accomplishments that we're proud of

  • Having a working project
  • Getting Singularity functioning
  • Edge detection to make passthrough look different than the default

What we learned

  • Working with AR passthrough
  • Sending Bluetooth signal strength data from the ESP-32
  • Working with hardware may take a lot of digging online for help
  • Getting data off a microcontroller might not be straightforward

What's next for AR Geocaching

Additional features could include:

  • Allowing multiple devices to connect
  • Geolocation
  • Improving the interface
  • QR codes
  • Add AR effect to the object when they find it

Use cases:

  • Hide and seek with phones
  • Locating tools (like AirTags)

References

Share this project:

Updates