BookwXRms

Inspiration

Our team’s initial inspiration came from our love of physical books and the unique experience they offer. Too often, people express a preference for physical books over eBooks, citing their tactile and immersive nature. However, we saw potential in mixed reality to not only enhance the reading experience but also cater to diverse learning modalities—kinesthetic, auditory, and visual.

We wanted to create a platform that blends the best aspects of traditional reading with cutting-edge XR technology, transforming how stories, lessons, and information are consumed. By leveraging mixed reality, BookWormsXR offers readers the ability to interact with content in dynamic ways—whether it’s through hands-on engagement, immersive audio, or visually stunning environments.

Education has always been important to us, but the way traditional systems approach learning and literacy often leaves many students behind. Many students disengage for various reasons. While we can’t address every challenge, we believe immersive learning can reach those who feel overlooked—sparking curiosity and helping them hold on just a bit longer. That moment of curiosity can transform a life, changing its trajectory in profound ways. We believe everyone has the potential to be a genius, but not everyone is taught in ways that resonate with their unique learning styles. This is what inspired us to innovate on reading: creating tools that make reading exciting, personalized, and immersive.

There’s something irreplaceable about physical books—the texture of the pages, the scent of ink, the weight in your hands. No Kindle or PDF can truly replicate that experience. But what happens when you stumble upon an unfamiliar word? Or when a passage sparks curiosity about a historical event, a scientific concept, or an artistic masterpiece? Interrupting your flow to grab your phone or search for context often breaks the immersion of reading.

That’s why we created BookwXRm, an mixed reality reading companion that enhances books without replacing them. Read a passage and hear its world come alive. Look at a symbol to summon immersive audio, images, or videos. With BookwXRm, books are not just read—they’re experienced.

What it does

Page Tracking: seamless AR integration to recognize the text you’re reading. Environmental Immersion: dynamic 3D environments inspired by the book’s content. Audio Enhancements: Curated soundscapes and narrations for an immersive auditory experience. Interactive Features: Engaging elements like gamified reading, skyboxes, and interactive characters to spark curiosity. With BookwXRm, books come alive—making learning and reading an unforgettable journey.

How we built it

Using Xcode, Reality Composer, and Swift for Apple Vision Pro (AVP)

Challenges we ran into

The MIT WiFi was spotty, so we had to rely on personal cellphone's hotspot in order to build to our device (AVP). Two of our teammates didn’t have a mac, so we definitely used pair programming and teamwork on the apple devices that were available. It was a learning curve for everyone, as our team came from a Unity development background but migrated native xCode for this project. However, we wanted to take the challenge and learn visionOS development with some of the best mentorsat oour disposal.

One issue we had during development was that we didn’t have access to camera feed, so we couldn’t use OCR tools to process the text and determine which page the user was on. This led us to our next approach, image tracking. The only issue is that image tracking is designed to track QR codes first and pictures second, so text heavy images with little to no pictures would have difficulty being tracked. To solve this we used a picture book, as well as QR codes in the corner of each page of the book. Another issue we had was that the book wasn’t visible when the user entered immersive mode. To fix this issue we created an occlusion box that spawned where the book anchor was. This allowed the user to have a window into the real world where they can see the book and read to their heart's content. Though there were challenges, we’d like to thank the incredible help that was provided to us. With just 3 Macs in our 5 person team, we were able to pull off a complete, immersive experience for AVP.

Accomplishments that we're proud of

We’re incredibly proud of challenging ourselves to use entirely new tools—Xcode, Reality Composer, Swift, and Vision Pro—especially as first-time users. Even team members unfamiliar with macOS gained hands-on experience, which felt like a huge leap forward. We turned what seemed like a daunting technical challenge into a working prototype.

What we learned

Setting up and connecting Vision Pro to Xcode and Reality Composer. Importing objects, videos, and dynamic skyboxes. Understanding how to utilize AR tools to create immersive learning experiences. This project was an incredible crash course in Apple’s ecosystem, and we came out with a deeper understanding of AR development.

What's next for BookwXRms

We’re excited to continue refining BookwXRm and experimenting with its potential. Here’s what’s ahead:

Mac Users: Prototyping more AR experiences with Xcode and Reality Composer using the Vision Pro simulator. Vision Pro Holder: Further development and iteration with the physical device. Non-Mac Users: Inspired to invest in Apple hardware someday and dive deeper into the possibilities of VisionOS. The journey of BookwXRm is just beginning. We can’t wait to see how this technology evolves and how it can inspire learners in ways we never imagined.