Capsule

Inspiration

Isolation and memory loss deeply affect elderly individuals, particularly those separated from loved ones. This isolation, coupled with memory challenges, can make it difficult to retain emotional connections and recall cherished moments. Elderly individuals face obstacles such as fragmented memories, the emotional toll of prolonged separation, and few opportunities to relive meaningful experiences. Even joyful memories fade with each missed call or delayed visit, widening the gap between loved ones.

In response, we created Capsule—an immersive VR experience for the elderly, rooted in reminiscence and music therapy. Through "memory capsules" that represent unique, cherished moments, users can interact with personalized environments, familiar sounds, and music, encouraging comforting memory recall. Inspired by Matthew’s bond with his mother, our idea developed further through insights from a Unity AR workshop with Greg Madison and guidance from volunteer Anna Tee on integrating music therapy. These combined influences shaped Capsule into a tool for meaningful memory and emotional support.

What it does

Capsule provides a virtual "Memory Palace" where users explore mementos from loved ones, each linked to a cherished memory. Interacting with these artifacts—like a seashell in a bottle—transports users to a personalized environment that embodies the essence of that memory. Users can then store these mementos within their Memory Palace, building a growing archive that fosters connection and recall.

How we built it

Capsule combines advanced VR technology and thoughtful design to deliver an immersive memory experience for elderly users. Developed on Unity for Meta Quest 3, Capsule uses OpenAI's LLM prompting for Skybox AI to recreate memory scenes with vibrant visual and audio cues, immersing users in meaningful environments. We also used OpenAI's Whisper to extract the audio from the video into text for streamlining this process. For a calm, light aesthetic we utilized SketchFab and Meshy for low-poly assets, optimizing both performance and visual appeal. Design planning and mockups were crafted in FigJam, allowing us to build an intuitive, user-centered interface that preserves cherished memories in a deeply engaging way.

Challenges we ran into

Creating Capsule presented unique challenges, especially since three out of four team members were new to Unity, and none had prior experience coding for Meta Quest 3. Setting up seamless scene transitions on interactive clicks proved particularly difficult but essential to creating a smooth user experience. Finding suitable assets also required considerable effort; locating free, low-poly assets that conveyed our values of care, intimacy, and empathy involved a careful selection of calm colors and gentle aesthetics to establish a welcoming atmosphere for elderly users. Designing an accessible UI was another key focus. Crafting an interface that would be visually accessible to elderly users required us to emphasize a blend of audio and visual elements to create an immersive, easy-to-navigate experience. Despite these design and technical challenges, each obstacle strengthened our process, helping us build a more thoughtful, human-centered product.

Accomplishments that we're proud of

Entering the hackathon, our team was both excited and challenged by the unfamiliarity of Unity and Meta Quest 3. With limited time, we quickly adapted, combining our strengths in design and development to tackle steep learning curves and unexpected technical obstacles in this new field. We’re especially proud of creating Capsule, an immersive experience designed to enhance elderly well-being by reducing loneliness and aiding memory recall. Imagining the impact Capsule could have for elderly users—helping them feel closer to loved ones despite physical distance—gave the project profound significance for us. This experience taught us the value of designing with empathy and showed us the potential of technology to address deeply meaningful, real-world challenges.

What we learned

Participating in this XR hackathon has been an incredibly enriching experience, filled with hands-on learning and collaboration. Coming together as a team with diverse skill sets in design and development, we tackled a completely new challenge—none of us had ever developed for a mixed reality immersive experience before. Despite our varied levels of experience, we quickly found a good rhythm, brainstorming and aligning our ideas to create a project that addressed a meaningful need we all felt passionate about: supporting elderly individuals in preserving cherished memories. Throughout this journey, we pooled our different strengths, blending creativity in design with technical problem-solving to develop a user-friendly, immersive experience. Collaborating on every detail, from the user interface to the core functionality, was not only rewarding but also pivotal in setting us on the right track. This hackathon has been an invaluable opportunity to explore cutting-edge technology in VR and AR, expanding our skill set and inspiring us to think about the real-world impact our work can have.

What's next for Capsule

Our next steps for Capsule include integrating a LLM-powered chatbot for personalized, memory-focused conversations, building on past exchanges between loved ones using OpenAI's Whisper GPT4 and ElevenLabs, where the workflow may be as follows: User speaks -> Whisper transcribes -> GPT-4 generates response -> ElevenLabs converts to audio -> Chatbot plays response back. We’re also adding reaction features so recipients can acknowledge memory capsules, along with enhanced spatial awareness on the onboarding screen for a smoother start. To deepen the therapeutic impact, we plan to incorporate audio-visual mementos that combine voice recordings with imagery, leveraging music therapy techniques to support memory recall, especially for users with early dementia or Alzheimer’s.