CAREGIVR

CAREGIVR is a social XR experience designed to connect people going through the end-of-life process and facilitate informed decisions and provide support.

Comment

Inspiration

This project is inspired by a real story of one of our teammates, Steven. A year ago, Steven took care of his grandma in at home hospice. Even with the limited resources available in remote New Mexico, things for him and his family may have been better with more knowledge ahead of time. In the next decade, the number of patients requiring end-of-life care will increase rapidly. 9/10 caregivers are family members. That's 53 million people who could use a platform of communal support.

What it does

CAREGIVR is a social end of life XR experience providing a platform for user recorded stories for story sharing and resources from community and nursing professionals. Designed to connect people to share their experiences and facilitate informed decisions and support, equipping everyone with the ability to face end of life knowing they're not alone.

CAREGIVR takes users on an immersive journey, starting with an introductory poem, afterward they can explore a library of user-recorded stories, gaining insights and emotional connections and record their own story using motion capture technology to share their unique experiences. Another feature is the Reflection Room, a calming space designed for users to process difficult decisions and engage in meaningful conversations with family members. Together, these elements create a supportive and impactful experience that blends storytelling, reflection, and connection to empower caregivers and foster community.

How we built it

CAREGIVR was developed for the Meta Quest 3 using Unity and Meta SDKs. For our motion capture storytelling feature, we utilized Meta Body SDK and BVH Tools. The subtitles were created using Unity Simple SRT and Subtitle Edit. The multiplayer functionality was implemented using Fusion and Photon Voice, but temporarily scrapped due to networking issues at the hackathon. The avatar pipeline used front and back photos, segmentation of them to create a depth map/mesh, then shrinkwrap modifier using a base mesh with UVs that were fixed to the segmentation landmarks for quick texture projection. Rigging done using AutoRigPro.

Challenges we ran into

One of the main challenges we faced was setting up the multiplayer functionality because integrating Photon with avatars proved to be complex and required significant effort. Additionally, the Meta Avatar SDK and UI samples presented conflicts with building blocks, which created compatibility issues during development. Beyond the technical hurdles, the emotional weight of the project was also a challenge. Recording and discussing end-of-life topics took an emotional toll on the team, and we needed to take breaks to process and recharge.

Accomplishments that we're proud of

We are proud of the product we created and the impact it has the potential to make. CAREGIVR not only addresses a deeply meaningful issue that impacts us all, but allowed us to play to our strengths as a team, leveraging our diverse skills and collaborating with great synergy. A special acknowledgment goes to Steven for courageously sharing his personal story, which became the foundation of our project. His experience inspired us to create something that can truly support caregivers and foster compassion within the end-of-life care community.

What we learned

The technical challenges we faced became great learning opportunities. But beyond the technical aspects, we also learned about creating a business model, developing pitch decks, and effectively pitching to investors. Additionally, we explored the market and developed a marketing strategy, which gave us a broader perspective on how to position and promote a product like CAREGIVR. This project was not only a technical journey but also an invaluable lesson in entrepreneurship and strategy for XR startups.

What's next for CAREGIVR

Looking ahead, we have several plans to further develop and enhance CAREGIVR. First, we aim to automate the avatar creation process and potentially scene sense and implement a more stable multiplayer solution. On the design level, we aim to make the platform as accessible as possible for non-experienced users and people with all abilities. We will create a backend system for ingesting the recorded motion and voice data, making the platform fully automated with exception for human content moderation. Another exciting possibility is integrating an AI-powered nurse helper with cliticizations to provide personalized guidance and support to users in real time that are saved and sent to the user via email / smtp. Additionally, we envision developing a dedicated webpage with support resources, potentially using AI to offer tailored recommendations. In the future, we may expand CAREGIVR to include features specifically designed for nurses and hospital chaplains, based on further research into their unique needs. Lastly, we aim to apply for licensing and explore collaborations with insurance providers to make CAREGIVR accessible and impactful for even more people.

Built With

Share this project:

Updates