RecallAR

Overview

RecallAR is an AI-powered augmented reality (AR) cognitive companion built on Snap Spectacles, designed to support individuals experiencing memory loss, such as those with Alzheimer’s disease or cognitive decline. The system addresses critical real-world challenges including disorientation, difficulty recognizing familiar people, and exposure to unsafe environments—issues that significantly impact both users and their caregivers.

Problem Context

Individuals with cognitive impairment often struggle with:

Recognizing close family members
Understanding their current location
Navigating even familiar environments
Managing anxiety caused by confusion

Existing solutions, such as smartphones or tracking devices, require active interaction, which is often unrealistic during moments of distress. This creates a need for a passive, real-time support system that operates seamlessly within the user’s environment.

Core Approach

RecallAR leverages computer vision, AR overlays, and AI to provide continuous, hands-free cognitive assistance. By embedding support directly into the user’s field of view, the system reduces cognitive load and enables intuitive interaction without requiring user input.

The design centers around restoring familiarity—both socially and spatially—during moments of confusion.

Key Features (MVP)

4.1 Familiar Face Recognition with Emotional Feedback

RecallAR uses on-device face detection and a privacy-preserving, closed-world recognition system to identify pre-registered family members. When a match is found, the system displays contextual AR labels—such as name, relationship, and reassuring messages—next to the individual.

To further enhance user comfort, this visual information is paired with gentle audio guidance generated through a soft, calming voice, reinforcing emotional safety and reducing anxiety.

4.2 Caregiver-Recorded Navigation (Memory Replay)

Instead of traditional map-based navigation, RecallAR introduces a route recording and replay system. Caregivers can record familiar paths (e.g., from home to a nearby location), which are later replayed as AR guidance when the user becomes disoriented.

This approach supports memory reconstruction, allowing users to follow routes they already know but cannot recall.

4.3 Disorientation Simulation for Empathy

RecallAR includes an AR-based simulation mode that mimics the visual experience of cognitive decline. By introducing blur and perceptual distortion, this feature allows caregivers and others to better understand the challenges faced by affected individuals, promoting empathy and awareness.

System Design and Safety Considerations

The system adopts a closed-world recognition model, ensuring that only authorized, pre-registered individuals are identified. If confidence is low, the system defaults to “Unknown,” prioritizing user safety and preventing misidentification.

Additionally, RecallAR emphasizes:

Privacy-preserving processing
Minimal user interaction requirements
Stable and predictable feedback (visual + audio)

These design choices make the system suitable for vulnerable users in real-world environments.

Future Development

Beyond the MVP, RecallAR aims to evolve into a comprehensive cognitive support system by:

Integrating all modules into a seamless real-time experience
Adding risk-aware guidance (e.g., unsafe areas, wandering detection)
Enabling caregiver alerts with location and contextual information
Improving on-device processing for enhanced privacy and lower latency

Impact

RecallAR promotes cognitive and emotional wellbeing by:
Reducing confusion and anxiety
Supporting independent navigation and recognition
Lowering caregiver burden through passive assistance

Aligned with the CareXR mission of leveraging immersive technologies for healthcare and social good, RecallAR empowers vulnerable individuals to live more safely, confidently, and independently.

References

Alzheimer Society of Canada. What is Alzheimer’s disease? https://alzheimer.ca

World Health Organization (WHO). Dementia Fact Sheet. https://www.who.int/news-room/fact-sheets/detail/dementia

Deng, J. et al. (2019). ArcFace: Additive Angular Margin Loss for Deep Face Recognition. CVPR InsightFace. Open-source face recognition library. https://github.com/deepinsight/insightface

Snap Inc. Spectacles Developer Documentation. https://developers.snap.com