Real-time AI memory aid for people with dementia, face blindness, and memory impairment.

Demo • Features • How It Works • Getting Started • Architecture • Roadmap

"Because no one should forget who they love."

Faces watches your camera and listens to your conversations in real time. When someone walks up to you, it recognizes their face and shows you their name, your relationship, and every important detail from past conversations — instantly, with zero effort on your part.

Over 55 million people worldwide live with dementia, projected to hit 139 million by 2050. Another 10 million have prosopagnosia (face blindness), 2 million live with aphasia, and nearly 40% of people over 65 experience age-associated memory impairment. There is no real-time, frictionless tool that helps someone know who is standing in front of them.

Click here to view demo.

• Continuous real-time face detection and identification at full frame rate
• Handles multiple faces simultaneously in a single frame, each with independent labels
• Tracks people seamlessly as they move, turn, step out, and come back
• Persistent identity — recognizes the same person across sessions

• Say "My name is Sarah" → name updates on screen instantly
• Say "I'm your daughter" → relationship badge changes live
• Mention "doctor appointment on Tuesday" → captured as a key detail
• No hardcoded phrases — the AI understands natural introductions and relationship declarations

• Extracts dates, appointments, events, people, places, health info from natural speech
• Maintains a living memory list — corrects outdated info instead of duplicating
• Filters out small talk and filler — only stores what actually matters
• All details persist across sessions and server restarts

• Runs entirely in the browser — no install, no native dependencies
• Works on any device with a camera, microphone, and a browser
• Open a URL, allow permissions, done

1. Camera captures video → face-api.js detects and identifies faces client-side using 128-dim descriptors
2. Microphone streams raw PCM audio over WebSocket → custom audio gate filters silence → Whisper transcribes
3. Zen API (LLM) runs two parallel extraction tasks per transcription:
   • Identity extraction — names and relationships from natural speech
   • Key info extraction — dates, events, appointments, people, health details
4. Results are stored in SQLite and pushed to the frontend in real-time via WebSocket

• Python 3.9+
• pip
• A modern browser (Chrome recommended)
• AUTH API key (get one here)

# Clone the repository
git clone https://github.com/abacushacks/adahacks-2026.git
cd adahacks-2026

# Install Python dependencies
cd backend
pip install -r requirements.txt

# Run migrations
python3 manage.py migrate

# Set your AUTH API key
export AUTH_KEY="your--api-key"

# Start the server
python3 manage.py runserver

1. Allow camera and microphone access
2. Click anywhere on the screen to start the audio pipeline
3. Faces are detected automatically — start talking and watch the magic happen

The AI context engine is the core intelligence of Faces. Every transcription chunk triggers two parallel LLM calls via asyncio.gather():

1. Identity extraction — detects name/relationship introductions from natural speech. No regex, no hardcoded patterns — the model understands conversational phrasing.

2. Key info extraction — receives the full current memory state and returns an updated, curated list. Replaces outdated entries, filters noise, only keeps what matters.

Both are race-condition-safe: the face recognition pipeline re-reads the database after its API calls return to avoid overwriting speech-derived updates.

• AR glasses integration — context in your actual field of view
• Lip tracking — supplement audio with visual speech recognition
• Multi-language support
• Caregiver dashboard — family members can add context before visits
• Emotion & mood detection
• On-device LLM inference — fully offline, no API dependency

This project was built for people who need it most:

See LICENSE for details.

Built by Abacus at AdaHacks 2026