LA Hacks 2026 · Camera-powered reminiscence therapy for Alzheimer's patients and their caregivers.
AI / ML
- Gemini 2.0 Flash Vision — photo analysis + real-time emotion detection
- Gemma 4 31B — memory parsing agent (Google ADK)
- Gemini 2.0 Flash — world builder + streaming journal narrator
- Gemini Embedding 2 Preview — 768-dim vector embeddings
Infrastructure
- Google ADK — agent orchestration
- Firebase Realtime Database — live caregiver session bridge
- MongoDB Atlas + Vector Search — memory storage and resonance
- Vultr — backend hosting (Ubuntu 22.04, pm2)
- Cloudinary React SDK (
@cloudinary/react) — audio delivery
APIs
- ElevenLabs Voice Design — three voice profiles including the whisper system
- Cognition MCP — AI output validation
- World ID — caregiver verification
- Fetch.ai Agentverse — notification routing
Frontend
- React, Vite, Firebase JS SDK,
@cloudinary/react,@cloudinary/url-gen
Backend
- Python, Flask, google-generativeai, firebase-admin, pymongo, cloudinary
cd backend
cp .env.example .envFill in .env:
GOOGLE_AI_KEY=
FIREBASE_DATABASE_URL=
FIREBASE_SERVICE_ACCOUNT_PATH=
MONGODB_URI=
ELEVENLABS_API_KEY=
YOUNG_ELEANOR_VOICE_ID=
CLOUDINARY_CLOUD_NAME=
CLOUDINARY_API_KEY=
CLOUDINARY_API_SECRET=
pip install -r requirements.txt
python server.py
# Runs on http://localhost:3000cd frontend
cp .env.example .env.localFill in .env.local:
VITE_BACKEND_URL=http://localhost:3000
VITE_DEMO_PATIENT_ID=eleanor-park-demo
VITE_CLOUDINARY_CLOUD_NAME=
npm install
npm run dev
# Runs on http://localhost:5173Patient experience: http://localhost:5173
Caregiver dashboard: http://localhost:5173?caregiver=true
| Method | Endpoint | Description |
|---|---|---|
| GET | /health |
Health check |
| GET | /memory/demo-seed |
Pre-built Eleanor demo (no AI call) |
| POST | /memory/analyze-photo |
Gemini Vision photo → scene JSON |
| POST | /memory/transcribe |
Text transcript → full ADK pipeline |
| POST | /session/emotion |
Gemini Vision frame → emotion label → Firebase |
| POST | /session/whisper |
Text → ElevenLabs → Cloudinary → Firebase |
| POST | /session/capture-moment |
Gemini summary → MongoDB family journal |
| POST | /journal/stream |
Streaming SSE journal narration |
| POST | /memory/resonance |
MongoDB vector search → resonant count |
├── backend/
│ ├── agents/ # Google ADK pipeline (4 agents)
│ ├── routes/ # memory_vision, session, journal
│ ├── config.py
│ └── server.py
├── frontend/
│ ├── src/
│ │ ├── components/ # CameraCapture, LibraryScene, EmotionMonitor
│ │ ├── pages/ # CaregiverDashboard
│ │ ├── cloudinary.js
│ │ ├── firebase.js
│ │ └── App.jsx
│ └── audioManifest.js
└── ai-prompts/
└── audio-manifest.js