Real-time assistive vision for colorblind & low-vision users
Hackathon Winner • View on Devpost

TrueLight turns your phone into an intelligent camera that detects objects, analyzes their colors, and alerts you when something in your environment may be hard to distinguish given your specific type of color vision deficiency.

• 🎯 Real-time object detection — YOLOv3-tiny identifies objects in your camera feed
• 🎨 Personalized color analysis — Flags colors you specifically struggle with (protanopia, deuteranopia, tritanopia)
• 🔊 Audio alerts — Natural ElevenLabs voice (or built-in TTS) announces hazards hands-free
• 🗣️ Voice commands — Say "Hey TrueLight, what do you see?" for instant scene descriptions
• 🧪 Built-in Ishihara test — Determines your color vision type during onboarding
• ♿ Low Vision mode — Prioritizes alerts by proximity instead of color
• 🤖 AI Assistant — Gemini-powered conversational Q&A about your surroundings
• 🎨 Adaptive UI — Interface colors automatically adjust so they're always visible to you

Three decoupled services, each with a single responsibility:

┌─────────────────┐     REST     ┌──────────────┐     REST     ┌─────────────────────┐
│  Mobile App     │ ──────────▶  │  Next.js API │ ──────────▶  │  Python Detection   │
│  React Native   │ ◀──────────  │  Gateway     │ ◀──────────  │  FastAPI + YOLO     │
│  Expo           │              │  Auth / DB   │              │  OpenCV HSV         │
└─────────────────┘              └──────────────┘              └─────────────────────┘

The Python service runs YOLOv3-tiny inference in under 80ms on CPU. Detected regions are passed through an HSV color analyzer that maps pixel distributions against your CVD profile. Red detection uses dual-range masking (0-10° and 170-180° on the hue wheel) to handle the HSV wraparound.

• ElevenLabs — Natural-sounding voice for a premium experience
• Expo Speech — Offline fallback that works without internet
• Priority-based debouncing ensures critical alerts are spoken first

Hands-free interaction powered by wake word detection:

Built-in Ishihara plate test with enhanced plates to determine your specific CVD type during onboarding — no external diagnosis needed.

Prerequisites: Node.js 18+, Python 3.8+, Expo Go on your phone. Computer and phone must be on the same Wi-Fi network.

cd python-detection
pip install -r requirements.txt
python main.py
# Runs on localhost:8000

cd backend
cp .env.example .env    # Fill in API keys
npm install
npm run dev
# Runs on localhost:3000

cd mobile
cp .env.example .env    # Fill in API keys and backend URL
npm install
npx expo start
# Scan QR code with Expo Go

mobile/.env

backend/.env

TrueLight/
├── mobile/                     # React Native (Expo) app
│   ├── app/                    # Screens (home, camera, settings, login, onboarding)
│   ├── components/             # CameraView, BoundingBoxOverlay, ColorTestPlate
│   ├── constants/              # Accessibility colors, color profiles, Ishihara plates
│   ├── contexts/               # Auth context provider
│   ├── services/               # API, speech, voice commands, audio alerts,
│   │                           # motion tracking, speed detection, screen recording
│   └── store/                  # Zustand state management
├── backend/                    # Next.js API gateway
│   ├── app/api/                # REST endpoints (detect, auth, health, TTS)
│   └── lib/                    # Auth (JWT), DB (SQLite), Roboflow detection
└── python-detection/           # FastAPI computer vision service
    ├── main.py                 # Server + dynamic sensitivity configs
    ├── detector.py             # YOLOv3-tiny + NMS + fallback color regions
    ├── color_analyzer.py       # HSV color mapping + CVD problematic detection
    └── models/                 # YOLO weights and config files

MIT