Icarus

Prevent eye strain with real-time eye landmark detection and voice-based eye exercises with Icarus, the Chrome extension!

Comment

Inspiration

According to projections published in the British Journal of Opthamology (Chen et al.. 2023), an estimated 40% of the world will be nearsighted by 2050. Already, most office workers average 8+ hours of screen time a day. But most of us don't realize our eyes are under stress until symptoms are already severe.

What It Does

Icarus is a Chrome extension that uses your webcam and computer vision to detect early signs of eye fatigue as you work, and intervenes the moment strain appears. It includes voice-based eye exercises to combat fatigue.

Detection: Icarus runs MediaPipe locally in the browser to extract facial landmarks and blendshape scores to detect signs of eye strain. From these, it continuously tracks:

  • Blink rate: low blink frequency is the earliest sign of eye strain
  • Eye squinting: brow depression + eye narrowing signals active discomfort
  • Eye rubbing: hand landmark detection catches the reflexive rub that signals fatigue
  • Screen distance: face width in normalized landmark coordinates detects when you're leaning too close
  • Scoring: a 60-second sliding window computes a composite strain score (0–100) that factors blink deviation, squint duration, and eye rub events.

Every 10 seconds, the extension posts time-stamped data to the backend.

Features

  • Eye Exercises (ElevenLabs API): all eye exercises are narrated with text-to-speech using ElevenLabs API, so that you can do them while looking away from the screen and give your eyes a rest.

  • Dashboard (Lovable AI): a full analytics dashboard, built with Lovable, visualizes your strain history, blink trends, squint patterns, and screen distance over time.

  • Insights (Gemini AI API): after three or more sessions, Gemini AI API analyzes your weekly metrics and surfaces four personalized insight cards

  • Eye Strain Metrics Storage (MongoDB Atlas): Stores eye strain metrics from timed work sessions. Image data is stored locally.

How we built it

  • Extension: Chrome Manifest V3, MediaPipe FaceLandmarker + HandLandmarker, ElevenLabs API, JavaScript
  • Backend: Next.js 15 App Router, REST API, tunneled via ngrok SDK for external access
  • Database: MongoDB Atlas with Mongoose
  • AI: Gemini 2.5 Flash API, cached for 6 hours to conserve quota.
  • Frontend: React + Vite on Lovable, Tailwind CSS
  • Deployment: Used Lovable to deploy the app. We also registered a custom .tech domain name.

Challenges we ran into

Configuring webcam permissions and storing API keys for the Chrome Extension was a challenge, because it was our first time working with Chrome extensions.

Accomplishments that we're proud of

Built a fully functioning computer vision pipeline and webcam with a composite strain score, as well as a production-quality dashboard with real session data flowing through it by demo day!

What we learned

Building real-time computer vision and integrating API keys in a browser extension is very different from designing a web app.

What's next for Icarus

We’d love to work toward more comprehensive data analysis and display. Features like low-light detection and more targeted exercise interventions would help create a more comprehensive interface.

Built With

Share this project:

Updates