Amity AI

Helping seniors navigate the web with voice-guided AI assistance for shopping, bills, and everyday browsing

Comment

Amity AI – Submission Write-up

Millions of seniors and people with disabilities struggle to complete basic online tasks—shopping, paying bills, or avoiding scams.
Amity AI is a voice-guided browser companion that sits beside them and walks them through the web, step by step.

Important – Testing

  1. Download the ZIP from the GitHub repository
  2. Unzip the file
  3. Navigate to chrome://extensions
  4. Enable Developer mode (top right)
  5. Click Load unpacked
  6. Select the unzipped folder and confirm
  7. Open the Extensions menu
  8. Pin the extension
  9. Click the Amity AI logo
  10. Test the experience

What it does

Amity AI is an intelligent accessibility layer for the modern web, built to support seniors and users with cognitive, visual, or motor challenges.

Instead of forcing users to adapt to complex websites, Amity adapts the web to the user.

  • Guides users visually
    Highlights exactly where to click and what to focus on, reducing confusion and cognitive load.

  • Explains actions in plain language
    Breaks tasks like checkout or form-filling into calm, step-by-step instructions.

  • Simplifies decision-making
    Filters overwhelming choices and presents clear, understandable options.

  • Protects users in real time
    Warns users before they interact with scams, phishing attempts, or misleading UI patterns.

  • Supports natural interaction
    Allows users to communicate conversationally, reducing reliance on typing, precision, or technical knowledge.

Inspiration

The modern digital world is optimized for speed and technical fluency—designed primarily for younger users.

For seniors and people with disabilities, this creates unnecessary barriers that lead to dependence, anxiety, and isolation.

Amity AI was built to restore independence and confidence by making the web understandable, safe, and humane.

“Crafted with care for those who raised us.”

How we built it

Amity AI was designed around a simple principle: the AI must be able to see, reason, and act on behalf of the user—safely.

Core Intelligence

  • Google Gemini API – Multimodal Vision
    Analyzes real-time screenshots to understand page context and identify where users may be struggling.

  • Native Function Calling
    Executes actions like navigation, cart additions, and form completion safely, without brittle DOM manipulation.

  • Reasoning & Planning Models
    Plans multi-step tasks in advance to ensure actions are safe, logical, and user-appropriate.

  • Contextual Safety Guardrails
    Continuously scans content to detect phishing, scams, and common elder-fraud patterns.

Interface & Prototyping

  • React & TypeScript – Lightweight, stable browser overlay
  • Remotion – High-fidelity demo prototyping of guided interactions
  • ElevenLabs (Prototype) – Natural, comforting voice synthesis

Challenges we ran into

  • Teaching the AI to accurately understand both user intent and page context
  • Balancing powerful automation with a calm, non-overwhelming interface
  • Visualizing invisible AI guidance clearly in demo videos
  • Refactoring microphone permissions to comply with Chrome Manifest V3 CSP rules

Accomplishments we’re proud of

  • A Guided Access system that meaningfully reduces cognitive load
  • An interface designed for trust, clarity, and reassurance
  • A seamless flow that assists users without removing their sense of control

What we learned

  • Accessibility is about reducing anxiety, not just changing font sizes
  • Clear explanations matter more than speed
  • Designing for seniors and disabled users improves usability for everyone

What’s next for Amity AI

  • Full voice-to-voice interaction for hands-free use
  • Family Connect for optional support and safety alerts
  • Expansion into banking, healthcare, and government services

Our long-term vision:
Amity AI as a universal accessibility layer for the modern web, ensuring no one is left behind as technology advances.

Built With

Share this project:

Updates