simpliFY

Supercharging daily accessibility for people with hearing, sight, or autism spectrum experiences. Our Chrome extension uses Gemini Nano to make the web inclusive for 1 billion, with on-device privacy.

Comment

About simpliFY

Inspiration

The internet offers a lot of information, but it isn't accessible to everyone. More than 1 billion people, including those who are deaf, blind, or have autism, face difficulties using complex or hard-to-understand websites. We developed simpliFY because we connected with friends and family who found dense articles, unclear social media posts, or missing audio recordings difficult to manage. We created simpliFY as a Chrome extension to make the web more inclusive, so everyone can easily access and enjoy online content.

What it does

simpliFY improves web accessibility for users with hearing, sight, or autism. Using Gemini Nano’s AI, our Chrome extension simplifies complex text, provides captions for deaf users, and offers audio narration for blind users. It changes reading levels, adds clear visual elements, and works locally to protect your privacy, changing any webpage into an accessible experience for 1 billion people.

How I built it

We built simpliFY using a Chrome extension that uses Gemini Nano’s built-in AI tools:

  • Rewriter API: Simplifies complicated sentences and technical language while keeping the original meaning.
  • Summarizer API: Condenses long articles into easier-to-read summaries.
  • Writer API: Creates audio descriptions and text for pictures.
  • Prompt API: Analyzes text and images to make sure simplification is accurate.
  • Translator API: Adapts content for people who don't speak English. The extension has a settings page where users choose their needs (like hearing, sight, or autism) and customize settings like simplification levels or audio preferences. All processing happens locally for privacy and offline use. We used modern JavaScript and Chrome’s extension framework, using Tailwind CSS for a simple, accessible look.

Challenges I faced

Integrating multiple AI tools while keeping things running well on different devices was difficult. Making sure simplification was accurate for complex topics, like sarcasm or slang, required improving the Prompt API. Designing a simple and easy-to-use interface for people with different needs was another problem—finding the right balance between simplicity and features. Debugging real-time audio captions for deaf users was also hard due to different audio quality on websites. Finally, making sure everything was private by working locally required testing to avoid data leaks.

Accomplishments I'm proud of

We’re proud of creating a Chrome extension that makes any webpage accessible. Achieving real-time text simplification and audio captions with Gemini Nano’s tools was a big success. Our user-friendly settings page, designed for hearing, sight, and autism needs, ensures inclusivity. Most importantly, we built a privacy-focused solution that works locally, allowing 1 billion users to access the web safely.

What I learned

We learned how to use Gemini Nano’s AI tools to address real-world accessibility problems. Using the Rewriter and Prompt APIs taught us to keep the meaning of simplified content. We learned about designing for different accessibility needs, from visual elements to reliable audio captions. We also learned more about Chrome’s extension system and the importance of on-device AI for privacy and performance.

What's next for simpliFY

Next, we plan to improve simpliFY with real-time emotional cues to help users with autism understand social situations. We’ll add more languages using the Translator API and expand accessibility features, like saving simplified content or connecting with email programs. We plan to improve the interface based on user feedback and work with accessibility organizations to reach more people, making the web truly available to everyone.

Built With

Share this project:

Updates