KAIRO-Accessibility Agent - Project Pitch
Inspiration
The inspiration came from witnessing how complex digital content creates barriers for users with different accessibility needs. We noticed people struggling with dense legal documents, language barriers, and privacy concerns when sharing sensitive information online. With Chrome's new built-in AI capabilities, we saw an opportunity to create a comprehensive accessibility toolkit that works entirely offline, ensuring privacy while breaking down these digital barriers.
What it does
KAIRO-Accessibility Agent is a Chrome extension that revolutionizes the way users interact with web content using various AI-powered tools:
- Text Processing: Summarizes content, rewrites text in different styles (professional, simple, casual), and translates to 12 languages
- Privacy Protection: Detects sensitive data (emails, credit cards, SSNs) and creates anonymized versions of text
- Legal Document Analysis: Simplifies Terms & Conditions, extracts key points, and identifies privacy/security risks
- Image Analysis: Captures screenshots and provides AI-powered descriptions with interactive follow-up questions
- Accessibility Features: All tools work offline, support keyboard navigation, and provide downloadable results
How we built it
We leveraged Chrome's cutting-edge built-in AI APIs (Gemini Nano) to create a privacy-first solution:
- Frontend: Built with vanilla JavaScript and Material Design-inspired CSS for clean, accessible UI
- AI Integration: Utilized Chrome's LanguageModel API for text processing and Summarizer API for content analysis
- Architecture: Implemented a modular system with background service workers, content scripts, and popup interface
- Privacy Design: All processing happens locally on-device with no data transmission to external servers
User Experience: Created context menus, keyboard shortcuts, and streaming responses for real-time feedback
Challenges we ran into
API Limitations: Chrome's AI APIs are experimental with strict character limits (4000 chars) requiring careful text chunking
Cross-browser Compatibility: Features only work in Chrome 138+ with specific flags enabled, limiting accessibility
Memory Management: Properly destroying AI sessions to prevent memory leaks while maintaining responsive streaming
UI Consistency: Ensuring popup positioning and styling works across different websites and screen sizes
Error Handling: Managing various failure states when AI models aren't available or pages are restricted
Accomplishments that we're proud of
Complete Offline Functionality: Built a full-featured AI assistant that works without internet connectivity
Privacy-First Design: Created sensitive data detection and anonymization tools that protect user information
Accessibility Focus: Designed intuitive interfaces that make complex AI capabilities accessible to all users
Multi-modal Capabilities: Successfully integrated both text and image analysis in a single cohesive tool
Real-world Impact: Developed specialized T&C analysis tools that help users understand legal documents
What we learned
Emerging AI APIs: Gained deep experience with Chrome's experimental AI capabilities and their practical limitations
Privacy Engineering: Learned to design AI applications that prioritize user privacy through local processing
Accessibility Design: Understood the importance of creating tools that genuinely improve digital accessibility
Performance Optimization: Mastered streaming AI responses and efficient memory management for browser extensions
User-Centered Development: Discovered how to balance powerful AI features with simple, intuitive user experiences
What's next for Kairo-Accessibility agent
Enhanced Language Support: Expand translation capabilities and add support for more languages and dialects
Advanced Privacy Tools: Develop more sophisticated data classification and automated redaction features
Accessibility Improvements: Add screen reader support, voice commands, and customizable UI themes
Cross-Platform Expansion: Adapt the toolkit for other browsers as AI capabilities become more widely available
Enterprise Features: Create team collaboration tools and compliance reporting for organizations
Mobile Integration: Explore mobile browser extensions and standalone app versions for broader accessibility
Log in or sign up for Devpost to join the conversation.