Inspiration

Vision loss is a silent epidemic affecting 2.2 billion people worldwide (WHO). Yet most cases go undetected until irreversible damage has occurred.

What struck us was the "invisible disability" of partial vision loss. Unlike total blindness, people with deteriorating or peripheral vision can read perfectly and see what's directly ahead, yet they constantly walk into door frames, miss people approaching from the side, and navigate with constant anxiety about what they cannot see. Vision impairment increases risk of depression by 10.6x and anxiety by 12.3x due to loss of independence (PubMed Central).

People can lose up to 40% of their vision before noticing they have a problem (Rosseye).

It could be your grandmother with glaucoma navigating the grocery store, bumping into shoppers she can't see approaching from the side.
Could be your diabetic uncle with retinopathy who's driven the same route for decades, now too anxious to leave the house.
Or it could be your aging parent with macular degeneration, walking into door frames in their own home, losing confidence with every collision they can't predict.

Current solutions weren't designed for this. White canes detect ground-level obstacles but miss hazards at head height. Guide dogs cost $50,000+. Smart glasses for total blindness overwhelm users who still have partial vision but desperately need spatial awareness.

This is exactly the opportunity: build specifically for partial vision loss. Something that doesn't replace sight—but extends and protects what remains. With 1 billion preventable cases and vision impairment projected to affect 3.5 billion people by 2050, this is urgent (WHO, PubMed Central).

Visionary emerged from one question:

→ What if we could restore the confidence to navigate independently, protecting the vision you still have?

What it does

Visionary creates artificial spatial awareness through intelligent multi-modal feedback, designed specifically for people with partial vision loss—whether peripheral, low vision, or progressive deterioration.

The Core System

Intelligent Vision Processing: Our platform captures the environment through a wearable camera, processes it in real-time using computer vision, and identifies obstacles specifically in YOUR vulnerable zones—the areas you can no longer see clearly.

Multi-Modal Feedback Architecture:

🔴 Haptic Guidance:

Directional vibration motors provide spatial warnings
Left obstacle → Left vibration
Right obstacle → Right vibration
Intensity encodes distance and urgency

🔊 Audio Feedback:

Distance-encoded buzzer patterns
Slow beeps → Object far
Rapid beeps → Collision imminent
Creates auditory "countdown" to obstacles

🗣️ AI Voice Assistant:

Natural language scene understanding
"Person approaching from your right"
"Chair two feet to your left"
"Clear path ahead"
Powered by Claude API for contextual awareness
Natural text-to-speech via ElevenLabs

The Key Innovation: Adaptive Awareness

What makes Visionary fundamentally different is our understanding that you're not blind—your vision is compromised in specific ways.

Our system intelligently:

✓ Identifies YOUR specific blind spots (peripheral, central, or patchy vision loss)

✓ Amplifies awareness in YOUR vulnerable zones

✓ Predicts collision paths before you commit to movement

✓ Provides only actionable warnings—no information overload

→ This creates a true "vision extension prosthetic" customized to your type of vision loss, rather than attempting to replace vision entirely.

How we built it

We designed Visionary with a focus on practical reliability and real-world deployment.

Hardware Architecture

Sensing Layer:

Wearable camera for environmental capture (prototyped using smartphone + Arduino)
Real-time video processing at 30fps
Compact form factor designed for glasses mounting

Feedback Systems:

Dual haptic motors (left/right channels)
Distance-encoded buzzer
Light sensors for proximity validation
Arduino/ESP32 microcontroller coordination

Software Pipeline

Computer Vision Stack: Camera Input → YOLOv8 Detection → Spatial Mapping → Decision Engine

Object Detection: YOLOv8 real-time recognition
Zone Division: Three-zone spatial mapping (left/center/right)
Depth Estimation: Distance calculation from bounding box analysis
Confidence Filtering: >85% threshold to reduce false positives

Intelligent Decision Layer:

Our custom algorithm prioritizes threats in vulnerable zones:
Identify all detected objects
Map objects to spatial zones
Suppress zones where user has functional vision
Calculate and predict collision probability for vulnerable zones
Determine urgency based on distance + trajectory
Route appropriate feedback (haptic/audio/voice)

Multi-Modal Coordination: Different feedback channels serve distinct purposes:

Haptics (Grover Buzzer): Immediate, reflexive response (<100ms latency)
Audio: Distance awareness through pattern variation
Voice: Semantic understanding when context needed

AI Integration:

Claude API processes visual data to generate natural language descriptions:

scene_context = claude.analyze(detected_objects, spatial_positions)

# Output: "Two people approaching from right,

# chair on left, path ahead clear"

ElevenLabs converts text to natural-sounding speech for clear, immediate audio feedback.

Design Philosophy: Practical Innovation

Our development was guided by three principles:

Reliability Over Novelty We chose proven, robust components over cutting-edge but unstable technology. Every hardware connection was tested until failure became impossible.
Specificity Over Generalization Rather than building for "blindness" broadly, we focused specifically on partial vision loss—augmenting what's compromised, not replacing what works.
Intuitive Integration Over Complex Features We prioritized seamless, natural feedback over feature count. The system should feel like an extended sense, not a tool requiring conscious operation.

Real-Time Optimization

Achieving reliable real-time performance required careful engineering:

Edge processing for <100ms end-to-end latency
Efficient object filtering (only relevant obstacles)
Predictive caching for repeated environments
Battery-aware processing throttling

Challenges we ran into

Building Visionary pushed us in unexpected ways:

Hardware Integration Complexity:

Coordinating multiple feedback modalities (haptic, audio, visual) with a camera system and processing unit created synchronization challenges.
We had to carefully manage timing to ensure warnings fired cohesively rather than creating a cacophony of conflicting signals.

Real-Time Performance Constraints:

Achieving <100ms latency required aggressive optimization.
We moved processing to the edge, streamlined our detection pipeline, and implemented efficient filtering algorithms. Every millisecond counts when user safety depends on immediate warnings.

Selective Awareness Calibration:

Determining what constitutes a "threat" versus background noise was philosophically challenging.
Too sensitive? = constant false alarms.
Too conservative? = missed dangers.
Solution? We developed adaptive thresholds that balance safety with usability.

Feedback Pattern Design:

Finding haptic and audio patterns that communicate urgency without causing alarm took extensive iteration.
We tested different vibration intensities, buzzer frequencies, and pattern variations until we found combinations that felt natural and informative.

Component Availability:

Hardware constraints and missing adapters forced us to be resourceful and strategic with what we built and implemented.
We kept improvising solutions and testing alternative connection methods to keep development moving forward.

Accomplishments that we're proud of

Despite these challenges, we achieved our core vision:

✓ Functional Multi-Modal System: Successfully integrated camera input, AI processing, and three distinct feedback channels working in harmony

✓ Adaptive Design: Built an assistive system specifically tailored to partial vision loss—whether peripheral, central, or progressive—rather than total blindness

✓ Real-Time Processing: Achieved low-latency performance necessary for safe real-world navigation

✓ Intelligent Spatial Filtering: Developed algorithms that focus on actionable threats in vulnerable zones while respecting functional vision

✓ Natural Feedback Integration: Created haptic and audio patterns that communicate information intuitively without requiring extensive training

✓ AI-Powered Context: Successfully integrated large language models (Claude API) and natural text-to-speech (ElevenLabs) to provide human-like scene understanding

✓ Production-Minded Design: Built with real-world deployment in mind—considering durability, battery life, and user comfort from day one

What we learned

This journey transformed our understanding of assistive technology and human-computer interaction:

*Designing for Specific Disabilities Requires Deep Understanding:

Generic "visual impairment" solutions miss crucial nuances.
Peripheral vision loss requires completely different assistance than central vision loss or total blindness.
The best solutions come from understanding the specific lived experience through research and user insights.

*Multi-Modal Feedback is Powerful When Orchestrated Thoughtfully:

Adding feedback channels only helps if each serves a distinct purpose.
We learned that haptics excel at immediate reflexive responses, audio communicates urgency through temporal patterns, and voice provides semantic context.
However, they must work together, not compete.

*Reliability Defines Real Assistive Technology:

Academic prototypes can afford to fail. Assistive devices cannot.
Users need absolute confidence that their system works every single time. This realization transformed our engineering approach—we tested until failure was impossible, not just until it worked once.

*The Power of Selective Information:

More information isn't better—the RIGHT information is better.
By focusing only on threats in vulnerable zones and trusting users' functional vision, we reduced cognitive load while maintaining safety.

*Hardware and Software Must Co-Evolve:

We learned that you can't design software in isolation and then add hardware (or vice versa).
The feedback systems, processing pipeline, and physical components had to be designed together, with constant iteration between all layers.

*Accessibility is Everyone's Problem:

2.2 billion people with vision impairment worldwide.
1 billion with preventable or treatable conditions.
Millions more at risk from diabetes, aging, and genetic conditions.
This isn't a niche problem—it's a MASSIVE population underserved by current technology. Assistive tech deserves the same engineering rigor and innovation as consumer products.

What's next for Visionary

This hackathon was day one. Here's the roadmap to change 2.2 billion lives.

Immediate Priorities (1-3 Months)

*User Testing & Real-World Refinement

Partnership with vision loss communities (glaucoma, diabetic retinopathy, macular degeneration, retinitis pigmentosa patients) for extensive testing across diverse environments
Iterative refinement based on lived experience feedback, using A/B testing of different feedback patterns and intensities to optimize user experience across different vision loss types

*Technical Optimization

Miniaturization to reduce form factor by 60%, making the device comfortable for all-day wear and user-friendly
Battery optimization for 12+ hour continuous operation without recharging
Seamless integration with prescription glasses so users don't need additional hardware
Enhanced mobile app development for personalization, settings control, and activity tracking

*Algorithm Enhancement

Machine learning for personalized warning patterns that adapt to individual navigation styles and vision loss types
Environmental mapping for "familiar space" optimization—the system learns your home, office, and frequent routes
Predictive collision warning that alerts BEFORE users commit to dangerous movements, not during
Adaptive threshold learning based on individual sensitivity and preferences

Clinical Validation (3-6 Months)

*Medical Partnerships

Collaboration with ophthalmology departments for clinical trials and IRB approval for formal efficacy and safety studies
Partnership with vision health organizations (Glaucoma Research Foundation, American Diabetes Association, etc.) to ensure our approach aligns with clinical best practices
Peer-reviewed publication of results to contribute to the broader assistive technology research community

*Real-World Deployment

Diverse environment testing, including urban streets, suburban neighborhoods, indoor spaces, and outdoor parks
Long-term usage studies measuring comfort, adoption rates, and effectiveness over months of use across different vision loss conditions
Caregiver and family feedback integration to understand the full impact on quality of life

Product Development (6-12 Months)

*Manufacturing Transition

Design for manufacturing—moving from prototype to production-ready device with reliable component sourcing
Quality assurance processes and rigorous testing protocols to ensure every unit works perfectly
Cost optimization targeting $200-300 consumer price point to make Visionary accessible to those who need it

*Regulatory Pathway

FDA wellness device classification research and medical device compliance assessment and documentation
Insurance reimbursement investigation, pursuing durable medical equipment (DME) classification
International regulatory strategy, including CE marking for European markets

Long-term Vision (12+ Months)

*Condition Expansion

Research adaptation for multiple vision loss types:
- Glaucoma (peripheral loss)
- Diabetic retinopathy (patchy vision loss)
- Macular degeneration (central vision loss)
- Retinitis pigmentosa (progressive tunnel vision)
- Hemianopia (stroke-related vision loss)
Each condition has unique navigation challenges—our platform architecture allows customization for different visual impairment patterns

*Geographic Scaling

International deployment reaching 2.2 billion people with vision impairment globally
Localization for different languages, cultural contexts, and urban/rural environments
Partnerships with international vision health organizations to ensure culturally appropriate deployment

*Platform Development

API for third-party developers to build complementary tools and integrations
Integration with smart home systems for seamless indoor navigation assistance
Crowdsourced obstacle reporting where users can flag recurring hazards on common routes
Data analytics platform for understanding navigation patterns and continuously improving algorithms

The Vision Forward

Visionary represents a philosophy of augmentation over replacement. We're not giving people artificial eyes—we're extending their remaining vision into spaces they can no longer see naturally.

Now and Then

2.2 billion people navigate confidently, not anxiously

Assistive technology feels seamless, not stigmatizing

Independence isn't lost along with vision

Technology adapts to human needs, not the other way around

We envision a world where Visionary becomes invisible—not a tool you consciously use, but an extended sense you naturally have. Where navigating your environment feels effortless again.

Where vision loss means adaptation, not dependence.

The future of assistive tech isn't about seeing more. It's about living freely.