Voithos

Inspiration

We've all experienced that awkward moment in a conversation where we're not sure what to say next, or when someone asks a question and we wish we could instantly access the answer without pulling out our phones. What if your glasses could whisper suggestions to help you navigate conversations naturally while giving you real-time access to verified web information—all without breaking eye contact or disrupting the flow of interaction? Voithos was born from the vision of creating a truly intelligent, context-aware AR companion that understands not just what you're saying, but how you're feeling, and provides the information you need exactly when you need it—all displayed seamlessly in your field of view.

What it does

Voithos is an AI-powered conversation assistant for Snap Spectacles that combines three powerful capabilities: 🎭 Emotion-Aware Context: Using REKA AI's vision capabilities, Voithos continuously analyzes your camera feed to understand the emotional context of your environment, helping tailor suggestions to the social situation you're in. 💬 Intelligent Conversation Assistance: Through natural voice interaction, Claude AI listens to your conversations and provides contextually relevant suggestions and responses—displayed as text on your AR glasses to help guide your discussions naturally. 🌐 Real-Time Web Search: When factual questions arise, Tavily's web search instantly delivers verified, real-time information from the internet. We've optimized the display to show ultra-compact results (around 10 words) that are perfect for AR glasses—giving you the essence of web sources without overwhelming your view. All of this happens hands-free, with information displayed directly in your AR glasses, allowing you to stay present in conversations while having an intelligent assistant at your fingertips.

How we built it

Platform: Snap Spectacles with Lens Studio v5.7.2+ Core Technologies: REKA AI: Real-time emotion and context analysis from camera feed Claude AI (Anthropic): Natural language processing for conversation suggestions Tavily API: Real-time web search with verified sources Lens Studio Speech-to-Text: Voice transcription JavaScript: Custom integration scripts connecting all services

Challenges we ran into

Script Initialization Issues: Initially, TavilySuggestions wasn't executing searches. We discovered it was referencing the wrong component—trying to read from the ClaudeIntegration script object instead of the actual Text component where Claude writes its output. Fixed by properly linking to claudeSuggestionText as a Component.Text type.
Irrelevant Search Queries: Tavily was initially searching based on Claude's conversational suggestions (e.g., "Ask if they've attended one") rather than the user's actual question (e.g., "What is hackathon?"). We solved this by: Modifying ClaudeIntegration.js to store and expose the user's raw transcript via script.api.getUserTranscript() Adding logic in TavilySuggestions.js to prioritize the user's actual question over Claude's suggestions Implementing smart query rewriting to convert fragments into proper search queries
Display Limitations: The biggest challenge was fitting web search results on AR glasses. Initial results were thousands of characters—way too long for the limited screen space. We solved this through aggressive optimization: Removed headers and footers Truncated AI summaries to 60 characters (attempting to capture first sentence) Limited to top 3 sources only

Accomplishments that we're proud of

Created a fully functional multi-AI system running entirely on AR glasses with no external server required ✅ Solved the "too much information" problem for AR displays—our ultra-compact format delivers useful web results in under 10 words per item ✅ Built intelligent query routing that distinguishes between conversational AI suggestions and factual web search needs ✅ Achieved real-time performance with emotion detection, voice transcription, AI processing, and web search all happening simultaneously ✅ Seamless integration of three separate AI services (REKA, Claude, Tavily) into a cohesive user experience ✅ Hands-free interaction that keeps users present in conversations while providing intelligent assistance

What we learned

Technical Insights: AR displays require extreme content optimization—every character counts Component referencing in Lens Studio can be tricky; always verify you're accessing the right object type Distinguishing between different types of AI outputs (conversational vs. factual) requires thoughtful architecture String manipulation and truncation strategies need to respect word boundaries for readability Design Insights: Less is more in AR—showing 3 compact results is better than 20 overwhelming ones Users need clear attribution (web sources vs. AI-generated) to trust information Context-aware computing requires coordination between multiple AI systems Development Process: Extensive logging was crucial for debugging multi-component systems Iterative refinement based on real-world testing (display length, query relevance) dramatically improved the user experience

What's next for Voithos

Short-term enhancements: Conversation memory: Track conversation history to provide more contextually relevant suggestions Multi-language support: Extend to languages beyond English Customizable display preferences: Let users adjust text size, result count, and verbosity Offline mode: Cache frequently asked questions for basic functionality without internet Long-term vision: Proactive assistance: Anticipate needs before users ask (e.g., showing restaurant info when looking at a storefront) Social graph integration: Provide context about people you meet (if they opt-in) Learning mode: Dedicated functionality for students to get instant explanations of concepts Professional applications: Specialized modes for medical professionals, technicians, or customer service roles Privacy-first sharing: Allow users to selectively share their AR assistant's insights with others in real-time