sleepAR 😴🥽

Inspiration 💡

In today's increasingly digital age, it's not uncommon for people to fall asleep scrolling through Reels, watching Youtube videos, or even just straining their eyes off of bright screens — habits that not only disrupt our sleep cycles but also make it harder to truly disconnect from tech.

But what if technology could actually help us fall asleep instead of keeping us awake?

Combining the immersive potential of AI with a seamless voice-based agent, sleepAR pays homage to classic sleeping techniques — from counting sheep to telling fairytales — all with the purpose of creating a modern sleep experience that keeps you engaged without you feeling guilty.

What it does ⚙️

sleepAR transforms your bedroom ceiling into a personalized sleep sanctuary using Snap Spectacles AR glasses. Instead of staring at your phone, you can:

Gaze at stunning starry skies and space visualizations projected onto your ceiling
Count sheep that actually appear and jump across your field of vision (generated on-the-fly with Snap3D)
Watch calming videos directly in your glasses without harsh blue light
Hear bedtime stories crafted specifically for you
Control everything through natural voice commands as you lie comfortably in bed

The experience is entirely hands-free and designed to gradually lull you into sleep rather than stimulate your brain like traditional screens.

How we built it 👷

We created a voice-controlled AR experience by connecting several cutting-edge technologies:

AR Interface: Built in Snap's Lens Studio to create immersive visuals and 3D models
Voice AI Pipeline:
- Speech recognition via Whisper Large v3 Turbo (Groq)
- Intent processing through GPT OSS for understanding commands
- Natural voice responses using PlayAI TTS

# Call Groq with the transcript
completion = groq_client.chat.completions.create(
    model="llama-3.1-8b-instant",  # Using a fast model for real-time conversation
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": transcript}
    ],
    temperature=0.7,
    max_tokens=max_tokens
)

Integration Layer: Custom websocket connections and API calls to bridge the voice AI with the AR visuals

The system works by capturing your voice commands, determining your intent, and then triggering the appropriate AR experience or content within the glasses.

Challenges we ran into ‼️

Developing for AR glasses presented unique hurdles:

Optimizing real-time 3D rendering for the limited processing power of wearable AR
Creating a reliable voice control system that works in even the loudest environments
Building smooth transitions between different AR experiences to avoid jarring the user
Integrating multiple AI services with different APIs into a cohesive pipelines
Dealing with version incompabilities and patching deprecated / buggy APIs

 async def send_to_client(self, client_id: str, data: Dict[str, Any]) -> None:
        """Send data to a specific client"""
        message = json.dumps(data)
        for websocket in self.connected_clients:
            if str(id(websocket)) == client_id:
                try:
                    await websocket.send(message)
                    return
                except websockets.exceptions.ConnectionClosed:
                    self.connected_clients.discard(websocket)
                    break
        logger.warning(f"Client {client_id} not found or disconnected")

Designing visuals that are calming rather than stimulating to actually promote sleep

Accomplishments that we're proud of 🏆

We're thrilled that we managed to:

Create a fully functional voice-to-AR pipeline that responds naturally to user commands
Generate dynamic 3D content (like sheep!) on demand within AR constraints
Successfully integrate three different AI services into one seamless experience

What we learned 🧠

This hackathon taught us so much about:

The ins and outs of AR development, especially with zero experience across all members of the team
Optimizing AI response times for natural-feeling conversations

vad_group = parser.add_argument_group('VAD & Audio Tuning')
vad_group.add_argument("--aggressiveness", type=int, default=3, choices=range(4), help="VAD aggressiveness (0-3).")
vad_group.add_argument("--silence-duration", type=float, default=0.7, help="Seconds of silence to wait before stopping.")
vad_group.add_argument("--frame-duration", type=int, default=30, choices=[10, 20, 30], help="Duration of each audio frame in ms.")
vad_group.add_argument("--pre-buffer", type=float, default=0.3, help="Seconds of audio to keep before speech starts.")
parser.add_argument("--energy-threshold", type=float, default=100.0, help="RMS energy threshold to consider audio as non-silent (higher = less sensitive).")
parser.add_argument("--use-llm", action="store_true", help="Use Groq LLM for response generation (if available).")

The importance of thoughtful UX design when creating experiences for vulnerable states like pre-sleep
How to rapidly prototype complex multi-technology systems

What's next for sleepAR 📈

We're excited to take sleepAR further by:

Expanding our library of sleep-inducing visualizations and experiences
Adding biometric feedback to adapt the experience based on detected sleep states
Creating personalized sleep profiles that learn what helps each user fall asleep fastest
Exploring additional wellness features like guided meditation and breathing exercises
Optimizing battery usage for all-night wear comfort
Conducting sleep studies to measure and improve effectiveness