Haven

Find, tour, and book an affordable, accessible home using only your voice — speak to search, walk it in 3D, then an AI agent calls the realtor for you.

Comment

Inspiration

Finding a home is hard for everyone — and nearly impossible if you can't drive to ten viewings, can't read tiny listings, or can't climb the stairs to tour a place. For people with mobility, vision, or hearing differences, the search is full of barriers before you even get to the rent. For the Housing Dignity track we asked: what if you could find, tour, and book an affordable, accessible home using only your voice — without leaving your chair?

What it does

Haven is a voice-first, accessibility-forward housing platform:

  • Find by voice — say what you need ("a two-bedroom under $1,500 in Milpitas, wheelchair accessible") and Claude turns it into a ranked list of affordable, accessible homes.
  • Tour in 3D — step inside a real, photorealistic Gaussian-splat capture of the home and walk through it hands-free, by voice ("go to the bookshelves," "turn around") or by dragging like Street View.
  • Ask anything — ask about any room or the listing ("is the kitchen accessible?", "how much is rent?") and hear a spoken answer grounded in the home's data — or an honest "that isn't listed."
  • Book by voice — say "book a viewing and call the realtor" and an AI voice agent actually places a phone call to set up the appointment, with a live transcript you can watch.

Find → tour → book, completely hands-free.

How we built it

  • Next.js 16 + React 19 + TypeScript + Tailwind, deployed on Vercel.
  • Claude (claude-opus-4-8) with structured outputs powers every language task — search parsing, 3D navigation, room Q&A, and an app-wide voice "command brain" that routes intent (search / open a home / navigate / ask / book).
  • Web Speech API for in-browser speech-to-text and text-to-speech.
  • Spark (@sparkjsdev/spark) on Three.js renders real Gaussian-splat captures in the browser.
  • ElevenLabs Conversational AI + Twilio place the autonomous outbound call to the realtor.
  • A live activity dashboard and call history with transcripts round it out.

Challenges we ran into

  • The voice talking over itself. The open mic kept hearing our own text-to-speech and re-triggering. We built echo detection, a grace window, and a watchdog so the agent ignores itself but still lets you barge in and cut it off.
  • Gaussian splats are finicky — real captures meant wrestling with splat formats and GPU-upload timing to render in-browser.
  • Making the call real — wiring ElevenLabs' agent to Twilio for a genuine outbound call, with a demo-safe simulation fallback.
  • Intent routing — telling "go to the kitchen" (navigate) from "how big is the kitchen?" (question) from "book it" (action), all from one voice stream.

What we learned

A great voice UX is 10% recognition and 90% the edges — turn-taking, echo, barge-in, graceful fallbacks. We also went deep on Gaussian splatting, agentic function-calling, and real telephony.

What's next

Live listing data, multi-room captures, saved searches, captions on the call transcript, and full screen-reader passes.

Built With

Share this project:

Updates