Haven: AI-Powered Housing & Referral Intelligence

Haven is a mission-critical platform designed to bridge the gap between vulnerable populations and essential housing services. By leveraging advanced AI voice interactions, real-time emotion analysis, and graph-based referral intelligence, Haven transforms the intake and placement process for social services.

About the Project

Haven was inspired by a simple problem: people who most urgently need housing support often face the most complex and exhausting intake workflows. We wanted to build a system that feels more human during intake, while still giving caseworkers structured, reliable data for decisions.

The project combines conversational AI, real-time transcript intelligence, and referral graph analysis into one workflow. Instead of forcing caseworkers to switch across disconnected tools, Haven captures voice conversations, extracts key client context, searches programs, scores relevance and eligibility, and maps outcomes in a single platform.

What Inspired Us

Intake interviews are usually repetitive, high-pressure, and hard to standardize.
Caseworkers need fast triage signals without losing the emotional context of a conversation.
Referral ecosystems are network problems, but many teams still operate with flat lists and spreadsheets.

How We Built It

Frontend: React + Vite dashboard for client management, live transcript viewing, and referral exploration.
Backend: Express + TypeScript APIs for call flow, analysis pipeline, and external service orchestration.
Data layer: PostgreSQL (Drizzle ORM) for operational records and Neo4j for referral-network relationships.
AI pipeline:
- OpenAI for conversational guidance and structured summarization.
- Per-sentence emotion classification from transcript segments.
- Fastino/GLiNER for entity extraction and classification.
- Tavily for contextual housing-program retrieval.
- Weighted relevance scoring and automated eligibility assessment.
Real-time system: WebSocket streaming for transcript + sentiment updates during active calls.

Program relevance is computed as a weighted blend of search and model judgment:

[ \text{RelevanceScore} = 0.6 \cdot \text{FastinoScore} + 0.4 \cdot \text{TavilyScore} ]

This gave us a practical ranking that balances semantic retrieval quality with domain-specific classification.

What We Learned

Real-time voice UX is as much about latency and turn-taking as model quality.
Emotion signals are useful for triage, but they must be treated as assistive indicators, not final truth.
Shared schemas across frontend/backend (shared/) reduce integration drift significantly.
Graph models (Neo4j) reveal bottlenecks and referral patterns that are hard to see in relational views alone.
Multi-service AI systems need clear fallback behavior and strict observability to stay reliable.

Challenges We Faced

Audio and streaming reliability: synchronizing browser speech recognition, TTS playback, and transcript delivery.
Prompt and extraction consistency: keeping structured outputs stable across varied, natural speech.
Service orchestration: coordinating multiple external APIs while preserving low response times.
Data quality under uncertainty: handling partial answers, interruptions, and noisy transcriptions.
Responsible intelligence: surfacing urgency/emotion insights without over-automating human decisions.

Haven’s core outcome is a faster, more compassionate intake-to-referral pipeline that helps caseworkers spend less time on administrative friction and more time supporting people.

Key Features

AI Voice Intake & Transcription

Natural Interaction: Real-time voice-to-text intake using high-fidelity transcription.
Dynamic Response: LLM-powered conversational agents that guide users through complex intake forms naturally.
Automated Summarization: Automatically extracts critical data points (income, dependents, veteran status) from spoken conversations.

Real-time Emotion & Sentiment Analysis

Affective Computing: Detects user emotions (anger, fear, sadness, etc.) and overall sentiment during calls.
Urgency Detection: Uses emotional context and keyword extraction to prioritize high-risk clients.
Emotion Profiles: Maintains a historical emotional profile for each client to track well-being over time.

Context-Aware Housing & Program Search

Tavily Integration: Leverages AI-native search to find the most relevant and up-to-date housing programs based on client specific needs.
Multi-dimensional Relevance: Classifies program relevance using LLM analysis of transcript context vs. program requirements.

Automated Eligibility & Application

Fastino Eligibility Grading: Instant classification of program eligibility based on client attributes and program rules.
Yutori Integration: Automates the submission of applications to external provider APIs or web portals.

Referral Network Graph

Neo4j Visualization: A complex relational graph showing the connections between clients, calls, housing programs, and successful placements.
React Flow Interface: Interactive dashboard to explore client journeys and program efficacy.

Technology Stack

Frontend

Framework: React 18 with Vite
Styling: Tailwind CSS & Shadcn UI
State/Data: TanStack Query (React Query), Wouter (Routing)
Visualization: React Flow, Recharts, Framer Motion

Backend

Server: Express.js with TypeScript
Real-time: WebSockets for live transcript and call event broadcasting.
Database:
- PostgreSQL: Primary transactional storage via Drizzle ORM.
- Neo4j: Graph database for tracking complex referral relationships.

AI & Services

LLM: OpenAI (GPT-4o)
Search: Tavily (AI-powered housing search)
Voice: ElevenLabs (TTS), Twilio (Voice Infrastructure)
Intelligence: Fastino (Entity extraction & classification), Modulate (Emotion analysis)

Getting Started

Prerequisites

Node.js (v20+ recommended)
PostgreSQL instance
Neo4j instance
API Keys for: OpenAI, ElevenLabs, Twilio, Tavily, Fastino, Modulate.

Installation

Clone the repository:
```
git clone <repository-url>
cd haven
```
Install dependencies:
```
npm install
```

Environment Setup: Create a .env file in the root and provide the necessary credentials:

DATABASE_URL=postgresql://...
NEO4J_URI=bolt://...
NEO4J_USER=neo4j
NEO4J_PASSWORD=...
OPENAI_API_KEY=...
ELEVENLABS_API_KEY=...
TWILIO_ACCOUNT_SID=...
TWILIO_AUTH_TOKEN=...
TAVILY_API_KEY=...
MODULATE_API_KEY=...
FASTINO_API_KEY=...

Database Setup:
```
npm run db:push
```
Run the application:
```
npm run dev
```
The app will be available at http://localhost:5000.