TreeHacks 2026
CareSync is an end-to-end healthcare platform where AI agents autonomously monitor patient health, detect anomalies, triage alerts, consult patients via a talking avatar, and book appointments — all with full observability and cryptographic trust.
/
apps/web/ Next.js 14 (App Router) — Dashboard + Patient UI + API
apps/api/ Python FastAPI — Doctor Agent backend
ios/ iOS + Apple Watch app (HealthKit integration)
packages/shared/ Shared TypeScript types, Zod schemas, LLM abstraction
mcp-server/ Model Context Protocol server for healthcare tools
Doctor_Agent_for_TreeHacks/ Doctor Agent (scheduling, triage, calendar)
pnpm install
cd apps/web && npx prisma db push && cd ../..
pnpm devOpen http://localhost:3000.
The database auto-seeds on first API request with users (pari, alex, dr_smith) and 30 days of health metrics.
Tap "Trigger Alert" on the Apple Watch. Haptic feedback confirms. Synthetic health data (low sleep, high resting HR, irregular heart rhythm) uploads to the backend via POST /api/health-data.
On the Clinical Dashboard (/dashboard/anomaly), the anomaly appears in real-time with:
- Severity score (0–100)
- Flags:
SLEEP_DROP,RHR_SPIKE,STEPS_DROP,SYMPTOM_SPIKE - Agent's autonomous triage decision and urgency level
- Live patient metrics (auto-refreshes every 30s)
Click into any alert to view the full chain:
- Anomaly Detection — flag detection and scoring from wearable data
- Secretary Agent Reasoning — LLM-powered analysis with tool-calling
- PubMed Evidence Lookup — clinical studies and guidelines from NCBI
- Triage Decision — intake questions, urgency scoring, escalation
- Appointment Booking — slot negotiation and calendar integration
Each step is logged with actor, event type, timestamp, and full data payload.
At /patient/call, the patient speaks with Dr. Smith — a HeyGen streaming avatar:
- Context-aware greeting referencing the specific anomaly (elevated HR, low sleep)
- Real-time voice conversation via Deepgram STT + HeyGen TTS
- AI doctor responses generated by Claude with the full health context
- Live transcript and clinical entity extraction (symptoms, medications, allergies)
- Red flag detection and automated triage actions
- 12-language support
The patient's agent and doctor's agent negotiate appointment slots autonomously:
- Scheduler Agent checks Google Calendar availability
- Doctor Agent proposes slots from the doctor's calendar
- Patient Agent accepts and confirms
- Booking appears on Google Calendar and in the patient's appointments view at
/patient/appointments
Copy .env.example to .env.local:
cp apps/web/.env.example apps/web/.env.local| Variable | Required | Description |
|---|---|---|
ANTHROPIC_API_KEY |
Yes | Claude LLM for agent reasoning, voice responses, entity extraction |
OPENAI_API_KEY |
No | Alternative LLM provider (GPT-4o-mini) |
HEYGEN_API_KEY |
Yes | Streaming avatar for Dr. Smith voice consultations |
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY |
Yes | Clerk authentication |
CLERK_SECRET_KEY |
Yes | Clerk server-side auth |
SUPABASE_URL |
Yes | Supabase project URL for health data storage |
SUPABASE_SERVICE_ROLE_KEY |
Yes | Supabase service role key |
DATABASE_URL |
Yes | Postgres connection string (Supabase) |
GOOGLE_CLIENT_ID |
No | Google Calendar OAuth |
GOOGLE_CLIENT_SECRET |
No | Google Calendar OAuth |
NEXT_PUBLIC_BASE_URL |
No | Defaults to http://localhost:3000 |
- Push to GitHub
- Import in vercel.com/new
- Set root directory to
apps/web - Add environment variables (see table above)
- Deploy — the build runs
prisma generate && next build
| Endpoint | Method | Description |
|---|---|---|
/api/health-data |
POST | Receive health data from iOS/Watch, run anomaly detection |
/api/trigger |
POST | Secretary Agent — route health triggers to sub-tools |
/api/anomaly/live |
GET | Live patient metrics for the clinical dashboard |
/api/anomaly/history |
GET | Alert history with filtering and pagination |
/api/voice/respond |
POST | Generate doctor responses + extract clinical entities |
/api/heygen/token |
POST | Create HeyGen streaming session tokens |
/api/calendar/events |
GET | Fetch calendar events for a patient |
/api/patient/agent |
POST | Patient agent stub (receives slot proposals from Doctor Agent) |
/api/demo/trace/:traceId |
GET | Retrieve full trace for inspection |
The Secretary Agent is an LLM-powered orchestrator that receives health triggers and autonomously chains tools:
analyze_anomaly— evaluates health anomaly severity and urgencylookup_clinical_evidence— searches PubMed and clinical guidelinestriage_patient— runs intake questions, books appointment if neededget_health_summary— retrieves 30-day health trendsschedule_appointment— finds available slots and books via Google Calendarnotify_doctor_agent— alerts the Doctor Agent for scheduling negotiation
Supports both Anthropic (Claude) and OpenAI (GPT-4o) with automatic fallback.
- HealthKit Integration — reads steps, heart rate, sleep, active energy, workouts, health events
- Trigger Alert — sends synthetic anomaly data to the backend for pipeline testing
- Haptic Feedback — success/failure haptics on the Watch
- Auto-Sync — periodic health data uploads
- Monorepo: pnpm workspaces
- Web: Next.js 14 (App Router), React 18, TypeScript, Tailwind CSS
- Auth: Clerk
- DB: Supabase (Postgres) + Prisma ORM
- LLM: Anthropic Claude (Messages API + Agent SDK), OpenAI GPT-4o
- Voice: HeyGen Streaming Avatar, Deepgram STT, ElevenLabs TTS
- iOS: Swift, SwiftUI, HealthKit, WatchKit
- Signing: Ed25519 via tweetnacl
- Clinical: PubMed E-utilities, AHA/CDC/WHO guidelines
- Calendar: Google Calendar API (OAuth 2.0)
- Schemas: Zod