Aitnemed.AI

AitneMed.ai: An autonomous vocal anchor that uses ElevenLabs voice cloning to restore identity and comfort to those lost in the silence of dementia.

Comment

Inspiration

Over 55 million people worldwide live with dementia, yet most of their waking hours are spent without access to trained dementia-care support. Caregivers — often family members with no clinical background — face an impossible task: communicating with a loved one whose language, memory, and sense of self are fragmenting hour by hour.

We kept coming back to one truth: a person with dementia has not lost their story. Their long-term autobiographical memory — the childhood home, the spouse's name, the career they were proud of — is often preserved far longer than short-term memory. What they lose is access to it. We wanted to build something that could hold that story for them and give it back, in their own language, in a voice they recognize, whenever they needed it.

What it does

AitneMed.AI is an ambient AI companion for elderly dementia patients. It listens to the patient speak in their native language, understands the emotional subtext of what they're communicating, and responds with clinically-grounded warmth — in the patient's own language, optionally in the cloned voice of a loved one.

  • Multilingual voice conversation — real-time speech-to-text and text-to-speech across 10+ languages including Vietnamese, Hindi, Tagalog, Arabic, and Mandarin
  • Life story integration — caregiver-authored biographical data is injected into every conversation, enabling the AI to anchor patients to their own identity during moments of confusion
  • Voice cloning — families can record a loved one's voice; the AI speaks back in that voice during sessions
  • Clinical dementia protocols — responses follow Validation Therapy, the VERA Framework, Person-Centred Care, Reminiscence Therapy, and the SPECAL Method
  • Care alert system — when the patient asks for water, medication, or bathroom help, or escalates into distress, a real-time alert is sent to the caregiver dashboard
  • Caregiver dashboard — live alerts with urgency tiers, timestamps, and dismiss controls

How we built it

  • Frontend: Next.js 14 with Tailwind CSS — deep-purple UI designed for senior readability (large text, high contrast, generous spacing)
  • AI: Groq LLaMA 3.3-70b for fast multilingual chat with a clinically structured system prompt encoding dementia-care frameworks and patient biographical data
  • Speech-to-text: Browser Web Speech API for zero-latency multilingual transcription
  • Text-to-speech: ElevenLabs eleven_multilingual_v2 with language-matched voice IDs per language, and browser window.speechSynthesis as a guaranteed fallback so the patient always hears a voice
  • Voice cloning: ElevenLabs Instant Voice Cloning API — families record audio in-browser, we submit to ElevenLabs and store the voice ID for the session
  • Care alerts: AI prefixes responses with [ALERT:type] when physical needs are detected; parsed client-side, displayed in the caregiver dashboard in real time
  • Backend: Next.js API routes with Prisma ORM and SQLite, NextAuth for authentication

Challenges we ran into

The translator trap. Our first iterations produced an AI that translated correctly but had no dementia-care instincts. It gave philosophical deflections when patients expressed identity confusion — "it matters where you are in your heart" — instead of immediately anchoring them with biographical facts. We rebuilt the entire system prompt around clinical frameworks and explicit failure-mode examples.

Dialect and pronunciation fidelity. ElevenLabs clones trained on English-accented samples produced incomprehensible output in Vietnamese or Hindi. We solved this by routing TTS through language-matched native ElevenLabs voices, reserving the cloned voice only for the patient's recorded language.

API reliability under pressure. Groq's free-tier daily token limit, ElevenLabs 400 errors on newly-cloned voices, and an invalid Google TTS key all hit during the hackathon. We built a layered fallback chain — ElevenLabs → browser speech synthesis — so the patient always hears a response regardless of API status.

Duplicate audio from React StrictMode. The opening nurse greeting was firing twice in development due to React's double-invocation of useEffect. Fixed with a startedRef guard.

Accomplishments that we're proud of

  • A fully working real-time multilingual voice pipeline — record, transcribe, respond, speak — with silent fallbacks at every layer
  • Biographical anchoring that actually works: when a patient says "I forgot where I'm from," the AI immediately provides the answer from their life story
  • Voice cloning fully integrated in-browser — record, upload, and speak in a loved one's voice in under a minute
  • A care alert system that bridges AI conversation and real human response — water, medication, bathroom, and distress requests reach the caregiver instantly
  • Five distinct clinical dementia-care frameworks encoded into coherent, testable AI behavior

What we learned

Building for dementia care forced us to confront the difference between correct and therapeutic. A factually accurate response can be deeply harmful — telling a patient their parent has died causes fresh grief every single time. The most important engineering decisions were not in the code but in the clinical design: what the AI is forbidden to say matters as much as what it is told to say.

We also learned that multilingual AI requires far more than translation. Dialect, script, and vocal prosody each carry emotional weight. A Vietnamese response without full diacritical marks is not Vietnamese — it is noise. Getting these details right required language-specific voice routing and script enforcement at the prompt level.

Fallback design is user safety design. Every API failure in this system is not a minor inconvenience — it is a vulnerable elderly person hearing silence when they needed comfort.

What's next for Aitnemed.AI

  • Separate caregiver accounts with SMS push notifications (Twilio) for true remote monitoring from a different device
  • Longitudinal mood tracking — stress-level time series on the caregiver dashboard to surface sundowning windows and trigger patterns over days and weeks
  • Proactive scheduled check-ins — AI-initiated morning greetings, medication reminders, and family visit announcements
  • Facility deployment — multi-patient, multi-caregiver architecture for nursing homes and memory care units
  • Clinical outcome research — partnership with dementia care researchers to measure agitation frequency and quality-of-life indicators against a control group

Built With

Share this project:

Updates