Everly Storybook

Everly is a Gemini-powered storyteller that weaves family photos, videos, notes, and voice memories into a multimodal storybook—helping people with Alzheimer’s reconnect with the life they’ve lived.

Comment

Inspiration

Alzheimer’s and dementia represent one of the fastest-growing global health challenges — not only clinically, but emotionally and economically. Today, more than 55 million people worldwide live with dementia, and that number is projected to triple by 2050. In the U.S. alone, dementia-related care exceeds $300 billion annually, much of it driven by non-medical needs such as repeated reassurance, emotional distress, and caregiver burnout.

Alt text

Despite this, most digital solutions focus on medical monitoring or compliance. Very few address what families consistently identify as the most painful loss: identity, familiarity, and connection to a life once lived.

Alt text

That gap becomes personal with Jane, a 65-year-old living with Alzheimer’s. Jane has a husband, Michael, two children, Emily and Daniel, and two grandchildren, Lily and Noah. While her family remembers a lifetime of shared moments, Jane doesn’t always have access to those memories. What her family wanted wasn’t another reminder app, but a humane way to carry their voices, faces, and stories with her.

Everly was inspired by this need: to preserve connection when memory fades.

What It Does

Everly is a multimodal memory companion, powered by Gemini 3, that helps people with Alzheimer’s reconnect with their life story.

Families collaboratively upload:

  • Photos

  • Audio messages

  • Good old songs

  • Short videos

  • Simple written memories

Everly organizes these into a living storybook that:

  • Plays automatically with voice narration

  • Flips pages on its own, like a video

  • Can be interrupted naturally for conversation

  • Answers questions using only family-approved memories

Everly is voice-first, calm, and requires no learning or navigation.

The meaning behind the name Everly:

Everly was chosen as the name because it feels like “ever” — something lasting, continuous, and always there — which reflects the purpose of the app: preserving identity even when memory fades. It also sounds gentle and human, like a person’s name, which matters because Everly isn’t meant to feel like a tool or a medical device. It’s meant to feel like a companion.

Alt text

In the context of Alzheimer’s, where memories can become fragmented or disappear, Everly represents the idea that love, family, and personal story can still remain—even if they show up in small moments.

That meaning connects directly to the logo too: the soft spinning shape with no concrete form, sometimes fading, reflects how memory can be fluid, incomplete, and drifting, yet still beautiful and present.

How We Built It

Alt text

Everly System Architecture

Everly is a multimodal memory reconstruction system that transforms fragmented life media into structured, emotionally safe narratives for dementia patients.

The architecture operates in two modes:

  • Caregiver Mode — story creation and memory reconstruction
  • Patient Mode — accessible delivery through a guided story interface

Caregiver Mode — Creation Studio

The Creation Studio is a guided environment where caregivers build a patient's life narrative from fragmented media.

Memory Studio

Caregivers upload:

  • photos
  • videos
  • notes
  • voice recordings

Instead of simple labeling, Gemini 3 analyzes the media and asks contextual questions to reconstruct missing narrative details.

Example prompts:

Who is the baby in this photo?
Was this before or after your wedding?

The system converts raw uploads into structured story-ready memories.

Management

Caregivers configure the support environment:

  • patient profile and safety context
  • family collaboration
  • medication and hydration reminders

Publishing

When memories are complete, the system compiles them into a structured narrative storybook.

Patient Mode — Delivery Experience

The patient interface is designed for maximum cognitive ease and emotional grounding.

Interactive Storybook

Memories are presented inside a guided story viewer featuring:

  • auto-turning pages
  • cinematic scene introductions
  • paced narration

Each story reinforces orientation with reminders such as:

You are at home. Today is Tuesday morning.

Safety Sidebar

Persistent orientation cues include:

  • time
  • date
  • location
  • emergency contact button

Core AI Pipeline

Everly uses Gemini 3 as a multimodal reasoning engine, transforming fragmented media into a coherent life narrative.

Pipeline flow:

Creator Inputs → Gemini 3 Pro → Sequence Planner → Gemini 3 Flash → Media Buffer → Story Renderer

Creator Inputs

Caregiver content forms the input layer:

  • text prompts
  • image references
  • video clips
  • audio context
  • patient profile data

Gemini 3 Pro — Creative Direction

The first stage uses gemini-3-pro to act as an AI biographer.

Responsibilities:

  • analyze all caregiver inputs
  • construct narrative structure
  • generate scene descriptions
  • create illustration prompts

This stage performs deep reasoning and narrative planning.

Interleaved Sequence Planner

The planner prepares structured prompts for multimodal synthesis by organizing:

  • narrative segments
  • illustration prompts
  • story pacing

This stage determines where visuals should appear within the narrative.

Gemini 3 Flash — Multimodal Synthesis

The second stage uses gemini-3-flash to generate the story.

Unlike traditional pipelines, Flash produces interleaved multimodal output.

Instead of generating text and images separately, the model returns a single sequential stream of narrative and visuals.

Interleaved Multimodal Output

A core innovation of Everly is interleaved storytelling.

The AI generates a synchronized sequence such as:

  1. narrative paragraph
  2. illustration of the scene
  3. narrative continuation
  4. supporting illustration

The Core Engine: Gemini 3 Multimodal Reasoning

This system transforms fragmented media into a "life stream" for dementia patients, utilizing Gemini 3 as a multimodal reasoning engine rather than a simple chatbot. It bridges the gap between raw uploads (photos, vague videos, notes) and structured, emotionally safe narratives. The system uses the gemini-3-pro-preview model in the service layer to perform three high-order reasoning tasks: The AI Interviewer (processCaptionTurn): Instead of generic labeling, the model identifies missing context (e.g., "Who is the baby?") and interviews caregivers to fill narrative gaps. The Biographer (compileBookNarrative): It synthesizes the entire "Memory Circle" into a structured JSON narrative, ensuring emotional consistency and safety-first messaging. The Expert Companion (createCaregiverChat): A consultant-style interface providing dementia-care advice grounded specifically in the patient's personal life data.

Why Gemini 3? The product shifts from a "media gallery" to a "memory reconstruction system" because Gemini 3 can: Fuse Cross-Modal Context: Aligning a text note from 1995 with a blurry photo from the same era. Handle Long Context: Maintaining narrative consistency across dozens of fragmented memories. Ensure Safety: Structuring outputs to be gentle, repetitive, and grounding for those with cognitive decline. While Gemini 3 Pro serves as the "Biographer" (handling complex, slow-burn reasoning), Gemini 3 Flash is the "Voice" and "Conversationalist." We specifically migrated the patient-facing interactions to Gemini 3 Flash to ensure the experience feels human and safe.

Why We Use Gemini 3 Flash for Voice For a dementia patient, even a two-second delay in conversation can cause confusion or anxiety. Gemini 3 Flash is built for low-latency, high-frequency interactions.

Feature Gemini 3 Pro (The Brain) Gemini 3 Flash (The Voice)
Primary Goal Deep Multimodal Reasoning Speed & Fluid Interaction
Response Time Slower (2–5+ seconds) Ultra-Fast (Sub-second TTFT)
Voice Persona Detailed & Analytical Natural & Low-Latency
Context Window 2M+ Tokens 1M Tokens (Optimized for scale)
Token Efficiency Standard 30% more efficient (less "wasted" thought)

Because Everly is a supportive, non-medical companion, it requires no FDA approval and can be deployed through existing consumer platforms such as Google Home devices.

Challenges We Ran Into

The hardest challenge was designing for emotional safety.

We had to ensure:

  • Everly never invents memories

  • Repetition never feels frustrating

  • Instructions don’t overwhelm

  • Voice interaction feels natural, not robotic

Technically, managing uninterrupted narration alongside natural voice interruption required careful prompt design and state control. We originally tried to run everything through the most powerful model (Gemini 3 Pro). However, the "thinking time" created a 3-5 second silence. For a person with memory loss, those three seconds are a "cognitive cliff"—by the time the AI spoke, the patient had forgotten they asked a question. This is why we pivoted to Gemini 3 Flash; we sacrificed a tiny bit of "deep reasoning" for sub-second speed, which is much more important for maintaining a patient's flow.

Accomplishments That We’re Proud Of

  • Creating a multimodal storytelling experience that feels human

  • Using Gemini 3 for emotional consistency, not raw intelligence

  • Designing an interface that works without menus or tutorials

  • Making memory-building a family collaboration, not a medical task

  • Delivering a functional, demo-ready product in a short timeframe

What We Learned

We learned that Alzheimer’s care isn’t about fixing memory — it’s about preserving identity and emotional safety.

We learned:

  • Voice-first design outperforms visual complexity

  • Multimodal memories are more powerful than text alone

AI is most impactful when it knows its limits

Gemini 3 enabled us to build something gentle, reliable, and grounded.

We also consulted with nurse Alec Arro, who works in elder-care hospital physical therapy and regularly supports patients with dementia. He also shared that elderly care centers, hospitals, and family members would be highly interested in an app like this, since it supports both patient safety and everyday caregiving needs. Based on his real-world experience, he strongly recommended expanding beyond memory books by adding practical safety-focused features like a Safety Map, Emergency Call, and Daily Reminders. His insight was clear: while memory tools help preserve identity, safety and routine tools help patients stay independent and protected—especially in moments of confusion or disorientation. From that consulting session, we quickly implemented the two new features to the storybook to make it even more beneficial for the intended user.

Alt text

What’s Next for Everly

Next, we plan to:

  • Support multiple languages

  • Enable caregiver-controlled pacing and personalization

  • Expand family collaboration features

Integrate more deeply with home voice assistants

Our long-term vision is simple: to help people like Jane feel remembered, connected, and valued — every day. Everly uses Gemini 3 to turn family memories into a living, multimodal storybook — so even when memory fades, a life well lived is never lost.

Built With

Share this project:

Updates