• 

  Custom Luno Plushie and ESP32 Toy Hardware

• 

  Demo Simulator

• 

  Extensive Hardware Testing and Development

• 

  Luno Toy Hardware

• 

  Parental Companion App Screens

• 

  Parental Companion App Screens

Luno AI Toys

A screen-free AI plushie that turns everyday conversation into learning + emotional growth — with a parent app for safety, personalization, and progress tracking.

---

Inspiration

Kids today are growing up in a world where entertainment is infinite — but attention, curiosity, and emotional growth are shrinking.

In our own families, we watched younger cousins spend hours hopping between short videos — always stimulated, but rarely deeply engaged. Over time, it felt like play stopped teaching and started distracting.

We kept coming back to one question:

If every generation grows up with toys… why can’t this generation grow up with a toy that protects attention, nurtures curiosity, and supports emotional growth — without a screen?

That question became Luno.

---

What it does

Luno is a screen-free AI plush companion that makes conversation feel magical while quietly helping a child grow.

A child talks to Luno like a friend. Luno listens, understands, and replies with a warm voice. Each interaction improves personalization over time — without needing a screen.

Parents stay in control (companion app)

• Set boundaries (topics, time limits, bedtime rules)
• Review session summaries
• Flag anything for review
• Track growth insights (top interests, emerging skills, emotional patterns)

In short:
Child talks to Luno → Luno responds screen-free → Luno learns what helps that child grow.

---

What makes Luno different: the private “Growth Map” (Knowledge Graph)

Most AI toys stop at “a plush that talks.”
Luno is built to improve week-over-week by learning in a structured, explainable way.

After each conversation, Luno updates a private Growth Map — a lightweight knowledge graph that captures patterns like:

What it tracks (nodes)

• Topics the child loves (e.g., Dinosaurs, Space)
• Skills they demonstrate (e.g., counting, storytelling)
• Interests that keep showing up (e.g., science exploration)
• Concepts they’re learning (e.g., gravity, extinction)
• Personality signals (e.g., curious, creative)

How it connects them (edges)

• Co-occurrence: what the child talks about together (Dinosaurs ↔ Fossils)
• Learning pathways: skill progression (Counting to 10 → Counting to 20)
• Emotional associations: what excites or worries them (Space ↔ excitement)

Why this matters (real personalization + parent transparency)

This Growth Map enables:

• Smarter personalization: Gemini responds grounded in what the child truly cares about
• Long-term learning value: Luno gets better over time, not just per session
• Explainability: parents see clear insights, not “black-box vibes”

---

How we built it (end-to-end)

We built Luno as a real product: hardware → voice pipeline → Gemini reasoning → structured memory → parent controls.

System loop (high level)

Plush (ESP32-S3) → Backend Orchestration → Google Cloud (Gemini + Firestore) → Voice Response → Plush

1. Child speaks to Luno
   
2. Luno records audio and sends it to the backend
   
3. Backend runs: STT → Gemini (with safety) → Growth Map update → TTS
   
4. Plush streams back the voice response and plays it (screen-free)

---

Key components

1) Plush Companion (Hardware + Embedded)

Goal: Fast, warm, reliable — it should feel magical, not glitchy.

• ESP32-S3
  
• I2S mic for capture + I2S speaker/DAC for playback
  
• Button to start/stop interactions
  
• BLE for first-time setup + Wi-Fi for real-time conversations
  
• Chunked audio upload + streaming audio playback

2) Orchestration Backend (API Layer)

• Receives audio uploads
• Coordinates STT → Gemini → Growth Map extraction → TTS
• Streams audio back to the plush
• Saves session summaries + growth insights for the parent dashboard

3) Google Cloud Intelligence

Gemini

• Child-friendly tone + short responses
• Safety layer for age-appropriate conversation
• Structured outputs for reliable Growth Map updates

Firestore

• Stores structured memory so personalization compounds over time
• Powers dashboards like top interests, emerging skills, and emotional patterns

---

Challenges we ran into

Reliable structured outputs

We needed consistent machine-readable updates (not just free text).
We solved it with strict schemas + prompt iteration + confidence scoring + fallbacks.

Latency (perceived intelligence)

In hardware AI, latency is everything.
We improved responsiveness with chunked uploads, tight response constraints, streaming playback, and retries/backoff.

Emotional safety that still feels human

Kids don’t respond well to “I can’t answer that.”
Our safety strategy gently deflects + redirects while staying warm and supportive.

Demo reliability

We built a web simulator so judges can experience the same end-to-end pipeline even without physical hardware.

---

Accomplishments we’re proud of

• A working end-to-end loop: plush → AI → voice → plush (screen-free)
• Google Cloud-powered intelligence that goes beyond chat: Gemini + structured memory
• A private, evolving Growth Map that turns conversation into longitudinal insight
• A product-first approach where trust, safety, and reliability are core features
• A judge-friendly simulator path using the same backend + Gemini flow

---

What we learned

• Latency defines perceived intelligence more than model quality.
• Structured memory beats one-off prompting for true personalization.
• Safety must feel supportive, not robotic.
• Parents want agency + transparency, not novelty alone.

---

What’s next

• Pilot with families to validate engagement + emotional safety outcomes
  
• Manufacturing-ready custom PCB
  
• Weekly parent highlights + growth trajectories (skills, interests, emotions)
  
• Deeper graph personalization (interest clusters, learning pathways, reinforcement)
  
• Partnerships with educators and child psychologists
  
• Multiple Luno characters powered by the same safe intelligence engine
  

---

Built with

Google Cloud: Gemini, Firestore, Google AI Studio
Hardware: ESP32-S3, I2S mic, I2S speaker/DAC, BLE, Wi-Fi
Apps: React Native parent companion + web simulator
Backend: API orchestration (STT → Gemini → Growth Map → TTS) + streaming voice pipeline

---

Built With

• amazon-ec2
• c++
• elevenlabs
• esp32
• expo.io
• firebase
• firestore-db
• flask
• gemini
• google-cloud-stt
• javascript
• knowledge-graph
• llm
• python
• react
• react-native
• typescript