main page
user dashboard
support page

Learno - A real-time social context support for neurodivergent teens in the classroom

Inspiration

Navigating school is hard. For neurodivergent students, those with autism, ADHD, anxiety, dyslexia, or sensory processing differences, it can feel overwhelming.

The numbers are alarming:

44% of students in grades 7–12 report moderate-to-serious psychological distress, up from 24% in 2013 [(PMC, 2023)] link
17% of university students identify as neurodivergent, yet less than half disclose their diagnosis to their institution [(University of Calgary, 2024)] link
Students don't disclose due to fear of stigma, inflexible processes, and diagnostic barriers [(University Affairs Canada, 2024)] link
Neurodivergent teens experience a heavier emotional burden at school, leading to higher rates of anxiety and depression [(The Conference Board of Canada, 2024)] link

What it does

Learno is a desktop application that analyzes real-time classroom audio to detect conversational tone and social context. It combines this with individualized user patterns to deliver subtle, non-intrusive support during potentially stressful moments, without drawing attention to the student. The system is designed to enhance accessibility rather than "fix" behavior. It listens, interprets, and quietly reassures, the way a calm, understanding friend might sit beside you and whisper, "That's not about you."

Core Features

🎙️ Real-Time Audio Analysis: Analyzes classroom audio using Google's Gemini Live API to detect conversational tone, teacher emotions (calm, urgent, frustrated), speech patterns, and environmental triggers like sudden laughter or rapid pace changes.
😊 Facial Emotion Detection: Optional mood tracking through facial analysis to provide personalized emotional support.
🔍 Context Clues: Live lesson transcription with social context interpretation—translating ambiguous phrases and social cues into clear, reassuring explanations.
💙 Silent Interventions: Haptic feedback and calming prompts delivered privately when stressors are detected, without drawing attention.
📝 Post-Class Recaps: AI-generated summaries at adjustable reading levels (Grade 6-10) to help students review what they learned.
📊 Weekly Dashboard: Pattern tracking that identifies when students feel calmer or more stressed, helping them understand their emotional landscape without judgment.
🧑‍🏫 Teacher Communication: Live feedback from student to teacher to keep track of pace, confusion, and additional help needed.

How we built it

Frontend Architecture

Vite + React + TypeScript for a fast, type-safe development experience.
Tailwind CSS for responsive, accessible design.
Framer Motion for subtle animations that enhance UX without overwhelming.
PWA capabilities with service workers for offline support and add-to-home-screen functionality.

Backend Services

Node.js + Express + TypeScript API server.
MongoDB Atlas for storing user profiles, sessions, and analytics.
Google Gemini Live API for real-time audio streaming, transcription, and social context analysis.
ElevenLabs API integration for voice synthesis and analysis.
face-api.js for analyzing emotion via webcam in real time.
Web Audio API for browser-based audio capture and processing.

AI Implementation

We implemented a multi-layered AI approach:

Audio Analysis Service: Real-time streaming to Gemini for tone detection and transcription.
Social Context Service: Interprets transcripts to identify emotionally loaded phrases and provides reassuring context.
Facial Analysis Service: Optional visual mood tracking.
Adaptive Intervention System: Combines audio, facial, and contextual signals to determine when and how to provide support.

Challenges we ran into

Technical Challenges

Real-Time Audio Streaming: Implementing bidirectional audio streaming with Gemini Live API required careful buffer management and WebSocket handling to minimize latency.
Privacy-First Design: Building an audio-based classroom support tool while maintaining strict privacy boundaries, we process audio locally where possible and minimize data retention.
API Migration: We initially explored multiple APIs for transcription but migrated to the Gemini Live API for more unified real-time analysis and better integration with social context understanding.

Accomplishments that we're proud of

🎯 Real-Time Multi-Modal Analysis: Successfully integrated audio transcription, tone analysis, and facial emotion detection into a cohesive support system.
🧠 Contextual Intelligence: Built a social context interpretation system that translates ambiguous classroom phrases into reassuring explanations (e.g., "We'll talk later" → "They may be busy, not mad")..
📱 Full-Stack Integration: Created a production-ready application with frontend, backend, database, and multiple AI service integrations.
💪 Neurodivergent-Centered Design: Treating accessibility as a core feature, not an afterthought, with non-judgmental language and user autonomy at every step.

What we learned

Technical Learnings

Real-time audio processing in web browsers is complex but achievable with proper buffering strategies.
Modern AI APIs (Gemini, ElevenLabs) can be combined to create nuanced, multi-dimensional support systems.
TypeScript across the full stack significantly improves code quality and developer experience.

Human-Centered Design Insights

Effective accessibility tools must be flexible, respectful, and user-driven.
Neurodivergent experiences are highly individual; technology must prioritize personalization, autonomy, and trust.
The most impactful support is often invisible; reducing cognitive load is more important than adding features.
Privacy and transparency are essential for building trust with vulnerable users.

What's next for Learno

Record affirmations in your own voice to play before stressful moments:
- Custom audio recordings for presentations, group work, or raising your hand
- Your voice is the most calming voice; backed by psychology research
Smartwatch integration for even more discreet haptic feedback.
Multi-language support for ESL students.