Overview

MIT Media Lab's NeuroSkill (Kosmyna & Hauptmann, March 2026) introduces a first-of-its-kind proactive agentic system that models a user's "Human State of Mind" in real-time by integrating Brain-Computer Interface (BCI) signals with foundation models. Its custom harness, NeuroLoop, implements a per-turn signal detection → context injection → proactive engagement loop that enables the agent to respond to both explicit commands and implicit human states — fatigue, stress, emotional shifts, cognitive load — without being asked.

The core insight is architectural, not hardware-specific: most AI agents are purely reactive (wait for explicit request → respond). NeuroSkill demonstrates a general pattern where the agent continuously senses signals, injects relevant external context into every LLM turn, and proactively engages based on detected state. While their implementation uses BCI hardware, the pattern generalizes to any signal source: text-based emotional/cognitive detection, time-of-day awareness, system metrics, calendar events, activity patterns, IoT sensors, or health wearables.

This feature proposes implementing NeuroSkill's proactive context loop architecture in Hermes Agent — a pluggable framework for external context sources that inject state into the agent's system prompt, combined with signal detection in user messages that triggers contextual data fetching and proactive behavior. This would make Hermes one of the first general-purpose agents to support proactive, state-aware engagement.

Research Findings

How NeuroSkill/NeuroLoop Works

The system has two components: NeuroSkill (the data layer) and NeuroLoop (the agentic harness).

NeuroSkill — Signal Acquisition & Embedding

NeuroSkill is a Tauri/Rust desktop app that:

1. Streams EEG from consumer BCI wearables (Muse, OpenBCI, AttentivU) via BLE/WiFi/USB at 256 Hz
2. Processes signals with GPU-accelerated FFT (Hann-windowed 512-sample across 4 channels at ~4 Hz)
3. Creates neural embeddings using ZUNA, a GPU-accelerated transformer encoder that converts 5-second EEG epochs into 32-dimensional vectors
4. Stores locally in SQLite + HNSW (Hierarchical Navigable Small World) indexes for similarity search
5. Exposes via WebSocket API (_skill._tcp mDNS discovery) with commands: status, search, compare, sleep, sessions, labels

Computed metrics include: EEG band powers (delta through gamma), composite scores (Focus = β/(α+θ), Relaxation = α/(β+θ), Engagement), complexity measures (Hjorth, permutation entropy, fractal dimension), cardiac metrics (HR, HRV, SpO2), and research indices (ADHD, anxiety, depression risk indicators).

All processing runs fully offline on the edge — no cloud, no data harvesting.

NeuroLoop — The Proactive Agentic Harness

NeuroLoop is a Node.js CLI agent (built on the Pi coding agent framework) that implements the proactive engagement loop:

Per-Turn Processing Pipeline (before every LLM call):

User Message
    │
    ├─ 1. Poll EXG Status ─────────── neuroskill status (10s timeout, graceful fallback)
    │
    ├─ 2. Detect Signals ──────────── 36 regex-based domain detectors on user text
    │     │
    │     ├─ sleep, stress, focus, mood, grief, anger, loneliness...
    │     ├─ social, dating, family, therapy, goals, creative...
    │     ├─ consciousness, philosophy, existential, identity...
    │     └─ protocols (explicit practice requests)
    │
    ├─ 3. Fetch Contextual Data ───── Parallel execution of matched commands
    │     │                            (session metrics, label searches, sleep data,
    │     │                             compare cache, protocol repertoire)
    │     └─ Deduplicates commands, caps searches, uses 10-min TTL cache
    │
    ├─ 4. Read Memory ─────────────── ~/.neuroskill/memory.md (persistent across sessions)
    │
    ├─ 5. Read Capability Index ───── NEUROLOOP.md (skill catalog)
    │
    ├─ 6. Calibration Check ───────── Once-per-day nudge (invisible to user)
    │
    └─ 7. Assemble System Prompt ──── STATUS_PROMPT guidance + live EXG data
                                       + signal-matched context + memory + skills

The 36 Signal Categories (key architectural detail):

Each signal is a set of regex patterns with word boundaries and fuzzy proximity matching. Categories span:

When a signal fires, it triggers domain-specific data fetches (session metrics, label searches with domain-specific terms, sleep staging data, compare computations). Results are injected as labeled text blocks into the system prompt — domain-aware RAG without embeddings.

Proactive Behavior Patterns:

The STATUS_PROMPT guidance (injected every turn, invisible to user) instructs the LLM to:

• Infer felt quality from metrics (contracted/open, turbulent/still) without mentioning raw numbers
• Auto-label notable states silently (grief, awe, flow, insights) via the labelling API
• Propose protocols when appropriate (but never impose — "one at a time, no repetition, calibrate to state")
• Proactively prewarm compare cache when trends are mentioned
• Send OS notifications for high drowsiness, stress, or user-requested alerts
• Never mention EXG/metrics unless the user asks — the agent acts on the data but doesn't expose it

Layered Information Architecture:

• User sees: Clean EXG snapshot + memory (display layer)
• LLM sees: Behavioral guidance + EXG data + signal-matched context + memory + nudges (system layer)
• Display and system content are decoupled — the agent knows more than it shows

Key Design Decisions

1. Signal detection is regex, not ML — Fast, deterministic, no false positives from model hallucination. 36 categories with 3-12 patterns each cover a wide domain space.
2. Parallel data fetching — All signal-matched commands execute concurrently via Promise.all, bounded by deduplication and search caps.
3. On-demand skill loading — The 634-line protocol repertoire is only injected when the protocols signal fires, saving context window.
4. Graceful degradation — Every external call has a 10s timeout. Failed calls are silently skipped. Cache misses trigger background builds. WebSocket auto-reconnects. Missing memory = no-op.
5. Compare cache with TTL — Expensive cross-session comparisons are cached for 10 minutes and prewarmed proactively.
6. Extension architecture — Built as an ExtensionFactory with hooks: before_agent_start, session_start, session_shutdown. Clean separation from the base agent.

Software & Licensing

• NeuroSkill app: github.com/NeuroSkill-com/skill — GPLv3
• NeuroLoop harness: github.com/NeuroSkill-com/neuroloop — GPLv3
• SKILL.md files: AI100 license (ethical alignment)

⚠️ GPLv3 is copyleft — cannot import code directly into Hermes Agent (MIT). Must implement from architectural patterns, not source code.

Current State in Hermes Agent

What We Have

What's Missing

1. Per-turn context injection framework — No hook to inject dynamic external state before each LLM call
2. Signal detection in user messages — No mechanism to detect implicit user states from text patterns
3. External context source registry — No pluggable system for registering and polling external data sources
4. Proactive engagement patterns — Agent never initiates actions based on detected state
5. Contextual skill/data loading — Skills loaded by explicit name match, not by detected domain signals

Relevant Existing Issues

• Feature: PAHF Personalization Loop — Pre-Action Clarification, Preference Grounding & Post-Action Feedback Integration #362 — PAHF Personalization Loop: Complementary — PAHF learns preferences from feedback; this senses state in real-time. PAHF's preference grounding + this issue's signal-driven context injection would create a powerful combination.
• Feature: Structured Memory System — Typed Nodes, Graph Edges, and Hybrid Search #346 — Structured Memory System: Infrastructure prerequisite for Phase 3 (semantic signal-to-memory retrieval).
• Feature Request: Segmented Motivational Systems #90 — Segmented Motivational Systems (Dynamic Parts): Related concept — internal agent perspectives that activate based on context. Could be informed by external signals from this feature.

Implementation Plan

Skill vs. Tool Classification

This should be a core codebase enhancement (not a skill or tool) because:

• It modifies the agent's per-turn processing pipeline (prompt_builder.py, agent run loop)
• It adds new infrastructure: context source registry, signal detector, injection hooks
• It changes how the system prompt is assembled dynamically based on external state
• Skills cannot modify the agent's behavioral loop or inject context before LLM calls
• The signal detection and context injection must execute deterministically on every turn

The individual context sources (BCI, calendar, system metrics) would be skills or MCP servers that plug into this framework.

What We'd Need

1. Context Source Protocol — Interface for external data providers (poll/subscribe, schema, TTL)
2. Signal Detector — Regex-based text analyzer for implicit user state detection
3. Context Injection Hook — New stage in prompt assembly that queries active context sources
4. Proactive Behavior Guidance — System prompt additions for signal-aware agent behavior
5. Configuration — User-facing config for enabling/disabling context sources and signal categories

Phased Rollout

Phase 1: Signal Detection & Contextual Skill Loading (Behavioral Enhancement)

• Implement a lightweight signal detector inspired by NeuroLoop's 36-category system, adapted for general use (not BCI-specific). Categories like: frustration, urgency, confusion, learning, creative, planning, debugging, emotional, health, time-sensitive
• Add signal detection to the per-turn pipeline: before skill matching, scan user message for implicit signals
• When signals fire, proactively load relevant skills and inject their context into the system prompt (e.g., detecting "exam stress" loads a study/focus skill)
• Add PROACTIVE_GUIDANCE to prompt_builder.py instructing the agent to:
  • Acknowledge implicit states without being told (e.g., sensing frustration → adjust tone)
  • Suggest relevant actions/resources based on detected signals
  • Never mention the detection mechanism to the user
• Deliverable: Agent detects and responds to implicit user states from text alone
• Effort: Medium (~200-300 lines in prompt_builder.py + new signal_detector.py)

Phase 2: External Context Source Framework (Infrastructure)

• Define a ContextSource protocol/interface:

  class ContextSource:
      name: str
      poll_interval: float  # seconds, 0 = on-demand only
      async def poll(self) -> Optional[str]  # returns context text or None
      async def on_signals(self, signals: dict[str, bool]) -> Optional[str]  # signal-triggered fetch

• Implement a ContextSourceRegistry that manages sources, polls on schedule, caches with TTL
• Add built-in context sources:
  • TimeContext: Current time, day of week, timezone (enables "it's late, you might be tired" awareness)
  • SystemContext: CPU/memory load, disk space, running processes (enables "your system is under load" context)
  • ActivityContext: Recent git commits, file modifications, session duration (enables "you've been working for 4 hours" awareness)
• Inject aggregated context into system prompt via new build_context_sources_prompt() in prompt_builder.py
• Add configuration in ~/.hermes/config.yaml:

  context_sources:
    time: true          # time-of-day awareness
    system: false       # system metrics (opt-in)
    activity: false     # activity patterns (opt-in)
    custom: []          # user-defined MCP servers as context sources

• Deliverable: Pluggable framework for injecting external state into every LLM turn
• Effort: Large (~500-700 lines: context_source.py, registry, built-in sources, config integration)

Phase 3: Proactive Engagement & Advanced Sources (Polish)

• Implement proactive engagement patterns:
  • Nudges: Agent-initiated suggestions based on detected state ("You've been debugging this for a while — want me to try a different approach?")
  • Anticipatory actions: Precompute likely next steps based on activity patterns
  • State-aware tone adjustment: Automatically soften/energize tone based on detected user state
• Add advanced context sources:
  • CalendarContext: Upcoming meetings, deadlines (via Google Workspace skill or MCP)
  • WeatherContext: Local weather for mood/energy awareness
  • HealthContext: Integration with NeuroSkill BCI via WebSocket API, or fitness wearable APIs
  • NotificationContext: Unread messages, pending PRs, CI/CD status
• Implement signal-to-memory bridge: when signals fire, proactively search session history for relevant past context (builds on Feature: Structured Memory System — Typed Nodes, Graph Edges, and Hybrid Search #346)
• Add per-turn context budget: cap total injected context to prevent prompt bloat (e.g., 2000 chars max from all sources combined)
• Deliverable: Fully proactive agent that senses, anticipates, and engages based on user state
• Effort: Large (~800+ lines, depends on Phase 2 and available context source integrations)

Pros & Cons

Pros

• First-mover advantage — No general-purpose AI agent does proactive context injection today. Most are purely reactive. This would differentiate Hermes significantly.
• Generalizable architecture — The pattern works with ANY signal source, from simple (time of day) to exotic (BCI). Start simple, extend over time.
• Immediate value from Phase 1 — Just detecting implicit user states from text and adjusting behavior requires zero external dependencies.
• Complements existing features — Works with PAHF (Feature: PAHF Personalization Loop — Pre-Action Clarification, Preference Grounding & Post-Action Feedback Integration #362) for preference-aware proactivity, with structured memory (Feature: Structured Memory System — Typed Nodes, Graph Edges, and Hybrid Search #346) for signal-to-memory retrieval, with MCP for external data transport.
• User autonomy — All context sources are opt-in via config. Privacy-sensitive sources (health, activity) default to off.
• Graceful degradation — Following NeuroLoop's pattern: every external call has a timeout, failures are silently skipped, the agent works fine with zero sources enabled.
• Academically grounded — The pattern is validated by MIT Media Lab research with real users and formal evaluation.

Cons / Risks

• Context window pressure — Every per-turn injection consumes context. Must implement strict budgeting (NeuroLoop mitigates this with on-demand skill loading and signal gating).
• Latency — Polling external sources before every LLM call adds latency. Must be async with timeouts (NeuroLoop uses 10s timeout + parallel fetch).
• Creepiness factor — An agent that "senses" your emotional state without being told could feel invasive. Must be transparent about capabilities and keep detection subtle.
• Signal detection accuracy — Regex-based detection will have false positives/negatives. Too aggressive = annoying irrelevant context; too conservative = no proactivity. Need careful calibration.
• Scope creep — The framework is extensible by design, but each new context source adds maintenance. Must resist the urge to add every possible source.
• GPLv3 constraint — Cannot copy NeuroLoop code. Must implement from architectural understanding only. The 36-signal regex set is copyrighted under GPLv3; Hermes must create its own signal categories.

Open Questions

• Should Phase 1's signal detection influence skill matching (proactively loading skills based on detected signals) or just behavioral guidance (adjusting tone/approach)? The former is more powerful but risks loading irrelevant skills.
• How should proactive context interact with PAHF (Feature: PAHF Personalization Loop — Pre-Action Clarification, Preference Grounding & Post-Action Feedback Integration #362)? Scenario: PAHF knows the user prefers concise responses, but signal detection senses frustration → should the agent override conciseness to be more empathetic?
• What's the right default set of signal categories? NeuroLoop's 36 are BCI-oriented. For a general-purpose coding/productivity agent, a different set makes sense (debugging frustration, feature exploration, learning mode, time pressure).
• Should context sources be MCP servers? Using MCP as the transport protocol for context sources would leverage existing infrastructure but adds overhead for simple sources (time, system metrics).
• What's the context budget per turn? NeuroLoop injects everything that matches. For Hermes with a typical 32K-128K context, how much budget should external context get?
• Should the signal detector be configurable per user? Power users might want custom signal categories; casual users might want none.

References

• NeuroSkill Paper — arXiv:2603.03212 (Kosmyna & Hauptmann, MIT Media Lab, March 2026)
• NeuroSkill App — github.com/NeuroSkill-com/skill (GPLv3, Tauri/Rust BCI application)
• NeuroLoop Harness — github.com/NeuroSkill-com/neuroloop (GPLv3, Node.js agentic harness)
• MIT Media Lab Project Page
• Pi Coding Agent (base framework for NeuroLoop)
• AI100 License (ethical licensing for SKILL.md)
• Hermes Agent Feature: PAHF Personalization Loop — Pre-Action Clarification, Preference Grounding & Post-Action Feedback Integration #362 — PAHF Personalization Loop (complementary: preference learning)
• Hermes Agent Feature: Structured Memory System — Typed Nodes, Graph Edges, and Hybrid Search #346 — Structured Memory System (infrastructure for signal-to-memory retrieval)
• Hermes Agent Feature Request: Segmented Motivational Systems #90 — Segmented Motivational Systems (related: internal agent perspectives)
• Hermes Agent agent/prompt_builder.py — System prompt assembly (primary integration point)
• DAIR.AI Academy — AI agent education resources