NeuroTrace

Inspiration

Every time you speak, your brain leaves a fingerprint.

Cognitive decline, ADHD, early-onset depression, and dementia all embed detectable signals in the way a person talks — years before a clinical diagnosis. The type-token ratio in your vocabulary. The coherence between your sentences. The frequency of your hesitations. These aren't abstract metrics; they're real neurological markers that researchers like Snowdon et al. (2001) showed can predict Alzheimer's decades before symptoms emerge.

But this science has never escaped the lab. Nobody has made it visible, interactive, or accessible to a non-clinical audience. A clinician in a rural area has no fast way to flag a concern. A family member can't point to anything concrete. The linguistic fingerprint exists — it's just invisible.

We wanted to change that. We wanted to build something that lets anyone speak for 90 seconds, and immediately see their cognitive signature light up a 3D brain.

What it does

NeuroTrace is an agentic AI pipeline that maps speech into neuroscience.

You speak — or paste text — and NeuroTrace extracts 30 linguistic biomarkers across five cognitive domains:

Lexical — type-token ratio, lexical density, filler word rate → Broca's area
Semantic — coherence score, idea density, tangentiality → Wernicke's area
Prosody — speech rate, pause frequency, hesitation ratio → SMA / basal ganglia
Syntax — mean length of utterance, clause depth, passive voice → DLPFC
Affective — valence, arousal, certainty language → Amygdala

Each domain maps to a real MNI152 neuroimaging coordinate — the same coordinate space used in clinical fMRI research. The result is a 3D brain that glows in proportion to what your speech reveals, with interactive region panels explaining the science behind every activation.

A cited narrative report generates at the end, grounded in DementiaBank corpus studies, Elvevåg et al. (2010), and LIWC (Pennebaker et al.) — so every insight traces back to published neuroscience.

How we built it

We built a fully containerized stack with a clear separation of concerns across a four-person team.

Frontend — Next.js 15 (App Router), React 19, Tailwind v4, TypeScript strict. The UI uses a glass morphism design system with a WebGL dithered background, smooth two-phase layout transitions, and a Three.js 3D brain viewer with OrbitControls, activation sphere rendering at exact MNI coords, and HTML overlay panels on click.

Speech-to-Text — Whisper API with verbose_json and timestamp_granularities[]=word. We extract a pause map from word-level timestamps (gaps > 100ms between words), which feeds directly into the prosody analysis. The waveform panel visualizes the transcript with interactive pause markers.

NLP Sidecar — FastAPI Python service using spaCy (en_core_web_sm) for lexical and syntactic features, sentence-transformers (all-MiniLM-L6-v2) for semantic coherence via adjacent-sentence cosine similarity, and rule-based prosody math from the pause map.

Agentic Pipeline — Langflow orchestrating 7 agents in a fan-out pattern: STT preprocessor, four domain agents running in parallel, a biomarker mapper that normalizes all scores to 0–1 and emits RegionActivation[] JSON, and a report composer using Claude claude-sonnet-4-6 to write cited clinical narrative.

Streaming — The backend proxies Langflow's output as NDJSON line-by-line (type:step, type:token, type:end, type:error). The frontend reads with a ReadableStream reader loop, updating agent step status in real time and lighting up the brain the moment scores arrive.

Challenges we ran into

Langflow was our biggest battle. Langflow is a powerful visual pipeline builder, but it does not behave the way its documentation suggests at the edges. Three issues consumed the majority of our debugging time:

Output schema unpredictability. Each agent's output format varied subtly depending on how nodes were connected. Sometimes scores came back as { lexical: 0.72 }, sometimes as { lexical: { overall: 0.72 } }. We had to write a defensive extractScore() function that handles both shapes, and we wasted hours before we even knew this was the problem — the brain just wasn't lighting up and there was no obvious error.

Streaming vs. non-streaming inconsistency. Langflow claims streaming support, but the behavior changed between our dev and production Langflow instances. We ended up writing a dual-path proxy: pipe the body through if streaming is detected, fall back to a non-streaming response wrapped in our NDJSON end-event format.

Session state. Langflow's session handling is stateful in ways that aren't clearly documented. Reusing a session_id across different inputs sometimes produced stale context bleed. We ultimately treat each analysis as a fresh session unless explicitly continuing a conversation.

The combined effect was that the pipeline worked — Langflow returned data — but the brain showed nothing because the frontend was looking for scores.lexical.overall while the backend was sending scores.lexical. A single console.log("RAW SCORES:", ev.scores) in the type:end handler was the moment everything clicked.

The 3D brain was the other major engineering challenge. We initially built on NiiVue (WebGL MRI atlas viewer), which required a 20MB MNI152_T1_1mm.nii.gz atlas file and a custom voxel overlay generation loop. The approach worked in theory — Gaussian blobs centred on MNI coordinates, painted into a Float32Array, added as an overlay volume — but NiiVue's API is underdocumented and its addVolumeFromBuffer method requires header structures that aren't exposed cleanly. We pivoted to Three.js with a displaced-sphere brain mesh, activation spheres at MNI-projected positions, and PointLight per active region. Simpler, faster, and the visual result is actually more readable for a demo audience.

Coordinate system translation tripped us up between all three systems: MNI space (mm, RAS orientation), NiiVue voxel space (origin at [90, 126, 72] for 1mm isotropic), and Three.js world space. Each translation required careful sign-flipping. The MNI → Three.js mapping that finally worked: x = mni[0] * 0.011, y = mni[2] * 0.011, z = -mni[1] * 0.011.

Accomplishments that we're proud of

The science grounding is real. Every MNI coordinate we use is sourced from published neuroimaging literature. Every biomarker maps to a peer-reviewed mechanism. When a judge asks why Broca's area activates on high filler word rate, we can cite Baddeley (2000) and explain the phonological loop. That traceability matters to us.

The pipeline is genuinely agentic — not a wrapper. Seven Langflow agents with a fan-out architecture, streaming step-by-step status to the UI, updating the 3D brain the moment the biomarker mapper fires. Watching the agents tick from pending to running to done in real time, and seeing the brain light up at the end, is exactly the demo moment we imagined building on day one.

The waveform panel is a detail we're proud of. We visualize the Whisper pause map directly — each detected silence > 100ms appears as an interactive marker on the waveform, linked to the word in the transcript. Clicking a pause marker scrolls to the word that follows it. It turns an abstract number (hesitation_ratio: 0.38) into something you can see and touch.

What we learned

Langflow is powerful but opaque at the boundary between nodes. Treat its output as untrusted data — always inspect the raw payload before building any downstream logic that depends on its shape. The lesson generalises: visual pipeline builders make the happy path fast, but edge-case debugging requires dropping into raw payloads immediately.

Working across four specialisations simultaneously — NLP backend, 3D graphics, agentic pipeline, frontend shell — requires locking interfaces on day one. We defined RegionActivation, BiomarkerScores, and the NDJSON stream protocol in CLAUDE.md on the first morning, and that document became the single source of truth that kept four people unblocked in parallel.

Neuroimaging coordinate systems are genuinely hard. MNI, voxel, and world space are all right-handed but oriented differently. Every axis translation deserves a comment and a unit test.

What's next for NeuroTrace

The immediate step is replacing mock biomarker scores with real NLP — completing the spaCy and sentence-transformers pipeline so the Langflow agents receive actual computed features rather than the demonstration values we used to validate the full stack.

Longer term, NeuroTrace has a clear path toward longitudinal tracking: speak once a week, watch your cognitive signature shift over time. That's where the early detection value becomes real — not in a single snapshot, but in the trajectory. The infrastructure for session management and history is already in the schema.

We also want to publish the MNI coordinate mapping and biomarker taxonomy as an open dataset, so other researchers can use it as a reference implementation for browser-based neuroimaging visualization.## Inspiration