MindMesh

MindMesh Landing Page
MindMesh Join Page
MindMesh Call View Page
MindMesh Canvas View Page

Inspiration

MindMesh started from a very common problem: great ideas happen in live conversations, but the structure of those ideas gets lost almost immediately. In meetings, people speak naturally, interrupt each other, correct themselves, and jump between concepts, but most tools still expect someone to manually take notes or build diagrams afterward. We wanted to make that process feel automatic and immediate.

Our goal was to build a tool that listens to a live conversation and turns it into a visual map in real time, so teams can see how ideas connect while they are still talking.

What it does

MindMesh turns live speech into evolving visual diagrams during a meeting.

It captures transcript events in real time, sends them through a backend pipeline, classifies the conversation into a diagram style like a flowchart, timeline, mind map, or org chart, and updates the frontend canvas as the discussion changes. Instead of waiting until the meeting is over, users can watch the conversation become a structured visual model while it is happening.

The product also wraps that experience in a meeting-style interface, so it feels like a collaborative live session rather than a static diagram tool.

How we built it

We built MindMesh as a full-stack real-time app.

The frontend uses Next.js and React to create the meeting interface and render diagrams on a React Flow canvas. We also added WebRTC signaling so the experience feels like a live collaborative room.

For speech capture, the client streams partial and final transcript events from browser speech recognition into a FastAPI backend over WebSockets. The backend keeps session state in memory, buffers transcript updates, checks when the conversation is ready to generate a diagram, and sends structured diagram events back to the frontend.

To make the system more reliable, we used a hybrid backend pipeline:

rules-based logic for fast triggering and intent classification
an OpenAI-compatible structured generation layer for harder cases, corrections, and more nuanced diagram reasoning
deterministic rendering logic that turns semantic facts into stable nodes, edges, and patch updates That separation helped us keep the AI focused on understanding meaning while the backend stayed in control of layout, versions, and consistency.

Challenges we ran into

The biggest challenge was making real-time AI feel stable.

Speech is messy. Partial transcripts change constantly, final transcripts arrive later, and people often correct themselves mid-sentence. If we regenerated the whole diagram on every transcript fragment, the interface became noisy and hard to trust.

We had to solve that by buffering transcript deltas, adding trigger logic and pause detection, and deciding when to send a full diagram replacement versus a smaller patch. Another major challenge was keeping graph IDs, versions, and layout stable enough that the visual felt like it was evolving naturally instead of flickering on every update.

We also ran into the usual real-time collaboration issues: reconnects, desync states, multi-user signaling, and making sure the frontend and backend stayed in sync during fast changes.

Accomplishments that we're proud of

We are proud that MindMesh works as a live system, not just a static mockup.

Some highlights:

We built a full real-time loop from speech input to live visual output.
We created a meeting-style UI that makes the experience feel natural and collaborative.
We designed a backend that can send both full diagram replacements and incremental patches.
We supported multiple diagram types instead of locking the project into a single visual format.
We built the system so the AI generates structured meaning, while the app controls rendering and consistency.

What we learned

We learned that building a good real-time AI experience is less about generating more output and more about generating the right output at the right time.

We also learned that separating semantic understanding from UI rendering makes the system much more reliable. Instead of asking AI to directly produce frontend-ready visuals, it worked far better to have it generate structured facts and let the backend convert those facts into stable graph updates.

Most importantly, we learned that collaboration tools become much more powerful when they help people see shared understanding, not just record what was said.

What's next for MindMesh

Next, we want to make MindMesh more production-ready and more useful in real meetings.

Our next steps include:

upgrading speech input to a more robust streaming speech-to-text pipeline
improving diagram accuracy for messy or multi-topic conversations
adding persistent sessions, saved diagrams, and meeting history
supporting exports and sharing for teams who want to reuse the generated visuals
detecting action items, decisions, and owners alongside the live diagram
improving multi-user collaboration so everyone in the meeting sees the same evolving map in sync Long term, we see MindMesh becoming a tool that does not just visualize conversations, but helps teams leave meetings with clear structure, shared context, and next steps.