fix(ai): buffer streaming preamble text before code fence appears by temrjan · Pull Request #8911 · marimo-team/marimo

temrjan · 2026-03-28T19:04:51Z

Summary

Buffer incoming AI stream chunks in CellCreationStream until a code fence (```) appears, preventing conversational preamble from becoming a separate Python cell
Flush buffered content as a cell in stop() if the stream ends without any fence (backward compatibility for models that return code without fences)
Add tests for: preamble + fence, code without fence, fence from first chunk

Root cause

CellCreationStream.stream() called codeToCells(buffer) on every chunk. Before any fence arrived, codeToCells treated the entire buffer as Python code (line 166 of completion-utils.ts: if (!code.includes("```")) return [{ language: "python", code }]), creating a cell from preamble text like "I'll create a fibonacci function...".

What this does NOT fix

The "Inline AI edit" flow (problem #2 in the issue) uses a different code path through the backend without_wrapping_backticks function. That is a separate issue.

Test plan

All 25 staged-cells tests pass
TypeScript clean (no new errors)
Test: preamble text buffered, cell created only when fence arrives
Test: code without fence → cell created on stream end (backward compat)
Test: fence in first chunk → cell created immediately (no delay)

Fixes #8880

🤖 Generated with Claude Code

When AI models emit conversational preamble before code fences (e.g. "I'll create a cell that..."), the preamble was incorrectly created as a separate Python cell. This happened because CellCreationStream called codeToCells on partial buffer before any fence arrived, treating plain text as Python code. Now CellCreationStream buffers incoming chunks until a code fence (```) appears. Once a fence is found, codeToCells correctly extracts only the code. If the stream ends without any fence, the buffer is flushed as a cell on stop() for backward compatibility. Note: This fixes the "Generate AI cell" flow. The "Inline AI edit" flow (backend without_wrapping_backticks) is a separate issue. Fixes marimo-team#8880 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel · 2026-03-28T19:04:56Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
marimo-docs	Ready	Preview, Comment	Apr 8, 2026 5:27am

github-actions · 2026-03-28T19:05:01Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

temrjan · 2026-03-28T19:15:59Z

I have read the CLA Document and I hereby sign the CLA

manzt

Nice. I think buffering until a fence appears makes sense, and the stop() fallback for fence-less responses is a good backward-compat touch.

manzt · 2026-04-01T18:47:53Z

    });
  });

-  it("should handle delta chunks", () => {


This test covered the case where the first chunk is a partial fence, which got replaced. Would we add that scenario back?

Added! The new test (should buffer partial fence and create cell when fence completes) covers the split-fence scenario: first chunk has just ``````, second chunk completes the fence — no premature cell creation.

Cover the scenario where a code fence arrives split across chunks (e.g. first chunk has just "``", second completes the fence). Ensures no premature cell creation from incomplete fences. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR adjusts the frontend AI “staged cells” streaming logic to avoid turning conversational preamble into its own Python cell by delaying parsing until a code fence is observed (or until the stream ends).

Changes:

Buffer streamed text in CellCreationStream.stream() until a triple-backtick code fence appears (unless cells were already created).
On stream end (stop()), flush buffered content into a cell if no cells were created (maintains behavior for models that emit code without fences).
Add/adjust staged-cells streaming tests for preamble buffering, no-fence flushing, fences in first chunk, and partial-fence completion.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
frontend/src/core/ai/staged-cells.ts	Buffers streamed chunks until a code fence appears; flushes buffered content on stop when no cells were created.
frontend/src/core/ai/tests/staged-cells.test.ts	Adds test coverage for buffering behavior and stop-time flushing across multiple stream chunk patterns.

Copilot · 2026-04-08T03:42:08Z

+      }
+    }
    // Clear all state
    this.buffer = "";


stop() says it clears all state, but it only resets buffer. createdCells (and hasMarimoImport) remain populated, which can leak state if any additional text-delta chunks arrive after text-end/finish, and also makes the comment inaccurate. Consider resetting createdCells/hasMarimoImport here (and/or nulling the stream ref after stop) so a completed stream cannot update previous cells.

Suggested change

this.buffer = "";

this.buffer = "";

this.createdCells = [];

this.hasMarimoImport = false;

Good catch — applied in 2d784fc. CellCreationStream is single-use (new instance per text-start), so this is purely defensive, but it makes the comment accurate.

Clear createdCells and hasMarimoImport alongside buffer so the "Clear all state" comment is accurate. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-05-08T09:36:20Z

This pull request has been automatically marked as stale because it has not had activity in 30 days. It will be closed in 14 days if no further activity occurs. If this PR is still relevant, please leave a comment or push new changes to keep it open. Thank you for your contribution!

Light2Dark · 2026-05-11T19:17:53Z

needs some manual testing as I couldn't reproduce this issue.

temrjan · 2026-05-12T04:47:27Z

Hi @Light2Dark, thanks for the review!

Issue #8880 actually describes two distinct bugs:

Generate AI cell — conversational preamble becomes its own Python cell.
Inline AI edit — preamble, code, and trailing text all land mixed in one cell.

This PR addresses only Bug 1 (frontend-only change to staged-cells.ts). Bug 2 goes through a different code path (without_wrapping_backticks in marimo/_server/ai/providers.py) and would need a separate fix. Worth noting that Inline AI edit would still exhibit Bug 2's symptom — the two look similar but stem from different code paths.

Repro for Bug 1

Configure Anthropic Haiku 4.5 as the AI model (any model that emits conversational preamble works)
Use Generate AI cell (not Inline AI edit)
Prompt: fibonacci

On main: two staged cells — one with prose, one with code.
On this branch: one staged cell with only the code; the preamble is buffered until a fence appears, then discarded.

The screenshots in #8880 show the exact symptom on main.

Validation

4 unit tests cover the buffering paths (preamble + fence, no-fence flushing, fence in first chunk, split fence). CI is green.
@kyrre (original reporter) — would you be able to confirm this PR resolves Bug 1 in your setup?

Happy to record a side-by-side GIF (main vs. this branch) if it would help.

github-actions · 2026-06-11T10:33:35Z

This pull request has been automatically marked as stale because it has not had activity in 30 days. It will be closed in 14 days if no further activity occurs. If this PR is still relevant, please leave a comment or push new changes to keep it open. Thank you for your contribution!

kirangadhave · 2026-06-13T01:04:30Z

@cubic-dev-ai

cubic-dev-ai · 2026-06-13T01:04:41Z

@cubic-dev-ai

@kirangadhave I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

1 issue found across 2 files

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="frontend/src/core/ai/staged-cells.ts">

<violation number="1" location="frontend/src/core/ai/staged-cells.ts:249">
P2: Buffered no-fence content is lost when streams terminate via abort/error because those paths do not call `stop()` to flush `buffer`. The new buffering logic defers all cell creation until `stop()` is reached, but `onStream` only calls `stop()` on `text-end`/`finish`; `abort`/`error` paths just log and return, so buffered content is silently dropped.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant AI as AI Service
    participant CS as CellCreationStream
    participant Buffer as Stream Buffer
    participant CTC as codeToCells()
    participant Cell as Cell Manager

    Note over AI,Cell: Streaming AI Response Processing

    loop For each text-delta chunk
        AI->>CS: stream({ delta })
        CS->>Buffer: Append delta to buffer
        
        alt Buffer has no fence AND no cells created yet
            Note over CS: Buffer preamble content<br/>Wait for code fence
            CS-->>AI: Return (no cell created)
        else Fence found OR cells already exist
            CS->>Buffer: Read full buffer
            CS->>CTC: codeToCells(buffer)
            CTC-->>CS: Parsed cells [{language, code}]
            
            loop For each parsed cell
                alt Cell is new (not yet in createdCells)
                    CS->>Cell: onCreateCell(code)
                    Cell-->>CS: newCellId
                    Note over CS: Track each cell ID
                else Cell already exists
                    CS->>Cell: onUpdateCell(cellId, code)
                    Note over CS: Update existing cell content
                end
            end
        end
    end

    opt Stream ends (text-end event)
        AI->>CS: stop()
        
        alt Buffer has content but no cells were created
            Note over CS: Fallback: code without fences
            CS->>CTC: codeToCells(buffer)
            CTC-->>CS: Parsed cells
            CS->>Cell: onCreateCell(code) for each
        end
        
        CS->>CS: Clear buffer, reset state
    end

    Note over CS: Key boundary: Preamble vs Code<br/>Fence "```" triggers cell creation

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

cubic-dev-ai · 2026-06-13T01:07:16Z

+    // buffer the content and wait. This prevents conversational preamble
+    // (e.g. "I'll create a cell that...") from becoming a Python cell.
+    // Once a fence appears, codeToCells will correctly extract only the code.
+    if (!this.buffer.includes("```") && this.createdCells.length === 0) {


P2: Buffered no-fence content is lost when streams terminate via abort/error because those paths do not call stop() to flush buffer. The new buffering logic defers all cell creation until stop() is reached, but onStream only calls stop() on text-end/finish; abort/error paths just log and return, so buffered content is silently dropped.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At frontend/src/core/ai/staged-cells.ts, line 249: <comment>Buffered no-fence content is lost when streams terminate via abort/error because those paths do not call `stop()` to flush `buffer`. The new buffering logic defers all cell creation until `stop()` is reached, but `onStream` only calls `stop()` on `text-end`/`finish`; `abort`/`error` paths just log and return, so buffered content is silently dropped.</comment> <file context> @@ -241,6 +241,15 @@ class CellCreationStream { + // buffer the content and wait. This prevents conversational preamble + // (e.g. "I'll create a cell that...") from becoming a Python cell. + // Once a fence appears, codeToCells will correctly extract only the code. + if (!this.buffer.includes("```") && this.createdCells.length === 0) { + return; + } </file context>

temrjan requested review from Light2Dark and manzt as code owners March 28, 2026 19:04

vercel Bot deployed to Preview March 28, 2026 19:06 View deployment

manzt reviewed Apr 1, 2026

View reviewed changes

vercel Bot deployed to Preview April 2, 2026 02:26 View deployment

Light2Dark requested a review from Copilot April 8, 2026 03:37

Copilot started reviewing on behalf of Light2Dark April 8, 2026 03:38 View session

Light2Dark added the bug Something isn't working label Apr 8, 2026

Copilot AI reviewed Apr 8, 2026

View reviewed changes

fix: reset all state in CellCreationStream.stop()

2d784fc

Clear createdCells and hasMarimoImport alongside buffer so the "Clear all state" comment is accurate. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel Bot deployed to Preview April 8, 2026 05:27 View deployment

github-actions Bot added the stale label May 8, 2026

Light2Dark marked this pull request as draft May 11, 2026 18:58

temrjan marked this pull request as ready for review May 12, 2026 04:47

github-actions Bot removed the stale label May 12, 2026

github-actions Bot added the stale label Jun 11, 2026

cubic-dev-ai Bot reviewed Jun 13, 2026

View reviewed changes

github-actions Bot removed the stale label Jun 13, 2026

Conversation

temrjan commented Mar 28, 2026

Summary

Root cause

What this does NOT fix

Test plan

Uh oh!

vercel Bot commented Mar 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

temrjan commented Mar 28, 2026

Uh oh!

manzt left a comment

Choose a reason for hiding this comment

Uh oh!

manzt Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

temrjan Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

temrjan Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 8, 2026

Uh oh!

Light2Dark commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

temrjan commented May 12, 2026

Repro for Bug 1

Validation

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

kirangadhave commented Jun 13, 2026

Uh oh!

cubic-dev-ai Bot commented Jun 13, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot Jun 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

vercel Bot commented Mar 28, 2026 •

edited

Loading

github-actions Bot commented Mar 28, 2026 •

edited

Loading

Light2Dark commented May 11, 2026 •

edited

Loading