Skip to content

fix(ollama): yield during dense stream processing#87818

Merged
steipete merged 1 commit into
mainfrom
fix/ollama-dense-stream-yield
May 31, 2026
Merged

fix(ollama): yield during dense stream processing#87818
steipete merged 1 commit into
mainfrom
fix/ollama-dense-stream-yield

Conversation

@vincentkoc

@vincentkoc vincentkoc commented May 28, 2026

Copy link
Copy Markdown
Member

Summary

  • add cooperative yielding while Ollama native streams process dense NDJSON bursts
  • preserve caller abort handling during those yields so aborted runs surface as aborted instead of generic stream errors
  • add focused coverage that proves timers can fire before dense stream completion

Fixes part of #86599.

Verification

  • node scripts/run-vitest.mjs extensions/ollama/src/stream.test.ts --reporter=dot
  • git diff --check origin/main...HEAD
  • AWS Crabbox pnpm check:changed provider=aws lease=cbx_2487c84328ab slug=violet-barnacle run=run_b85266d8d566 base=e18099b8c32420fa2ff5eddf8731e24c7278851b exit=0

@vincentkoc vincentkoc self-assigned this May 28, 2026
@vincentkoc vincentkoc marked this pull request as ready for review May 28, 2026 23:49
@clawsweeper

clawsweeper Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed May 31, 2026, 11:42 AM ET / 15:42 UTC.

Summary
The PR adds bounded cooperative yields and abort-aware error reporting while processing dense Ollama native NDJSON streams, with focused stream tests.

PR surface: Source +47, Tests +86. Total +133 across 2 files.

Reproducibility: yes. for the scoped dense Ollama stream path: current main parses all NDJSON lines from one body chunk without a cooperative wait, and the PR adds a regression test for timer progress before stream completion. The full Windows Gateway starvation scenario remains unverified in this review.

Review metrics: 1 noteworthy metric.

  • Yield cadence surface: 2 constants added, 1 awaited scheduler call in the stream loop. These values define the hot-path scheduling behavior maintainers may want to own before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] A maintainer can request live Windows Gateway/Ollama proof if they want this PR to claim more than the scoped dense-stream repair.

Risk before merge

Maintainer options:

  1. Accept The Scoped Ollama Fix (recommended)
    A maintainer can land this as an Ollama-only responsiveness repair while leaving the broader Windows starvation issue open for remaining providers and live proof.
  2. Require Live Windows Gateway Proof
    Before merge, ask for a real Windows Gateway/local-model run showing timers, RPCs, or channel polling stay responsive during an Ollama stream.
  3. Pause For A Shared Scheduler Decision
    Maintainers can pause if they want the OpenAI and Ollama cooperative-yield logic moved to a shared provider-stream seam instead of duplicating the cadence locally.

Next step before merge

  • [P2] Protected maintainer ownership and the provider-stream availability tradeoff need human merge judgment; there is no narrow automation repair to request.

Security
Cleared: The diff only changes Ollama stream processing and tests; it does not touch dependencies, CI, secrets, package scripts, lockfiles, or downloaded code execution paths.

Review details

Best possible solution:

Merge after a maintainer accepts the scoped Ollama proof and availability tradeoff, while keeping the broader Windows local-provider starvation issue open until all related paths are proven fixed.

Do we have a high-confidence way to reproduce the issue?

Yes for the scoped dense Ollama stream path: current main parses all NDJSON lines from one body chunk without a cooperative wait, and the PR adds a regression test for timer progress before stream completion. The full Windows Gateway starvation scenario remains unverified in this review.

Is this the best way to solve the issue?

Yes for the scoped Ollama native stream problem: bounded cooperative yielding mirrors the existing OpenAI transport cadence and preserves abort classification. It is not a complete solution for every local-provider starvation path in the related issue.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against e0e7bae61219.

Label changes

Label justifications:

  • P1: The PR targets a user-facing local-model responsiveness regression that can make Gateway/channel workflows effectively unusable during model calls.
  • merge-risk: 🚨 availability: Changing cooperative scheduling in the Ollama stream loop can affect Gateway responsiveness, provider throughput, and cancellation behavior under dense streams.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body records focused stream tests plus an AWS Crabbox changed gate for the scoped branch, while clearly noting that the full Windows Gateway scenario was not proven here.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body records focused stream tests plus an AWS Crabbox changed gate for the scoped branch, while clearly noting that the full Windows Gateway scenario was not proven here.
Evidence reviewed

PR surface:

Source +47, Tests +86. Total +133 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 51 4 +47
Tests 1 86 0 +86
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 137 4 +133

What I checked:

  • Repository policy read: Root AGENTS.md and extensions/AGENTS.md were read fully enough for this review; the extension boundary and provider hot-path guidance apply to this PR. (AGENTS.md:17, e0e7bae61219)
  • Current main lacks the Ollama scheduler: Current main has cooperative stream scheduling in OpenAI transport paths, but no Ollama cooperative scheduler or dense native stream test in extensions/ollama. (extensions/ollama/src/stream.ts:1291, e0e7bae61219)
  • Patch changes the dense stream loop: The PR adds a 64-event/12ms cooperative scheduler, checks abort before and after yields, and awaits the scheduler after non-final NDJSON chunks. (extensions/ollama/src/stream.ts:54, 3a478d76384d)
  • Stream error contract supports abort classification: The stream function contract requires request/runtime failures to be encoded in the returned stream, with final error messages using stopReason "error" or "aborted". (packages/llm-core/src/types.ts:181, e0e7bae61219)
  • Regression coverage added: The PR adds tests proving a timer can fire before dense stream completion and that caller aborts during dense processing surface as aborted. (extensions/ollama/src/stream.test.ts:360, 3a478d76384d)
  • Scoped validation evidence: The PR body records focused Vitest, git diff check, and AWS Crabbox pnpm check:changed proof for the branch; this is scoped proof, not a live Windows Gateway reproduction. (3a478d76384d)

Likely related people:

  • Jakub Rusz: Commit 8f44bd6 added incremental Ollama streaming events and the partial-snapshot text_delta path this PR now yields around. (role: introduced streaming behavior; confidence: high; commits: 8f44bd642660; files: extensions/ollama/src/stream.ts)
  • Sunwoo Yu: Commit 1170229 added the native Ollama /api/chat provider and original stream implementation before it moved under extensions. (role: feature introducer; confidence: medium; commits: 11702290ff80; files: src/agents/ollama-stream.ts, extensions/ollama/src/stream.ts)
  • vincentkoc: Vincent has prior merged Ollama stream/custom API history in commit 7e946b3 and is carrying the current maintainer PR. (role: recent area contributor; confidence: high; commits: 7e946b3c6c25, 3a478d76384d; files: extensions/ollama/src/stream.ts, extensions/ollama/src/stream.test.ts)
  • steipete: Git shortlog and provider seam commits show repeated Ollama/provider stream maintenance and SDK boundary refactors, including be526d6. (role: adjacent owner; confidence: medium; commits: be526d642312, 87abcfd6a6b8; files: extensions/ollama/src/stream.ts, src/plugin-sdk/provider-stream-shared.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P1 High-priority user-facing bug, regression, or broken workflow. labels May 28, 2026
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch from 78695aa to a421761 Compare May 29, 2026 00:05
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 29, 2026
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch 3 times, most recently from da846ca to c0f574b Compare May 29, 2026 02:50
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch 7 times, most recently from 5e82ef0 to 086c92c Compare May 30, 2026 17:28
@clawsweeper clawsweeper Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 30, 2026
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch 2 times, most recently from c7181b1 to 0e486c8 Compare May 31, 2026 00:59
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch 5 times, most recently from ee47a2c to 3023487 Compare May 31, 2026 09:48
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch 2 times, most recently from 623d5b2 to 41508c2 Compare May 31, 2026 10:40
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. labels May 31, 2026
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch 5 times, most recently from 075f77c to dd4520b Compare May 31, 2026 15:01
@clawsweeper clawsweeper Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch from dd4520b to 4ee4bb4 Compare May 31, 2026 15:15
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@vincentkoc vincentkoc force-pushed the fix/ollama-dense-stream-yield branch 2 times, most recently from b80967c to 3a478d7 Compare May 31, 2026 15:34
@steipete steipete force-pushed the fix/ollama-dense-stream-yield branch from 3a478d7 to d921190 Compare May 31, 2026 16:34
@steipete

Copy link
Copy Markdown
Contributor

Land-ready proof for d921190.

Local proof after rebasing on current main:

  • node scripts/run-vitest.mjs extensions/ollama/src/stream.test.ts --reporter=dot
  • git diff --check origin/main...HEAD
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main

Results:

  • Ollama stream tests passed: 12 tests in extensions/ollama/src/stream.test.ts.
  • diff check passed.
  • autoreview clean: no accepted/actionable findings reported.
  • Fresh GitHub CI for the rebased PR head is green: 112 successful checks, 27 skipped, 0 failures, 0 pending.

Known proof gap: no live Windows/Ollama gateway reproduction was run here; this PR is scoped to the native Ollama dense NDJSON stream path and covered by the focused regression tests plus CI.

@steipete steipete merged commit c7b190b into main May 31, 2026
139 checks passed
@steipete steipete deleted the fix/ollama-dense-stream-yield branch May 31, 2026 16:38
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 1, 2026
Co-authored-by: uday <udaymanish.thumma@gmail.com>
SYU8384 pushed a commit to SYU8384/openclaw that referenced this pull request Jun 3, 2026
Co-authored-by: uday <udaymanish.thumma@gmail.com>
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
Co-authored-by: uday <udaymanish.thumma@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: ollama maintainer Maintainer-authored PR merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants