Skip empty sherpa structured transcripts by TurboTheTurtle · Pull Request #84667 · openclaw/openclaw

TurboTheTurtle · 2026-05-20T18:33:49Z

Summary

Treat structured sherpa-onnx-offline JSON with an empty text field as an empty transcript instead of falling back to raw stdout.
Preserve normal sherpa JSON extraction for non-empty transcript output, including final-line JSON after startup logs.
Add direct CLI-runner and auto-detected audio regression coverage.

Real behavior proof

Behavior or issue addressed:
Moonshine/sherpa can emit a structured JSON object where text is empty. Before this patch, OpenClaw treated that as unrecognized sherpa output, fell back to the raw JSON string, and could pass that JSON to the LLM as a voice transcript.

Real environment tested:
Local macOS worktree with bundled Node.js v24.14.0, repository dependencies, and a fake executable named sherpa-onnx-offline invoked through the real CLI media runner.

Exact steps or command run after this patch:

PATH=/Users/andy/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin:$PATH node node_modules/tsx/dist/cli.mjs /private/tmp/openclaw-84660-proof/proof.ts
PATH=/Users/andy/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin:$PATH node scripts/run-vitest.mjs src/media-understanding/runner.cli-audio.test.ts src/media-understanding/apply.test.ts
git diff --check upstream/main..HEAD
git log --format='%h %an <%ae> %s' upstream/main..HEAD

Evidence after fix:
The standalone runtime proof invokes the real runCliEntry() path with a fake sherpa-onnx-offline binary. Empty structured JSON no longer returns a transcript, while non-empty structured JSON still extracts normally:

{
  "emptySherpaJsonReturnedTranscript": false,
  "emptySherpaJsonText": null,
  "nonEmptySherpaJsonText": "hello from sherpa",
  "nonEmptySherpaJsonKind": "audio.transcription"
}

Targeted tests:

Test Files  2 passed (2)
Tests       52 passed (52)

Observed result after fix:
Empty sherpa/moonshine JSON is skipped before it can become ctx.Transcript or an audio.transcription output, so the serialized Discord voice lane will not enqueue an agent turn for that empty segment. Non-empty sherpa JSON remains accepted.

What was not tested:
I did not run a live Discord voice session or the full repository test suite. The proof uses the real media CLI runner with a fake sherpa executable and targeted media-understanding tests for the affected normalization and auto-detect paths.

Attribution

If this PR is squashed or reworked, please preserve the commit author attribution for Andy Ye 35905412+TurboTheTurtle@users.noreply.github.com, or include:

Co-authored-by: Andy Ye 35905412+TurboTheTurtle@users.noreply.github.com

clawsweeper · 2026-05-20T18:35:18Z

Codex review: passed.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Summary
The PR changes sherpa-onnx CLI audio parsing so structured JSON with an empty text field becomes no transcript, while preserving non-empty JSON extraction and adding direct plus auto-detect regression coverage.

Reproducibility: yes. Source inspection on current main shows empty sherpa structured JSON misses extraction, falls back to raw stdout, and remains non-empty before Discord voice can skip empty transcripts; I did not run a live Discord reproduction in this read-only review.

PR rating
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Summary: Focused bug-fix PR with sufficient terminal proof, targeted regression coverage, and no blocking finding from this review.

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Sufficient (terminal): The PR body includes after-fix terminal proof through the real CLI media runner path with a fake sherpa executable, showing empty JSON returns no transcript while non-empty JSON still extracts normally.

Risk before merge

No live Discord voice/moonshine session or full repository suite is attached; exact-head CI should still gate broader regressions before merge.

Maintainer options:

Decide the mitigation before merge
Land this shared CLI media-understanding parser fix after exact-head CI/automerge gates, and leave broader voice queue-depth or stale-segment policy as a separate follow-up if maintainers want that resilience work.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge
No repair lane is needed because the exact-head PR has no discrete actionable blocking finding; CI, automerge, and normal maintainer review should gate the merge.

Security
Cleared: The diff changes media-output parsing, focused tests, a changelog entry, and a package override guardrail assertion only; it adds no dependencies, scripts, workflow permissions, secret handling, or new code-execution surface.

Review details

Best possible solution:

Land this shared CLI media-understanding parser fix after exact-head CI/automerge gates, and leave broader voice queue-depth or stale-segment policy as a separate follow-up if maintainers want that resilience work.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main shows empty sherpa structured JSON misses extraction, falls back to raw stdout, and remains non-empty before Discord voice can skip empty transcripts; I did not run a live Discord reproduction in this read-only review.

Is this the best way to solve the issue?

Yes. Updating the shared sherpa CLI output normalization before runCliEntry() emits an audio transcription is the narrowest maintainable fix, and the PR covers both direct CLI and auto-detected audio paths.

Label justifications:

P1: The linked bug affects an active Discord voice workflow by letting empty STT segments consume serialized agent turns and delay useful replies.
rating: 🦞 diamond lobster: Current PR rating is 🦞 diamond lobster because proof is 🦞 diamond lobster, patch quality is 🦞 diamond lobster, and Focused bug-fix PR with sufficient terminal proof, targeted regression coverage, and no blocking finding from this review.
status: 🚀 automerge armed: This PR is in ClawSweeper's automerge lane. Sufficient (terminal): The PR body includes after-fix terminal proof through the real CLI media runner path with a fake sherpa executable, showing empty JSON returns no transcript while non-empty JSON still extracts normally.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal proof through the real CLI media runner path with a fake sherpa executable, showing empty JSON returns no transcript while non-empty JSON still extracts normally.

What I checked:

Current main bug path: On current main, extractSherpaOnnxText() returns null when a sherpa JSON object's text field is an empty string, and resolveCliOutput() then falls back to params.stdout.trim(), so the raw JSON remains a non-empty transcript candidate. (src/media-understanding/runner.entries.ts:99, c0312748c4f7)
Voice path impact: Discord voice transcription only skips after normalizeOptionalString(result.text) is falsy, and segment processing returns early only for a falsy transcript; a raw JSON string therefore reaches the agent-turn path. (extensions/discord/src/voice/tts.ts:88, c0312748c4f7)
PR implementation: At PR head, sherpa output parsing distinguishes matched from non-empty text, so a structured empty text value returns an empty string; runCliEntry() trims that to empty and returns null instead of an audio transcription output. (src/media-understanding/runner.entries.ts:99, ac03171cfc3d)
Regression coverage: The PR adds a direct CLI-runner test for empty sherpa structured JSON and a final-line JSON preservation test, plus an auto-detect test that verifies empty sherpa output leaves Transcript unset and appliedAudio false. (src/media-understanding/runner.cli-audio.test.ts:85, ac03171cfc3d)
Auto-detect contract: OpenClaw docs and source both list sherpa-onnx-offline as the first local CLI audio auto-detect option when SHERPA_ONNX_MODEL_DIR has encoder/decoder/joiner/tokens, which matches the linked issue's runtime surface. Public docs: docs/nodes/audio.md. (docs/nodes/audio.md:19, c0312748c4f7)
History provenance: git blame shows the current sherpa parser, local sherpa auto-detect, and Discord voice transcription path all date to befb0f3d39bd33410cee85d0e193834af96d2773 (feat(discord): follow configured users in voice), while earlier media auto-detect and runner-entry refactors were carried by 2dfbd1c1f6a6ce197396f2b92b157bd6a3914f6a and 2a1f8b2615b1acbd0084bdd1d1c126d80d39284c. (src/media-understanding/runner.entries.ts:99, befb0f3d39bd)

Likely related people:

fuller-stack-dev: git blame ties the current sherpa parser, local sherpa auto-detect, and Discord voice STT bridge to befb0f3d39bd33410cee85d0e193834af96d2773. (role: introduced current behavior; confidence: high; commits: befb0f3d39bd; files: src/media-understanding/runner.entries.ts, src/media-understanding/runner.ts, extensions/discord/src/voice/tts.ts)
steipete: Recent media auto-detect and runner-entry history includes the main auto-detect feature/refactor commits and a release-test alignment commit touching this surface. (role: recent area contributor; confidence: high; commits: 2dfbd1c1f6a6, 2a1f8b2615b1, 384a161bbc2e; files: src/media-understanding/runner.ts, src/media-understanding/runner.entries.ts, docs/nodes/audio.md)
vincentkoc: History around transcribeAudioFile shows adjacent media-runtime contract work in 6e74d77a42e9a682fbbba6644c74e6435491c24f, relevant to the shared runtime boundary used by Discord voice. (role: adjacent owner; confidence: medium; commits: 6e74d77a42e9; files: src/media-understanding, extensions/discord/src/voice)

Codex review notes: model gpt-5.5, reasoning high; reviewed against c0312748c4f7.

clawsweeper · 2026-05-20T18:41:17Z

ClawSweeper PR egg

✨ Hatched: 🥚 common Pearl Signal Puff

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

Merged PRs are hatchable.
Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.

Rarity: 🥚 common.
Trait: collects tiny proofs.
Image traits: location branch lighthouse; accessory CI status badge; palette seafoam, black, and opal; mood focused; pose nestled inside a glowing shell; shell polished stone shell; lighting warm desk-lamp glow; background smooth stones and checkmarks.
Share on X: post this hatch
Copy: My PR egg hatched a 🥚 common Pearl Signal Puff in ClawSweeper.

What is this egg doing here?

Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

TurboTheTurtle · 2026-05-21T02:43:32Z

Pushed 3d6911bd44 to clear the stale CI blockers on the latest head.

What changed:

Updated the control UI i18n Pi fallback package version to match the workspace dependency (0.75.3).
Made the Bedrock runtime override guardrail compare against the Amazon provider package dependency instead of a stale hard-coded SDK version.
Removed now-linted empty object fallbacks in OpenRouter provider-routing spreads.
Updated the status summary redaction fixture for the current model-selection fields.

Local validation:

pnpm exec vitest run src/media-understanding/apply.test.ts src/media-understanding/runner.cli-audio.test.ts --reporter=verbose — 2 files / 52 tests passed
pnpm exec vitest run src/scripts/control-ui-i18n.test.ts test/scripts/root-package-overrides.test.ts — 2 files / 6 tests passed
pnpm exec vitest run src/commands/status.summary.redaction.test.ts — passed
pnpm run lint — passed
pnpm run lint:extensions:bundled — passed
pnpm run check:test-types — passed
pnpm exec oxfmt --check extensions/openrouter/provider-routing.ts scripts/control-ui-i18n.ts src/commands/status.summary.redaction.test.ts test/scripts/root-package-overrides.test.ts — passed
git diff --check — passed

@clawsweeper re-review

clawsweeper · 2026-05-21T02:45:25Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Superseded
Detail: A newer re-review for this item started before this run finished, so GitHub cancelled this older run. Check the latest ClawSweeper run for the current result.
Run: https://github.com/openclaw/clawsweeper/actions/runs/26202370415
Updated: 2026-05-21T02:48:19.309Z

TurboTheTurtle · 2026-05-21T02:48:04Z

Merged current origin/main into the branch and pushed 521a6cd310; #84667 should no longer be blocked by the dirty merge state once GitHub refreshes.

Conflict resolution:

Kept upstream status.summary.redaction.test.ts fixture values while preserving the required model-selection fields.
Kept the dynamic Bedrock override guardrail so it tracks the Amazon provider package dependency instead of drifting again.

Post-merge local validation:

focused PR/stale-CI tests: 5 files / 59 tests passed
pnpm run lint — passed
pnpm run check:test-types — passed
git diff --check — passed

@clawsweeper re-review

clawsweeper · 2026-05-21T02:49:59Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/26202509359
Updated: 2026-05-21T02:57:56.780Z

Takhoffman · 2026-05-21T03:41:31Z

@clawsweeper automerge

clawsweeper · 2026-05-21T03:43:28Z

🦞🧹
ClawSweeper automerge is enabled.

Head: ac03171cfc3d
Label: clawsweeper:automerge
Action: exact-head review queued (workflow sweep.yml, event repository_dispatch).
Flow: review this head, repair/rebase only if needed, then re-review the exact repaired head before merge.

Draft PRs stay fix-only until GitHub marks them ready for review. Pause with /clawsweeper stop.

Automerge progress:

2026-05-21 04:00:56 UTC automerge wait 9d131a689461 (blocked) in 14m 53s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204111070 GitHub checks failed: checks-node-core-runtime-media-ui:FAILURE

2026-05-21 04:00:57 UTC repair finished 9d131a689461 (pushed) in 14m 54s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204111070 repair_contributor_branch

2026-05-21 04:05:00 UTC review passed 9d131a689461 (structured ClawSweeper verdict: pass (sha=9d131a6894619ae82023377876999080c0c13...)

2026-05-21 04:05:16 UTC repair queued 9d131a689461 (autonomous) Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487

2026-05-21 04:07:49 UTC repair started (running) in 0s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 automerge-openclaw-openclaw-84667

2026-05-21 04:08:02 UTC validation plan (passed) in 13s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 pnpm check:changed; pnpm lint; pnpm check:test-types

2026-05-21 04:08:09 UTC Codex write preflight (passed) in 20s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 danger-full-access

2026-05-21 04:12:17 UTC Codex edit 1 9d131a689461 (complete) in 4m 28s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 exit 0

2026-05-21 04:15:27 UTC validation and review 1 ac03171cfc3d (base moved) in 7m 38s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 rebased

2026-05-21 04:15:32 UTC repair completed ac03171cfc3d (branch updated) in 7m 43s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 initial automerge rebase is delegated to Codex repair

2026-05-21 04:15:32 UTC review queued ac03171cfc3d (after repair)

2026-05-21 04:23:54 UTC automerge wait ac03171cfc3d (ready) in 16m 5s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 checks and exact-head review are ready

2026-05-21 04:23:55 UTC repair finished ac03171cfc3d (pushed) in 16m 7s Run: https://github.com/openclaw/clawsweeper/actions/runs/26204761487 repair_contributor_branch

2026-05-21 04:23:49 UTC review passed ac03171cfc3d (structured ClawSweeper verdict: pass (sha=ac03171cfc3d230a241c85deed74f9e25eadc...)

2026-05-21 04:24:02 UTC merged ac03171cfc3d (merged by ClawSweeper automerge)

2026-05-21 04:24:05 UTC review queued ac03171cfc3d (queued)

Summary: - The PR changes sherpa-onnx CLI audio parsing so structured JSON with an empty `text` field becomes no transcript, while preserving non-empty JSON extraction and adding direct plus auto-detect regression coverage. - Reproducibility: yes. Source inspection on current main shows empty sherpa structured JSON misses extraction ... scord voice can skip empty transcripts; I did not run a live Discord reproduction in this read-only review. Automerge notes: - PR branch already contained follow-up commit before automerge: Fix stale CI guardrails for sherpa transcript PR - PR branch already contained follow-up commit before automerge: Skip empty sherpa structured transcripts Validation: - ClawSweeper review passed for head ac03171. - Required merge gates passed before the squash merge. Prepared head SHA: ac03171 Review: openclaw#84667 (comment) Co-authored-by: Andy Ye <35905412+TurboTheTurtle@users.noreply.github.com> Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com> Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com> Approved-by: takhoffman Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>

openclaw-barnacle Bot added size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 20, 2026

openclaw-barnacle Bot added scripts Repository scripts commands Command implementations extensions: openrouter labels May 21, 2026

openclaw-barnacle Bot added cli CLI command changes and removed scripts Repository scripts commands Command implementations extensions: openrouter labels May 21, 2026

clawsweeper Bot added the clawsweeper:automerge Maintainer opted this PR into bounded ClawSweeper-reviewed automerge label May 21, 2026

clawsweeper Bot force-pushed the fix/sherpa-empty-transcripts-84660 branch from 521a6cd to 9d131a6 Compare May 21, 2026 03:57

openclaw-barnacle Bot removed the cli CLI command changes label May 21, 2026

clawsweeper Bot added status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. and removed status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels May 21, 2026

TurboTheTurtle and others added 4 commits May 21, 2026 04:15

Skip empty sherpa structured transcripts

8336a5b

Fix stale CI guardrails for sherpa transcript PR

a4c5985

Skip empty sherpa structured transcripts

777fc1a

Skip empty sherpa structured transcripts

ac03171

clawsweeper Bot force-pushed the fix/sherpa-empty-transcripts-84660 branch from 9d131a6 to ac03171 Compare May 21, 2026 04:15

clawsweeper Bot merged commit 46030f5 into openclaw:main May 21, 2026
97 checks passed

TurboTheTurtle deleted the fix/sherpa-empty-transcripts-84660 branch May 21, 2026 04:44

github-actions Bot mentioned this pull request May 21, 2026

📡 Upstream Digest — 2026-05-21 07:42 UTC curtismercier/openclaw-mods#906

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Skip empty sherpa structured transcripts#84667

Skip empty sherpa structured transcripts#84667
clawsweeper[bot] merged 4 commits into
openclaw:mainfrom
TurboTheTurtle:fix/sherpa-empty-transcripts-84660

TurboTheTurtle commented May 20, 2026

Uh oh!

clawsweeper Bot commented May 20, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 20, 2026

Uh oh!

TurboTheTurtle commented May 21, 2026

Uh oh!

clawsweeper Bot commented May 21, 2026 •

edited

Loading

Uh oh!

TurboTheTurtle commented May 21, 2026

Uh oh!

clawsweeper Bot commented May 21, 2026 •

edited

Loading

Uh oh!

Takhoffman commented May 21, 2026

Uh oh!

clawsweeper Bot commented May 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

TurboTheTurtle commented May 20, 2026

Summary

Real behavior proof

Attribution

Uh oh!

clawsweeper Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented May 20, 2026

Hatch command

Uh oh!

TurboTheTurtle commented May 21, 2026

Uh oh!

clawsweeper Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TurboTheTurtle commented May 21, 2026

Uh oh!

clawsweeper Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Takhoffman commented May 21, 2026

Uh oh!

clawsweeper Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented May 20, 2026 •

edited

Loading

clawsweeper Bot commented May 21, 2026 •

edited

Loading

clawsweeper Bot commented May 21, 2026 •

edited

Loading

clawsweeper Bot commented May 21, 2026 •

edited

Loading