fix(registry): render action params as JSON Schema instead of zod _def dump#33
Merged
Merged
Conversation
…f dump
RegisteredAction.promptDescription previously serialized each property of
the zod object schema by stringifying its private `_def` AST. For schemas
with `.default()` wrappers (e.g. ScrollActionSchema's `down` and
`num_pages`), the LLM would see something like:
"num_pages": {"type":"default","innerType":{"def":{"type":"number"},...},"defaultValue":1},
"down": {"type":"default","innerType":{"def":{"type":"boolean"},...},"defaultValue":true}
The model would plausibly copy the nearby `defaultValue: true` and emit a
boolean for `num_pages`. The schema correctly rejected, the same prompt
was fed back, and the same mistake recurred until `max_failures=3` tripped.
Replace the `_def` walk with `z.toJSONSchema(schema, {unrepresentable:'any'})`
(zod v4 native), strip the `$schema` dialect URL, and apply the existing
skipKeys filter to both `properties` and `required`. The LLM now sees:
{"type":"object","properties":{"down":{"default":true,"type":"boolean"},
"num_pages":{"default":1,"type":"number"},...}, "required":[...],
"additionalProperties":false}
— a familiar, well-known JSON Schema shape with no zod-internal leakage.
Surrounding `${description}: \n{${name}: ...}` envelope is unchanged so the
LLM sees the same outer layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
unadlib
reviewed
May 7, 2026
| schema: ZodTypeAny, | ||
| skipKeys: Set<string> | ||
| ): Record<string, unknown> { | ||
| const raw = z.toJSONSchema(schema, { unrepresentable: 'any' }) as Record< |
Member
There was a problem hiding this comment.
This should use input-mode JSON Schema generation: z.toJSONSchema(schema, { io: 'input', unrepresentable: 'any' }).
Without io: 'input', Zod treats .default() fields as required in the generated schema. That makes prompts say defaulted action params like scroll.down, scroll.num_pages, and done.success are required, even though they are optional inputs at runtime. Since this schema is shown to the LLM as the action input contract, it should describe accepted input rather than parsed output.
Per @unadlib's review on PR webllm#33: z.toJSONSchema without `io: 'input'` treats `.default()` fields as required in the generated schema. The schema is shown to the LLM as the action input contract, so it should describe accepted input, not parsed output. Without this fix, defaulted fields like scroll.num_pages and done.success show as required to the model, misleading the input contract. Also adds scripts/dump-schema.ts so the rendered schema for any registered action can be inspected from the CLI: bun run scripts/dump-schema.ts done bun run scripts/dump-schema.ts scroll bun run scripts/dump-schema.ts --all bun run scripts/dump-schema.ts --list Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Co-Authored-By: unadlib <unadlib@noreply.github.com>
caffeinum
commented
May 7, 2026
Refactors scripts/dump-schema.ts to call RegisteredAction.getPromptJsonSchema() instead of reimplementing z.toJSONSchema. The new method (and the underlying renderParamsJsonSchema helper) are exported from src/controller/registry/views.ts so the script exercises the exact code path RegisteredAction.promptDescription() uses to render the prompt for the LLM. Also drops the pnpm dump-schema package.json shortcut — the script stays runnable via bun/tsx directly. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
unadlib
approved these changes
May 8, 2026
Member
|
browser-use v0.6.1 has been released. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disclaimer: This is AI-generated, but I ran into this issue and tested the fix in prod. same in #34
Parity with Python upstream
The TypeScript port of
RegisteredAction.promptDescriptionwas JSON-stringifying zod's private_defAST, exposing internals likeinnerType/defaultValue/defto the LLM. Python upstream usespydantic.BaseModel.model_json_schema()and avoids this entirely. This PR brings the TS implementation to parity by switching toz.toJSONSchema()(zod v4, already in the dep tree).Reference:
browser_use/tools/registry/views.py:31-56onbrowser-use/browser-usemain. Specificallyviews.py:33:Problem
For example,
ScrollActionSchemawas rendered to the LLM as:{ "down": { "type": "default", "innerType": { "def": { "type": "boolean" } }, "defaultValue": true }, "num_pages": { "type": "default", "innerType": { "def": { "type": "number" } }, "defaultValue": 1 } }This is wrong-tier information for the model: it shows zod internals rather than a parameter schema. Worse, the adjacency of
"defaultValue": true(fordown) right next tonum_pagesplausibly causes the model to copy the wrong default — we observed the model emittingnum_pages: <boolean>repeatedly in real runs.Fix
Switch
promptDescriptionto render the param schema viaz.toJSONSchema(...). The sameScrollActionSchemanow renders as:{ "down": { "type": "boolean", "default": true }, "num_pages": { "type": "number", "default": 1 } }Clean, standard, and unambiguous about which default belongs to which field.
Impact
In a downstream consumer (canary-env), this fix combined with the retry-feedback fix (#34) took an eval from "BA bails at step 5 with
done(success=false)after a 3-fail cascade" to "BA completes at step 20 withrun_complete".Standalone effect of this PR: removes the
_defcascade entirely. Residualnum_pages: <boolean>errors still happen ~5x/run but now self-correct via retry feedback (#34).Tests
test/promptDescription.test.tscovering the JSON Schema rendering.Caveats
name=type (description)per param. Open to matching the Python format if reviewers prefer.z.toJSONSchemafrom zod v4, which is already a dep. No new dependency._defdump will need to read JSON Schema instead. (We don't believe anything was; this was an internal accident, not a contract.)🤖 Generated with Claude Code