Skip to content

fix(tools): flatten nested anyOf schemas for Vertex AI compatibility#409

Closed
carlulsoe wants to merge 1 commit intoopenclaw:mainfrom
carlulsoe:fix/vertex-json-schema-validation
Closed

fix(tools): flatten nested anyOf schemas for Vertex AI compatibility#409
carlulsoe wants to merge 1 commit intoopenclaw:mainfrom
carlulsoe:fix/vertex-json-schema-validation

Conversation

@carlulsoe
Copy link
Contributor

Summary

  • Fix JSON Schema validation errors when using Claude API on Vertex AI (Cloud Code Assist)
  • Nested anyOf schemas from TypeBox Type.Union([Type.Literal(...)]) patterns were being rejected as invalid JSON Schema draft 2020-12
  • Add tryFlattenLiteralAnyOf() to automatically convert literal unions to flat enum arrays
  • Flatten BrowserActSchema from discriminated union to single merged object
  • Simplify TelegramToolSchema to use Type.String() for IDs

Test plan

  • All existing tests pass (npx vitest run src/agents)
  • Type-check passes (bun run tsc --noEmit)
  • Verified tool schemas are properly flattened via runtime inspection
  • Tested sending messages through WhatsApp - no more 400 errors

🤖 Generated with Claude Code

@carlulsoe
Copy link
Contributor Author

AI Assistance Disclosure 🤖

Per CONTRIBUTING.md requirements:

  • AI-assisted: Built with Claude Code (Opus 4.5)
  • Testing level: Fully tested - ran test suite (182 tests passed), type-check, and manual verification sending messages through WhatsApp
  • I understand what this code does:
    • tryFlattenLiteralAnyOf() detects TypeBox's Type.Union([Type.Literal(...)]) pattern (which compiles to anyOf with const values) and converts it to a flat enum array
    • The BrowserActSchema discriminated union was flattened to a single object with all properties optional (except the discriminator kind)
    • Runtime validation still happens in the execute functions; the schema changes only affect what's sent to the LLM provider

Claude API on Vertex AI (Cloud Code Assist) rejects nested anyOf schemas
as invalid JSON Schema draft 2020-12. This change:

- Add tryFlattenLiteralAnyOf() to convert Type.Union([Type.Literal(...)])
  patterns from anyOf with const values to flat enum arrays
- Update stringEnum helper in bash-tools to use Type.Unsafe with flat enum
- Flatten BrowserActSchema from discriminated union to single object
- Simplify TelegramToolSchema to use Type.String() for IDs

Fixes 400 errors when sending messages through WhatsApp/Telegram providers.
@carlulsoe carlulsoe force-pushed the fix/vertex-json-schema-validation branch from f535e53 to e9c834f Compare January 7, 2026 17:17
@carlulsoe
Copy link
Contributor Author

Response to GPT-5.2-codex review

High (text for wait): The text field IS present in the flattened schema - it's shared between the type and wait actions. Verified:

Request properties: kind, targetId, ref, doubleClick, button, modifiers, text, submit, slowly, key, startRef, endRef, values, fields, width, height, timeMs, textGone, fn

Medium (relaxed validation): Acceptable tradeoff. The browser control server validates requests and returns clear error messages for invalid combinations. The LLM uses the schema as guidance, and runtime errors are handled gracefully.

Low (dropped metadata): Fixed in e9c834f - tryFlattenLiteralAnyOf now preserves title, default, and examples metadata in addition to description.

@steipete steipete self-assigned this Jan 7, 2026
@steipete
Copy link
Contributor

steipete commented Jan 7, 2026

Thanks ! Welcome to the Clawtributors! Say hi on Discord!

@steipete
Copy link
Contributor

steipete commented Jan 7, 2026

Landed on main as c3b3f57 + a2b3f2c. Added a README co-author credit commit (a882beb) so @carlulsoe shows up in the contributors graph. Closing PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants