Skip to content

feat(zeph-llm): GonkaProvider chat_with_tools and chat_typed (#3612)#3624

Merged
bug-ops merged 2 commits intomainfrom
gonka-chat-with-tools
May 5, 2026
Merged

feat(zeph-llm): GonkaProvider chat_with_tools and chat_typed (#3612)#3624
bug-ops merged 2 commits intomainfrom
gonka-chat-with-tools

Conversation

@bug-ops
Copy link
Copy Markdown
Owner

@bug-ops bug-ops commented May 5, 2026

Summary

  • Extends GonkaProvider with chat_with_tools and chat_typed<T> over the Gonka ECDSA-signed transport
  • Promotes decode_tool_chat_response to pub(crate) on OpenAiProvider with a provider_name parameter
  • Adds pub(crate) build_typed_chat_body<T> helper extracted from OpenAiProvider::chat_typed
  • Error bodies truncated before logging (256 chars) and InvalidInput messages (512 chars) to prevent unbounded memory from malicious nodes
  • 2 new wiremock tests; all 35 gonka tests pass; test(gonka_tools) filter matches 2 tests per acceptance criterion

Test plan

  • cargo nextest run -p zeph-llm --features gonka -E 'test(gonka_tools)' — 2 tests pass
  • cargo nextest run -p zeph-llm --features gonka -E 'test(gonka)' — 35 tests pass
  • cargo clippy --workspace --features full -- -D warnings — clean
  • cargo +nightly fmt --check — clean
  • cargo nextest run --workspace --lib --bins — 8892 tests pass
  • cargo doc --no-deps --features gonka -p zeph-llm — no broken intra-doc links
  • Live API round-trip (LLM serialization gate) — requires Gonka testnet credentials; to be verified by CI cycle before merge

Notes

  • signed_request swallows 429 inside the retry loop (converts to ApiError { status: 429 }), so RateLimited path is intentionally not present in these methods
  • Usage tracking syncs from inner tracker to self.usage after decode_tool_chat_response via last_usage()/last_cache_usage()

Closes #3612

Extend LlmProvider impl on GonkaProvider with tool-use and structured
output over the Gonka signed transport:

- Promote decode_tool_chat_response to pub(crate) on OpenAiProvider with
  a provider_name parameter to surface the correct name in errors.
- Add pub(crate) build_typed_chat_body<T> that extracts the JSON Schema
  body construction from OpenAiProvider::chat_typed for reuse.
- Implement chat_with_tools: body via debug_request_json, signed send,
  decoded via inner.decode_tool_chat_response. Usage synced back from
  inner tracker to self.usage via last_usage/last_cache_usage.
- Implement chat_typed<T>: body via inner.build_typed_chat_body, signed
  send, decoded with local OpenAiChatResponse, usage stored in self.usage.
- supports_tool_use and supports_structured_output now return true.
- Error bodies are truncated (256 chars logged, 512 bytes in InvalidInput)
  to avoid unbounded memory and information leakage from malicious nodes.
- Two wiremock tests: gonka_tools_chat_with_tools_returns_tool_use and
  gonka_tools_chat_typed_returns_struct. All 35 gonka tests pass.

Closes #3612
@github-actions github-actions Bot added the enhancement New feature or request label May 5, 2026
@bug-ops bug-ops enabled auto-merge (squash) May 5, 2026 23:12
@github-actions github-actions Bot added the size/L Large PR (201-500 lines) label May 5, 2026
@bug-ops bug-ops force-pushed the gonka-chat-with-tools branch from 4dba88d to 89ecd2c Compare May 5, 2026 23:12
@github-actions github-actions Bot added documentation Improvements or additions to documentation llm zeph-llm crate (Ollama, Claude) rust Rust code changes labels May 5, 2026
@bug-ops bug-ops merged commit 3b1cc09 into main May 5, 2026
32 checks passed
@bug-ops bug-ops deleted the gonka-chat-with-tools branch May 5, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request llm zeph-llm crate (Ollama, Claude) rust Rust code changes size/L Large PR (201-500 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(zeph-llm): GonkaProvider chat_with_tools / chat_typed (extract openai/wire.rs)

1 participant