feat(zeph-llm): GonkaProvider chat_with_tools and chat_typed (#3612)#3624
Merged
feat(zeph-llm): GonkaProvider chat_with_tools and chat_typed (#3612)#3624
Conversation
Extend LlmProvider impl on GonkaProvider with tool-use and structured output over the Gonka signed transport: - Promote decode_tool_chat_response to pub(crate) on OpenAiProvider with a provider_name parameter to surface the correct name in errors. - Add pub(crate) build_typed_chat_body<T> that extracts the JSON Schema body construction from OpenAiProvider::chat_typed for reuse. - Implement chat_with_tools: body via debug_request_json, signed send, decoded via inner.decode_tool_chat_response. Usage synced back from inner tracker to self.usage via last_usage/last_cache_usage. - Implement chat_typed<T>: body via inner.build_typed_chat_body, signed send, decoded with local OpenAiChatResponse, usage stored in self.usage. - supports_tool_use and supports_structured_output now return true. - Error bodies are truncated (256 chars logged, 512 bytes in InvalidInput) to avoid unbounded memory and information leakage from malicious nodes. - Two wiremock tests: gonka_tools_chat_with_tools_returns_tool_use and gonka_tools_chat_typed_returns_struct. All 35 gonka tests pass. Closes #3612
4dba88d to
89ecd2c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GonkaProviderwithchat_with_toolsandchat_typed<T>over the Gonka ECDSA-signed transportdecode_tool_chat_responsetopub(crate)onOpenAiProviderwith aprovider_nameparameterpub(crate) build_typed_chat_body<T>helper extracted fromOpenAiProvider::chat_typedInvalidInputmessages (512 chars) to prevent unbounded memory from malicious nodestest(gonka_tools)filter matches 2 tests per acceptance criterionTest plan
cargo nextest run -p zeph-llm --features gonka -E 'test(gonka_tools)'— 2 tests passcargo nextest run -p zeph-llm --features gonka -E 'test(gonka)'— 35 tests passcargo clippy --workspace --features full -- -D warnings— cleancargo +nightly fmt --check— cleancargo nextest run --workspace --lib --bins— 8892 tests passcargo doc --no-deps --features gonka -p zeph-llm— no broken intra-doc linksNotes
signed_requestswallows 429 inside the retry loop (converts toApiError { status: 429 }), soRateLimitedpath is intentionally not present in these methodsinnertracker toself.usageafterdecode_tool_chat_responsevialast_usage()/last_cache_usage()Closes #3612