Description
When onboarding NemoHermes with Google Gemini provider (gemini-2.5-flash), the inference smoke test during onboard passes, but chatting inside the sandbox fails with HTTP 400 INVALID_ARGUMENT. Gemini's API rejects the tool definitions sent by Hermes because they use OpenAI-style tool schema format (tools[].type + tools[].function) instead of Gemini's expected format. Specifically, tools at indices 5, 6, 7, and 19 contain "function" and "type" fields that Gemini cannot parse. The gateway routes Gemini through the Chat Completions API compatibility layer ("openai-completions"), but the tool schema conversion is incomplete for certain tool types.
Environment
Device: MacBook Pro (Apple Silicon M4 Pro)
OS: macOS 26.0.1 (Darwin, arm64)
Architecture: arm64
Node.js: v22.22.1
npm: 10.9.4
Docker: 29.2.1
OpenShell CLI: 0.0.44
NemoClaw: v0.0.50
Hermes Agent: v2026.5.16
Steps to Reproduce
nemoclaw onboard --agent hermes- Select "Google Gemini" as inference provider
- Enter Gemini API key
- Select model:
gemini-2.5-flash (default)
- Complete wizard (inference smoke test passes at step [4/8])
nemoclaw hermes connecthermes- Send any message (e.g. "What is 2+3?")
Expected Result
Hermes Agent sends the message to Gemini via the gateway and receives a model response.
Actual Result
API call failed (attempt 1/3): BadRequestError [HTTP 400]
Provider: custom Model: gemini-2.5-flash
Endpoint: https://inference.local/v1
Error: HTTP 400: Error code: 400 - [{'error': {'code': 400,
'message': 'Invalid JSON payload received. Unknown name "function"
at 'tools[5].function': Cannot find field.
Invalid JSON payload received. Unknown name "type" at
'tools[5].function': Cannot find field.
... (same for tools[6], tools[7], tools[19])',
'status': 'INVALID_ARGUMENT'}}]
Non-retryable error (HTTP 400) — trying fallback...
Non-retryable client error (HTTP 400). Aborting.
Logs
● What is 2+3? Answer with just the number.
Initializing agent...
⚠️ API call failed (attempt 1/3): BadRequestError [HTTP 400]
🔌 Provider: custom Model: gemini-2.5-flash
🌐 Endpoint: https://inference.local/v1
📝 Error: HTTP 400: Error code: 400 - [{'error': {'code': 400,
'message': 'Invalid JSON payload received. Unknown name "function"
at 'tools[5].function': Cannot find field.
Invalid JSON payload received. Unknown name "type" at
'tools[5].function': Cannot find field.
... (same for tools[6], [7], [19])
'status': 'INVALID_ARGUMENT',
'details': [{'@type': 'type.googleapis.com/google.rpc.BadRequest',
'fieldViolations': [...8 violations for tools[5,6,7,19]...]}]}}]
❌ Non-retryable client error (HTTP 400). Aborting.
NVB#6222637
Description
When onboarding NemoHermes with Google Gemini provider (
gemini-2.5-flash), the inference smoke test during onboard passes, but chatting inside the sandbox fails with HTTP 400INVALID_ARGUMENT. Gemini's API rejects the tool definitions sent by Hermes because they use OpenAI-style tool schema format (tools[].type+tools[].function) instead of Gemini's expected format. Specifically, tools at indices 5, 6, 7, and 19 contain"function"and"type"fields that Gemini cannot parse. The gateway routes Gemini through the Chat Completions API compatibility layer ("openai-completions"), but the tool schema conversion is incomplete for certain tool types.Environment
Steps to Reproduce
nemoclaw onboard --agent hermesgemini-2.5-flash(default)nemoclaw hermes connecthermesExpected Result
Hermes Agent sends the message to Gemini via the gateway and receives a model response.
Actual Result
Logs
NVB#6222637