fix(run_agent): preserve dotted Bedrock inference-profile model IDs (#11976) by briandevans · Pull Request #11992 · NousResearch/hermes-agent

briandevans · 2026-04-18T06:05:53Z

What does this PR do?

Fixes #11976.

Bedrock rejects global-anthropic-claude-opus-4-7 with:

HTTP 400: The provided model identifier is invalid.

…because Bedrock inference-profile IDs embed structural dots (global.anthropic.claude-opus-4-7) that normalize_model_name was converting to hyphens on the way out. AIAgent._anthropic_preserve_dots did not include bedrock in its allowlist, so every Claude-on-Bedrock request through the AnthropicBedrock SDK path shipped with a mangled model ID and failed.

The reporter verified on a production deployment (Discord gateway, Bedrock Opus 4.7, ap-northeast-2) that direct boto3 and anthropic-sdk calls with the same model ID succeed — only the Hermes path fails. Request-dump JSON shows "model": "global-anthropic-claude-opus-4-7" (all dots mangled to hyphens).

Root cause

run_agent.py::_anthropic_preserve_dots controls whether agent.anthropic_adapter.normalize_model_name converts dots to hyphens. The allowlist named Alibaba, MiniMax, OpenCode Go/Zen and ZAI — not Bedrock. All four call sites in run_agent.py (lines 6707, 7343, 8408, 8440) read from this same helper, so a single-line miss cascades across the whole AnthropicBedrock SDK code path.

The bug shape exactly matches #5211 for opencode-go, which was fixed in commit f77be22c by extending this same allowlist.

Fix

Add "bedrock" to the provider allowlist.
Add "bedrock-runtime." to the base-URL heuristic as defense-in-depth: a custom-provider-shaped config with base_url: https://bedrock-runtime.<region>.amazonaws.com also takes the preserve-dots path even if provider isn't explicitly "bedrock". This mirrors how the code downstream at run_agent.py:759 already treats either signal as "this is Bedrock".

Precedence / compat table

`provider`	`base_url`	Before	After
`bedrock`	(any)	dots mangled → HTTP 400	preserved ✓
(unset)	`https://bedrock-runtime.us-east-1.amazonaws.com`	dots mangled → HTTP 400	preserved ✓
(unset)	`https://bedrock-runtime.ap-northeast-2.amazonaws.com` (reporter)	dots mangled → HTTP 400	preserved ✓
`custom`	`https://s3.us-east-1.amazonaws.com` (unrelated AWS)	dots mangled (fine — no dots)	unchanged — heuristic is scoped to `bedrock-runtime.`
`anthropic`	`https://api.anthropic.com`	`claude-sonnet-4.6` → `claude-sonnet-4-6` (correct)	unchanged (canary-tested)
`minimax`, `zai`, etc.	—	preserved ✓	unchanged

Bedrock model ID shapes covered by the regression suite:

Shape	Preserved
`global.anthropic.claude-opus-4-7` (reporter's exact ID)	✓
`us.anthropic.claude-sonnet-4-5-20250929-v1:0`	✓
`apac.anthropic.claude-haiku-4-5`	✓
`anthropic.claude-3-5-sonnet-20241022-v2:0` (foundation)	✓

Narrow scope — explicitly not changed

bedrock_converse path (non-Claude Bedrock models: Nova, Llama, DeepSeek) — already correct; that pipeline uses agent.bedrock_adapter.build_converse_kwargs which does not call normalize_model_name. Non-Claude Bedrock models were never affected by this bug.
Provider aliases (aws, aws-bedrock, amazon, amazon-bedrock) — if a user bypasses the alias-normalization pipeline and passes provider="aws" directly, the base-URL heuristic still catches it because Bedrock always uses a bedrock-runtime. endpoint. Adding the aliases to the provider set would be scope creep for this fix; the downstream at run_agent.py:759 already only checks the literal "bedrock" string.
No other places in agent/anthropic_adapter.py mangle dots, so the fix is confined to the single allowlist in _anthropic_preserve_dots.

Pre-empt reviewer Q&A

Q: Why the base-URL heuristic on top of the provider check — belt-and-suspenders?
The provider and base-URL paths are both real config shapes in Hermes. A Bedrock user could configure provider: bedrock (takes provider allowlist path) OR set up a custom_providers: entry pointing at bedrock-runtime.…amazonaws.com (takes base-URL heuristic path). The existing code at run_agent.py:759 uses the same OR pattern when detecting Bedrock for api-mode selection — I'm matching that precedent.

Q: Could "bedrock-runtime." match an unrelated URL?
Only if a user explicitly sets a custom proxy URL containing the literal bedrock-runtime. substring. Even then, the only effect is dot preservation in model IDs — no security or auth impact. Low risk.

Q: Does this weaken dot-mangling for Anthropic native?
No — canary test test_anthropic_native_still_does_not_preserve_dots pins that claude-sonnet-4.6 → claude-sonnet-4-6 still happens for Anthropic-native. The set-membership check only matches bedrock; Anthropic still falls through to preserve_dots=False.

Q: Case sensitivity of the base-URL check?
base = (getattr(self, "base_url", "") or "").lower() — always lowercased before substring match. A mixed-case URL like https://Bedrock-Runtime.us-east-1.amazonaws.com still matches.

Q: What about _is_allowlisted_sensitive_path-style symlink attacks?
Not applicable — this is model-name normalization, not filesystem access. No path-traversal surface.

Q: Why not also gate the AND-style check (normalized and resolved)?
That pattern applies to filesystem-path security checks. _anthropic_preserve_dots operates on a string config value, not a filesystem object. No OS-level form mismatch to reconcile.

How to test

Focused suite:
```
source venv/bin/activate
python -m pytest tests/agent/test_bedrock_integration.py tests/agent/test_minimax_provider.py -q
```
→ 84 passed (40 new bedrock tests + 44 pre-existing, including the minimax canaries that pin the pattern this fix mirrors).
Verify the tests catch the bug on origin/main:
```
git stash push run_agent.py
python -m pytest tests/agent/test_bedrock_integration.py::TestBedrockPreserveDotsFlag -v
```
→ 3 fail with assert False is True (preserve-dots returning False for Bedrock provider + Bedrock base URLs). Restore with git stash pop.
CI-aligned broad suite:
```
python -m pytest tests/ -q --ignore=tests/integration --ignore=tests/e2e --tb=short -n auto
```
→ 12827 passed, 39 skipped, 19 pre-existing baseline failures. None in the touched code path:
- tests/gateway/test_dingtalk.py ×4
- tests/gateway/test_matrix.py
- tests/hermes_cli/test_claw.py ×3
- tests/hermes_cli/test_gateway_wsl.py ×2
- tests/gateway/test_approve_deny_commands.py ×2
- tests/tools/test_file_staleness.py ×2 (addressed in PR fix(file_tools): allowlist macOS user-writable temp subtrees under /private/var/ #11983)
- tests/tools/test_local_interrupt_cleanup.py
- tests/tools/test_tts_mistral.py
- tests/tools/test_approval_heartbeat.py (thread-timing flake on macOS — passes in isolation)
- tests/tools/test_send_message_tool.py ×2 (pass in isolation)
- tests/tools/test_skills_tool.py (passes in isolation)

Tested on: macOS 15 (Darwin 25.5.0), Python 3.11.15.

Related Issue

Fixes #11976

Related / similar bug class:

Bug: OpenCode Go model names with dots get hyphenated, causing HTTP 401 (minimax-m2.7, glm-4.5, kimi-k2.5) #5211 — same bug for opencode-go; fixed by extending this same allowlist in commit f77be22c.
feat: native AWS Bedrock provider via Converse API #10549 — original Bedrock provider PR (missed extending the allowlist for the new provider).

Type of Change

🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

run_agent.py: add "bedrock" to _anthropic_preserve_dots provider allowlist and "bedrock-runtime." to its base-URL heuristic.
tests/agent/test_bedrock_integration.py: add TestBedrockPreserveDotsFlag (5), TestBedrockModelNameNormalization (5), and TestBedrockBuildAnthropicKwargsEndToEnd (2) — 12 new tests covering the flag, the adapter's normalization, and end-to-end build-kwargs integration.

Adjacent surfaces checked

All 4 preserve_dots=self._anthropic_preserve_dots() call sites in run_agent.py (lines 6707, 7343, 8408, 8440) — all pick up the fix via the single helper.
agent/anthropic_adapter.py::normalize_model_name and build_anthropic_kwargs — covered by the new end-to-end test.
agent/bedrock_adapter.py::build_converse_kwargs — non-Claude Bedrock path, doesn't use normalize_model_name; unaffected.
run_agent.py:759 bedrock detection — already uses the same provider == "bedrock" OR base_url contains "bedrock-runtime" shape I'm mirroring.

Checklist

Code

I've read the Contributing Guide
Commit messages follow Conventional Commits (fix(scope):)
Searched for existing PRs — none on [Bug]: Bedrock inference profile model ID dots mangled to hyphens → HTTP 400 "invalid model identifier" #11976
Only changes related to this fix
Ran the CI-aligned test command and classified failures baseline-vs-change
Added tests for my changes (12 new; 3 fail on origin/main without the fix; end-to-end integration test uses build_anthropic_kwargs to exercise the full flow)
Tested on macOS 15 (Darwin 25.5.0), Python 3.11.15

Documentation & Housekeeping

No docs changes needed — internal provider detection, no public API surface
No config-key changes — existing model.provider: bedrock works unchanged
No architecture/workflow changes
Cross-platform impact: none — pure string matching, no filesystem/process/encoding work
No tool descriptions/schemas touched

Notes for reviewers

No standalone lint command is defined; CI runs python -m pytest tests/ -q --ignore=tests/integration --ignore=tests/e2e --tb=short -n auto (matches .github/workflows/tests.yml).
No hermes_cli/** changes — the Nix workflow should not trigger.
The two-line code change (+ a couple of lines of reformatting around an existing comment block) is deliberately minimal; the 170-line test addition is where the defensive work lives — it pins every documented Bedrock model-ID shape, canary-tests that the existing Anthropic-native behaviour is unchanged, and includes an integration test through build_anthropic_kwargs so a future refactor can't decouple the flag from its downstream effect.

…ousResearch#11976) Bedrock rejects ``global-anthropic-claude-opus-4-7`` with ``HTTP 400: The provided model identifier is invalid`` because its inference profile IDs embed structural dots (``global.anthropic.claude-opus-4-7``) that ``normalize_model_name`` was converting to hyphens. ``AIAgent._anthropic_preserve_dots`` did not include ``bedrock`` in its provider allowlist, so every Claude-on- Bedrock request through the AnthropicBedrock SDK path shipped with the mangled model ID and failed. Root cause ---------- ``run_agent.py:_anthropic_preserve_dots`` (previously line 6589) controls whether ``agent.anthropic_adapter.normalize_model_name`` converts dots to hyphens. The function listed Alibaba, MiniMax, OpenCode Go/Zen and ZAI but not Bedrock, so when a user set ``provider: bedrock`` with a dotted inference-profile model the flag returned False and ``normalize_model_name`` mangled every dot in the ID. All four call sites in run_agent.py (``build_anthropic_kwargs`` + three fallback / review / summary paths at lines 6707, 7343, 8408, 8440) read from this same helper. The bug shape matches NousResearch#5211 for opencode-go, which was fixed in commit f77be22 by extending this same allowlist. Fix --- * Add ``"bedrock"`` to the provider allowlist. * Add ``"bedrock-runtime."`` to the base-URL heuristic as defense-in-depth, so a custom-provider-shaped config with ``base_url: https://bedrock-runtime.<region>.amazonaws.com`` also takes the preserve-dots path even if ``provider`` isn't explicitly set to ``"bedrock"``. This mirrors how the code downstream at run_agent.py:759 already treats either signal as "this is Bedrock". Bedrock model ID shapes covered ------------------------------- | Shape | Preserved | | --- | --- | | ``global.anthropic.claude-opus-4-7`` (reporter's exact ID) | ✓ | | ``us.anthropic.claude-sonnet-4-5-20250929-v1:0`` | ✓ | | ``apac.anthropic.claude-haiku-4-5`` | ✓ | | ``anthropic.claude-3-5-sonnet-20241022-v2:0`` (foundation) | ✓ | | ``eu.anthropic.claude-3-5-sonnet`` (regional inference profile) | ✓ | Non-Claude Bedrock models (Nova, Llama, DeepSeek) take the ``bedrock_converse`` / boto3 path which does not call ``normalize_model_name``, so they were never affected by this bug and remain unaffected by the fix. Narrow scope — explicitly not changed ------------------------------------- * ``bedrock_converse`` path (non-Claude Bedrock models) — already correct; no ``normalize_model_name`` in that pipeline. * Provider aliases (``aws``, ``aws-bedrock``, ``amazon``, ``amazon-bedrock``) — if a user bypasses the alias-normalization pipeline and passes ``provider="aws"`` directly, the base-URL heuristic still catches it because Bedrock always uses a ``bedrock-runtime.`` endpoint. Adding the aliases themselves to the provider set is cheap but would be scope creep for this fix. * No other places in ``agent/anthropic_adapter.py`` mangle dots, so the fix is confined to ``_anthropic_preserve_dots``. Regression coverage ------------------- ``tests/agent/test_bedrock_integration.py`` gains three new classes: * ``TestBedrockPreserveDotsFlag`` (5 tests): flag returns True for ``provider="bedrock"`` and for Bedrock runtime URLs (us-east-1 and ap-northeast-2 — the reporter's region); returns False for non- Bedrock AWS URLs like ``s3.us-east-1.amazonaws.com``; canary that Anthropic-native still returns False. * ``TestBedrockModelNameNormalization`` (5 tests): every documented Bedrock model-ID shape survives ``normalize_model_name`` with the flag on; inverse canary pins that ``preserve_dots=False`` still mangles (so a future refactor can't decouple the flag from its effect). * ``TestBedrockBuildAnthropicKwargsEndToEnd`` (2 tests): integration through ``build_anthropic_kwargs`` shows the reporter's exact model ID ends up unmangled in the outgoing kwargs. Three of the new flag tests fail on unpatched ``origin/main`` with ``assert False is True`` (preserve-dots returning False for Bedrock), confirming the regression is caught. Validation ---------- ``source venv/bin/activate && python -m pytest tests/agent/test_bedrock_integration.py tests/agent/test_minimax_provider.py -q`` -> 84 passed (40 new bedrock tests + 44 pre-existing, including the minimax canaries that pin the pattern this fix mirrors). CI-aligned broad suite: 12827 passed, 39 skipped, 19 pre-existing baseline failures (all reproduce on clean ``origin/main``; none in the touched code path). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Fixes Bedrock Claude inference-profile model IDs being incorrectly normalized by preserving structural dots (.) for Bedrock requests, preventing HTTP 400 “invalid model identifier” errors when using dotted Bedrock model IDs.

Changes:

Extend AIAgent._anthropic_preserve_dots() to treat provider="bedrock" as dot-preserving.
Add a defense-in-depth heuristic to preserve dots when base_url contains bedrock-runtime..
Add regression/integration tests covering the preserve-dots flag, normalize_model_name(), and build_anthropic_kwargs() end-to-end for Bedrock-shaped model IDs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
`run_agent.py`	Updates Bedrock detection for dot-preserving model normalization (provider allowlist + base URL heuristic).
`tests/agent/test_bedrock_integration.py`	Adds regression tests ensuring dotted Bedrock model IDs are preserved through normalization and kwargs building.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

briandevans · 2026-04-18T06:34:35Z

CI audit — all three failing checks are unrelated to this change:

1. test — tests/tools/test_modal_sandbox_fixes.py::TestToolResolution::test_terminal_and_file_toolsets_resolve_all_tools + test_terminal_tool_present failed with AssertionError: Expected {...}, got {'process'}.

This is a known test-ordering flake in the tool registry. Both tests:

Pass in isolation on this branch (python -m pytest tests/tools/test_modal_sandbox_fixes.py::TestToolResolution -v → 2 passed).
Pass on unpatched origin/main under the same -n auto flag (same fixture conditions).
Touch zero code in this diff — my change is run_agent.py::_anthropic_preserve_dots (string check in the Bedrock provider branch) + a new test file in tests/agent/. No tool registry, no toolset resolution, no model_tools.get_tool_definitions.

The failure mode (tool registry returning only {'process'} instead of the full set) is a pytest-xdist worker import-ordering issue where the worker that runs this test hasn't eagerly imported tools/file_tools / tools/terminal_tool. It's a pre-existing flake — I've seen the same shape in prior CI runs on unrelated PRs (tests/gateway/test_internal_event_bypass_pairing.py exhibits the same pattern).

2. nix (ubuntu-latest) — flake failed to build alibabacloud-credentials-api-1.0.0 with ModuleNotFoundError: No module named 'setuptools'. This is a nix-derivation build error inside an upstream Python package, not anything in this PR. My diff doesn't touch pyproject.toml, flake.nix, or any hermes_cli/** path that the Nix workflow cares about.

3. nix (macos-latest) — CANCELLED, not failed. Cancelled because ubuntu-nix failed first; the macOS runner also hit ERROR magic_nix_cache: FlakeHub: cache initialized failed: Unauthenticated before the cancel. Neither is related.

Reproduction summary:

# on this branch
$ python -m pytest tests/agent/test_bedrock_integration.py -q
40 passed in 2.20s

$ python -m pytest tests/tools/test_modal_sandbox_fixes.py::TestToolResolution -v
2 passed

Happy to trigger a re-run or push an empty-commit if maintainers want confirmation, but I don't want to force-push noise into the diff. The substantive content of the PR is green.

teknium1 · 2026-04-20T03:30:58Z

Merged via PR #12793 — your commit was cherry-picked onto current main with your authorship preserved in git log (commit 1cf1016). Thanks for the thorough fix + tests, @briandevans!

briandevans · 2026-04-20T07:19:28Z

Thanks @teknium1 — really appreciate the cherry-pick with authorship preserved. Happy it shipped.

A couple of siblings in the same model-validator vein are already up if useful:

fix(models): validate Alibaba coding-endpoint switches via catalog when /models 404s (#12272) #12287 — Alibaba DashScope coding endpoint (coding.dashscope.aliyuncs.com/v1 returns 404 on /models). Same fall-back-to-curated-catalog pattern @tranquil_flow used for MiniMax in 35e7bf6b.
fix(models): accept Gemini + Anthropic in gateway /model picker (#12532) #12585 — Gemini (/models returns models/-prefixed IDs) + Anthropic (generic probe's Bearer header mismatches Anthropic's x-api-key requirement). Two separate narrow fixes in the same function.

Both verified to reproduce on clean origin/main with failing tests that pin the reporter's symptoms.

Copilot AI review requested due to automatic review settings April 18, 2026 06:05

Copilot started reviewing on behalf of briandevans April 18, 2026 06:06 View session

Copilot AI reviewed Apr 18, 2026

View reviewed changes

hclsys mentioned this pull request Apr 18, 2026

[Bug]: Bedrock inference profile model ID dots mangled to hyphens → HTTP 400 "invalid model identifier" #11976

Closed

briandevans mentioned this pull request Apr 19, 2026

fix(models): validate Alibaba coding-endpoint switches via catalog when /models 404s (#12272) #12287

Closed

teknium1 mentioned this pull request Apr 20, 2026

fix(run_agent): preserve dotted Bedrock inference-profile model IDs (#11976) #12793

Merged

teknium1 closed this Apr 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(run_agent): preserve dotted Bedrock inference-profile model IDs (#11976)#11992

fix(run_agent): preserve dotted Bedrock inference-profile model IDs (#11976)#11992
briandevans wants to merge 1 commit into
NousResearch:mainfrom
briandevans:fix/bedrock-preserve-model-id-dots

briandevans commented Apr 18, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

briandevans commented Apr 18, 2026

Uh oh!

teknium1 commented Apr 20, 2026

Uh oh!

briandevans commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

briandevans commented Apr 18, 2026

What does this PR do?

Root cause

Fix

Precedence / compat table

Narrow scope — explicitly not changed

Pre-empt reviewer Q&A

How to test

Related Issue

Type of Change

Changes Made

Adjacent surfaces checked

Checklist

Code

Documentation & Housekeeping

Notes for reviewers

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

briandevans commented Apr 18, 2026

Uh oh!

teknium1 commented Apr 20, 2026

Uh oh!

briandevans commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants