Skip to content

feat: add Amazon Bedrock as inference provider#6845

Closed
hheydaroff wants to merge 1 commit into
NousResearch:mainfrom
hheydaroff:feat/amazon-bedrock-provider
Closed

feat: add Amazon Bedrock as inference provider#6845
hheydaroff wants to merge 1 commit into
NousResearch:mainfrom
hheydaroff:feat/amazon-bedrock-provider

Conversation

@hheydaroff

@hheydaroff hheydaroff commented Apr 9, 2026

Copy link
Copy Markdown

Summary

Add Amazon Bedrock as a first-class inference provider using the Converse API via boto3.

Authentication

Three methods supported, selectable via hermes model:

  1. Bedrock API key (AWS_BEARER_TOKEN_BEDROCK) — generate in the Bedrock console → API keys
  2. AWS profile — picks from ~/.aws/credentials, sets AWS_PROFILE
  3. AWS access key pair — AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY

Region support

Interactive region picker in hermes model with common regions (us-east-1, eu-central-1, etc.) or custom entry. Stored as AWS_BEDROCK_REGION.

Model IDs

Bedrock uses cross-region inference profile prefixes:

  • global.anthropic.claude-opus-4-6-v1 — works from any region
  • eu.anthropic.claude-opus-4-6-v1 — EU cross-region routing
  • us.anthropic.claude-opus-4-6-v1 — US cross-region routing
  • anthropic.claude-opus-4-6-v1 — single-region (direct access only)

Usage

# Via hermes model (interactive setup)
hermes model  # → select Amazon Bedrock → pick auth → pick region → pick model

# Via /model in chat
/model eu.anthropic.claude-opus-4-6-v1 --provider bedrock

# Via env vars
export AWS_BEARER_TOKEN_BEDROCK="your-key"
export AWS_BEDROCK_REGION="eu-central-1"
hermes

Architecture

  • New API mode: bedrock_converse — joins existing chat_completions, codex_responses, anthropic_messages
  • New adapter: agent/bedrock_adapter.py — translates OpenAI-format messages ↔ Converse API format (system blocks, toolUse/toolResult content blocks, tool schemas)
  • No changes to existing provider flows — all modifications are additive elif branches

Files changed

File Change
agent/bedrock_adapter.py New — Converse API adapter
hermes_cli/auth.py Provider registry + region-aware URL resolution
hermes_cli/config.py AWS_BEARER_TOKEN_BEDROCK, AWS_BEDROCK_REGION env vars
hermes_cli/main.py Dedicated _model_flow_bedrock() with auth/region/model setup
hermes_cli/models.py Curated model list (global/eu/us/bare IDs)
hermes_cli/providers.py Overlay, aliases, labels
hermes_cli/runtime_provider.py bedrock_converse in valid modes
agent/models_dev.py Provider mapping
run_agent.py Full lifecycle: init, switch, build kwargs, call, validate, normalize
pyproject.toml boto3 dependency

Testing

All 9632 existing tests pass. Changes are additive — no existing provider flows modified.

@hheydaroff hheydaroff force-pushed the feat/amazon-bedrock-provider branch 2 times, most recently from 66ba505 to 2cafcb1 Compare April 9, 2026 23:05
@hheydaroff hheydaroff mentioned this pull request Apr 9, 2026
Closed
1 task
@trevorgordon981

trevorgordon981 commented Apr 9, 2026

Copy link
Copy Markdown

In agent/bedrock_adapter.py (lines 126-175), the _convert_content_to_converse function handles text and base64-encoded images but explicitly skips URL-referenced images (lines 165-167), noting they are "not supported by Converse."

Observation: If a user sends a message containing an image URL (e.g., https://...) instead of base64 data, such images are silently dropped.

Question: Have you considered how this adapter behaves in such scenarios? Should this trigger an error, a warning log, or a fallback mechanism (e.g., attempting to fetch and convert the URL to base64 before sending to Bedrock)? Specifically, what is the expected behavior for content blocks containing image_url objects with url fields pointing to external resources?

@hheydaroff hheydaroff force-pushed the feat/amazon-bedrock-provider branch from 2cafcb1 to e727c53 Compare April 10, 2026 13:03
@hheydaroff

hheydaroff commented Apr 10, 2026

Copy link
Copy Markdown
Author

@trevorgordon981 , yes you are right.
I made the changes.

URL images → fetched at call time, converted to bytes, sent to Converse. Supports jpeg/png/gif/webp, 25MB cap, 10s timeout.

Failure handling → logs a warning and skips the image (text parts still go through). No silent dropping.

@hheydaroff hheydaroff force-pushed the feat/amazon-bedrock-provider branch 2 times, most recently from f5fb0f3 to 3fbfd3a Compare April 10, 2026 13:16
@hheydaroff
feat: add Amazon Bedrock as inference provider
b4b7aa9 
Add Amazon Bedrock as a first-class inference provider using the Converse
API via boto3. Supports three authentication methods: Bedrock API keys
(Bearer tokens), AWS profiles (~/.aws/credentials), and AWS access key
pairs.

- agent/bedrock_adapter.py — Converse API adapter (message conversion,
  tool schema translation, response normalization)

- hermes_cli/auth.py — ProviderConfig, _resolve_bedrock_base_url(),
  region-aware endpoint construction, provider aliases
- hermes_cli/providers.py — HermesOverlay with bedrock_converse transport,
  aliases, labels
- hermes_cli/models.py — Curated model list with global/eu/us inference
  profile IDs and bare model IDs
- hermes_cli/config.py — AWS_BEARER_TOKEN_BEDROCK and AWS_BEDROCK_REGION
  env vars
- hermes_cli/runtime_provider.py — bedrock_converse in valid API modes
- agent/models_dev.py — PROVIDER_TO_MODELS_DEV mapping

- New api_mode 'bedrock_converse' with full lifecycle support:
  init, switch_model, _build_api_kwargs, _interruptible_api_call,
  response validation, finish_reason extraction, response normalization
- Safe vars() call in error logging (handles dict responses from boto3)

- Dedicated _model_flow_bedrock() with interactive setup:
  - Auth picker: existing API key, new API key, AWS profile, access keys
  - Region selector with common regions + custom input
  - Model selection from curated list
- Persists api_mode=bedrock_converse in config.yaml

- Added boto3 to pyproject.toml

Bedrock models use cross-region inference profile prefixes:
- global. — works from any region
- eu. — EU cross-region routing
- us. — US cross-region routing
- bare — single-region direct access
@hheydaroff hheydaroff force-pushed the feat/amazon-bedrock-provider branch from 3fbfd3a to b4b7aa9 Compare April 10, 2026 13:17
@JiaDe-Wu

JiaDe-Wu commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Hey @hheydaroff — saw your comment on #3863. We've got a parallel PR at #7920 that also uses the Converse API + boto3. Some differences worth noting:

  • We support both IAM credential chain AND Bedrock API Key (bedrock-mantle) — users pick during hermes model setup
  • Dynamic model discovery with ListFoundationModels + ListInferenceProfiles — auto-filters to text models, recommends cross-region profiles
  • Full ecosystem integration: hermes doctor, hermes auth, /usage pricing, error classification, context length table, Guardrails
  • IMDS credential detection (EC2 instance roles work without any env vars)
  • 107 automated tests + CloudFormation deployment template

Would be great to get one of these merged — the community clearly wants this (#3863 has been open since March). Happy to help consolidate if needed.

@hheydaroff hheydaroff closed this Apr 13, 2026
@hheydaroff

hheydaroff commented Apr 13, 2026

Copy link
Copy Markdown
Author

Closing it in favor of #7920

This was referenced Apr 15, 2026
Merged
Closed
@alt-glitch alt-glitch mentioned this pull request Apr 28, 2026
Closed
23 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hheydaroff @trevorgordon981 @JiaDe-Wu