Fix Bedrock Anthropic adapter by adding required `anthropic_version` field by harupy · Pull Request #17744 · mlflow/mlflow

harupy · 2025-09-16T01:50:37Z

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/17744/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/17744/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/17744/merge

Related Issues/PRs

Fixes #17699

What changes are proposed in this pull request?

This PR fixes an issue where LLM scorers (both built-in and custom) couldn't work with Amazon Bedrock Claude models. The root cause was that the Bedrock API requires an anthropic_version field when invoking Anthropic models, which was missing from the request payload.

The fix adds the required anthropic_version field with the value "bedrock-2023-05-31" to the Bedrock Anthropic adapter's payload transformation.

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

The existing tests should cover this change. The fix ensures that the required field is always present in the payload when using Bedrock with Anthropic models.

Does this PR require documentation update?

No. You can skip the rest of this section.

Release Notes

Is this a user-facing change?

Yes. Give a description of this change to be included in the release notes for MLflow users.

This fix enables LLM scorers (such as Correctness, Guidelines, and custom prompt judges) to work correctly with Amazon Bedrock Claude models by including the required anthropic_version field in API requests.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

How should the PR be classified in the release notes? Choose one:

rn/bug-fix - A user-facing bug fix worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

🤖 Generated with Claude Code

github-actions · 2025-09-16T02:03:40Z

Documentation preview for 5b2eb78 is available at:

https://pr-17744--mlflow-docs-preview.netlify.app/docs/latest/

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

serena-ruan

LGTM!

ravees · 2025-11-12T18:17:37Z

Hi @harupy @serena-ruan Any update on the PR merge?
Facing this issue for v3.6.0 mlflow.genai.scorers Correctness, and
Guidelines

ravees · 2025-11-12T19:58:51Z

The below mlflow.genai.evaluate samples with "anthropic_version": "bedrock-2023-05-31" field. Can you please verify the patch solution?

## 1.Run the evaluation with parameters including anthropic_version
results = mlflow.genai.evaluate(
    data=dataset,
    predict_fn=predict_fn,
    scorers=[
        # Built-in LLM judge
        Correctness(
            model="bedrock:/global.anthropic.claude-haiku-4-5-20251001-v1:0",
            parameters={
                "temperature": 0,
                "max_tokens": 256,
                "anthropic_version": "bedrock-2023-05-31",
            },
            ),
        # Custom criteria using LLM judge
        Guidelines(name="is_english", guidelines="The answer must be in English", 
        model="bedrock:/global.anthropic.claude-haiku-4-5-20251001-v1:0",
        parameters={
                "temperature": 0,
                "max_tokens": 256,
                "anthropic_version": "bedrock-2023-05-31",
            },
        ),
    ],
)



## 2.Run the evaluation with anthropic_version
results = mlflow.genai.evaluate(
    data=dataset,
    predict_fn=predict_fn,
    scorers=[
        # Built-in LLM judge
        Correctness(
            model="bedrock:/us.anthropic.claude-3-7-sonnet-20250219-v1:0",
            anthropic_version="bedrock-2023-05-31",
            ),
        # Custom criteria using LLM judge
        Guidelines(name="is_english", guidelines="The answer must be in English", 
        model="bedrock:/us.anthropic.claude-3-7-sonnet-20250219-v1:0",
        anthropic_version="bedrock-2023-05-31",
        ),
    ],
)

This change fixes an issue where LLM scorers couldn't work with Amazon Bedrock Claude models. The Bedrock API requires an 'anthropic_version' field when invoking Anthropic models, which was missing from the request payload. The fix adds the required 'anthropic_version' field with the appropriate version string 'bedrock-2023-05-31' to ensure compatibility with Bedrock's Anthropic model invocations. Fixes mlflow#17699 Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

The tests were failing because they were expecting request payloads without the anthropic_version field. This commit updates the test expectations to include the required anthropic_version field that was added to fix compatibility with Bedrock's Anthropic API. Signed-off-by: Harutaka Kawamura <harutaka@users.noreply.github.com> Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy · 2025-11-13T02:24:18Z

@ravees Thank for testing. I pushed 5b0d635. Can you test again?

harupy · 2025-11-14T15:46:50Z

@ravees did you have a chance to test?

ravees · 2025-11-14T20:23:57Z

Hi @harupy , tested with the new fix and it worked for bedrock antropic models like bedrock:/global.anthropic.claude-haiku-4-5-20251001-v1:0.
Which MLflow version will this fix be part of and when can we expect this to be released?
Thank you so much for the timely fix!

harupy · 2025-11-15T02:26:33Z

@ravees Thanks for testing! This patch will be included in the 3.7.0 release, which we’re targeting for the end of November.

…field (mlflow#17744) Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> Signed-off-by: Harutaka Kawamura <harutaka@users.noreply.github.com> Signed-off-by: Jackie Himel <jacqueline.himel@vanderbilt.edu>

…field (mlflow#17744) Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> Signed-off-by: Harutaka Kawamura <harutaka@users.noreply.github.com>

…field (mlflow#17744) Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> Signed-off-by: Harutaka Kawamura <harutaka@users.noreply.github.com> Signed-off-by: Tian Lan <sky.blue266000@gmail.com>

github-actions bot added area/evaluation MLflow Evaluation area/scoring MLflow Model server, model deployment tools, Spark UDFs rn/bug-fix Mention under Bug Fixes in Changelogs. labels Sep 16, 2025

harupy mentioned this pull request Sep 16, 2025

[BUG] Can't use LLM scorers with Amazon Bedrock (and probably some other model providers) #17699

Closed

11 tasks

harupy changed the title ~~Fix Bedrock Anthropic adapter by adding required anthropic_version field~~ Fix Bedrock Anthropic adapter by adding required anthropic_version field Sep 16, 2025

harupy added the team-review Trigger a team review request label Sep 16, 2025

github-actions bot requested review from B-Step62, BenWilson2, TomeHirata, WeichenXu123, daniellok-db, serena-ruan and xq-yin September 16, 2025 02:59

serena-ruan approved these changes Sep 16, 2025

View reviewed changes

github-actions bot added the v3.4.1 label Oct 7, 2025

harupy added 2 commits November 13, 2025 11:19

harupy force-pushed the fix-bedrock-anthropic-version branch from 108d6a0 to 7dd2904 Compare November 13, 2025 02:20

Add anthropic_version in chat_to_model

5b0d635

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

harupy added v3.6.1 and removed v3.4.1 labels Nov 13, 2025

Merge branch 'master' into fix-bedrock-anthropic-version

5b2eb78

harupy added this pull request to the merge queue Nov 15, 2025

Merged via the queue into mlflow:master with commit 5cdb505 Nov 15, 2025
45 checks passed

harupy deleted the fix-bedrock-anthropic-version branch November 15, 2025 02:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Bedrock Anthropic adapter by adding required `anthropic_version` field#17744

Fix Bedrock Anthropic adapter by adding required `anthropic_version` field#17744
harupy merged 4 commits intomlflow:masterfrom
harupy:fix-bedrock-anthropic-version

harupy commented Sep 16, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

serena-ruan left a comment

Uh oh!

ravees commented Nov 12, 2025

Uh oh!

ravees commented Nov 12, 2025

Uh oh!

harupy commented Nov 13, 2025 •

edited

Loading

Uh oh!

harupy commented Nov 14, 2025

Uh oh!

ravees commented Nov 14, 2025

Uh oh!

harupy commented Nov 15, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

harupy commented Sep 16, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Install mlflow from this PR

Related Issues/PRs

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Should this PR be included in the next patch release?

Uh oh!

github-actions bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serena-ruan left a comment

Choose a reason for hiding this comment

Uh oh!

ravees commented Nov 12, 2025

Uh oh!

ravees commented Nov 12, 2025

Uh oh!

harupy commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

harupy commented Nov 14, 2025

Uh oh!

ravees commented Nov 14, 2025

Uh oh!

harupy commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

harupy commented Sep 16, 2025 •

edited by github-actions bot

Loading

github-actions bot commented Sep 16, 2025 •

edited

Loading

harupy commented Nov 13, 2025 •

edited

Loading

harupy commented Nov 15, 2025 •

edited

Loading