Fix bedrock anthropic models to use system field#3438
Fix bedrock anthropic models to use system field#3438parshvadaftari merged 3 commits intomem0ai:mainfrom
Conversation
Signed-off-by: Andrew Carbonetto <andrew.carbonetto@improving.com>
Signed-off-by: Andrew Carbonetto <andrew.carbonetto@improving.com>
parshvadaftari
left a comment
There was a problem hiding this comment.
Please check the comments and incorporate the requested changes.
tests/memory/na_memory_example.py
Outdated
| @@ -0,0 +1,96 @@ | |||
| from mem0 import Memory | |||
There was a problem hiding this comment.
This is not required for the tests file. Please remove it.
There was a problem hiding this comment.
Yes. Will remove this test file.
Removed under 6c15e6d
| raise RuntimeError(f"Failed to generate response: {e}") | ||
|
|
||
| @staticmethod | ||
| def _convert_tools_to_converse_format(tools: List[Dict]) -> List[Dict]: |
There was a problem hiding this comment.
Will this fix it for all the LLM providers?
There was a problem hiding this comment.
It works for the anthropic llm models I tested. This should only affect the bedrock provider.
I'm not able to get amazon (e.g. nova-lite) models to work with either the released branch or this branch (I get a different error). So I'm not sure they are supported yet...?
Same story for openai's gpt-oss-120b model - but different error.
There was a problem hiding this comment.
So this fix resolves the error with the different providers? or specifically for anthropic.
There was a problem hiding this comment.
specifically for anthropic
There was a problem hiding this comment.
Cool then looks good to me. Will be releasing the fix for AWS issue with other models. It's difficult to recreate it neverthless will update it.
Signed-off-by: Andrew Carbonetto <andrew.carbonetto@improving.com>
parshvadaftari
left a comment
There was a problem hiding this comment.
Looks good to me.
Signed-off-by: Andrew Carbonetto <andrew.carbonetto@improving.com>
Description
Using bedrock against an anthropic model caused errors (see #3378). When using anthropic models, the system prompts were being ignored, and the response wasn't being returned in the proper format (causing errors). Anthropic models take a separate
systemfield when passing messages, which is now being used. See: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/system-promptsFixes #3378
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Please delete options that are not relevant.
Checklist:
Maintainer Checklist