Can anyone provide a script of how to get Claude 3 Sonnet V2 on AWS Bedrock to work with the library? I think I am missing a parameter or something. When I run this script with my AWS environment variables set, it starts working and is able to find my credentials but when it gets to generating topics, it crashes because "The maximum tokens you requested exceeds the model limit of 8192. Try again with a maximum tokens value that is lower than 8192."
Simple script:
from annotateai import Annotate
annotate = Annotate("bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0")
annotate("https://arxiv.org/pdf/2406.14657")
Summarized Traceback:
10:51:27 - LiteLLM:INFO: utils.py:952 - Wrapper: Completed Call, calling success_handler
2025-01-21 10:51:27,736 - LiteLLM - INFO - Wrapper: Completed Call, calling success_handler
Generating keywords: 100%|██████████| 1/1 [00:01<00:00, 1.72s/it]
Generating annotations: 100%|██████████| 24/24 [00:17<00:00, 1.41it/s]
Generating topics: 0%| | 0/106 [00:00<?, ?it/s]10:51:44 - LiteLLM:INFO: utils.py:2820 -
LiteLLM completion() model= anthropic.claude-3-5-sonnet-20241022-v2:0; provider = bedrock
2025-01-21 10:51:44,816 - LiteLLM - INFO -
LiteLLM completion() model= anthropic.claude-3-5-sonnet-20241022-v2:0; provider = bedrock
2025-01-21 10:51:45,118 - httpx - INFO - HTTP Request: POST https://bedrock-runtime.us-west-2.amazonaws.com/model/anthropic.claude-3-5-sonnet-20241022-v2:0/converse "HTTP/1.1 400 Bad Request"
raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: BedrockException - {"message":"The maximum tokens you requested exceeds the model limit of 8192. Try again with a maximum tokens value that is lower than 8192."}
Can anyone provide a script of how to get Claude 3 Sonnet V2 on AWS Bedrock to work with the library? I think I am missing a parameter or something. When I run this script with my AWS environment variables set, it starts working and is able to find my credentials but when it gets to generating topics, it crashes because "The maximum tokens you requested exceeds the model limit of 8192. Try again with a maximum tokens value that is lower than 8192."
Simple script:
Summarized Traceback: