Conversation
There was a problem hiding this comment.
Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.
There was a problem hiding this comment.
Code Review
This pull request introduces support for Anthropic's top-level automatic prompt caching. The changes ensure that the cache_control field is captured during inbound request parsing, propagated through the internal pipeline via TransformerMetadata, and restored in the outbound request to the upstream API. When automatic caching is active, the system's internal per-block breakpoint optimization is bypassed. Feedback was provided to explicitly clear any existing per-block cache controls when the top-level field is present to avoid potential 400 Bad Request errors from the Anthropic API due to mixed caching modes.
| if anthropicReq.CacheControl == nil && countCacheControls(anthropicReq) > 0 { | ||
| optimizeCacheControl(anthropicReq) | ||
| } |
There was a problem hiding this comment.
Anthropic does not allow mixing automatic prompt caching (top-level cache_control) with explicit cache breakpoints (per-block cache_control). If the top-level field is present, any existing per-block breakpoints should be cleared to avoid a 400 Bad Request error from the upstream API. Since optimizeCacheControl already performs a clear-and-rebuild, we should explicitly clear when skipping optimization.
| if anthropicReq.CacheControl == nil && countCacheControls(anthropicReq) > 0 { | |
| optimizeCacheControl(anthropicReq) | |
| } | |
| if anthropicReq.CacheControl != nil { | |
| clearCacheControls(anthropicReq) | |
| } else if countCacheControls(anthropicReq) > 0 { | |
| optimizeCacheControl(anthropicReq) | |
| } |
Uh oh!
There was an error while loading. Please reload this page.