What happened?
Using an Anthropic-compatible provider, the model returned thinking blocks even though thinking was set to off. The saved session JSONL shows those blocks have an empty thinkingSignature.
On the next turn, pi-ai cannot replay them as Anthropic thinking blocks because Anthropic requires a valid signature, so it converts the thinking text into normal content. This causes internal reasoning text to be sent back in the visible conversation context.
This is fine with claude models and anthropic provider, but most anthropic-compatible api don't provide meaningful thinkingSignature
Steps to reproduce
https://www.alibabacloud.com/help/en/model-studio/anthropic-api-messages#ah1a2b3c4d5e8 is an example of such provider which always returned an empty thinkingSignature. turns out they fixed it
I can confirm fireworks's anthropic endpoint also return empty thinkingSignature, though don't have a documented response example.
Expected behavior
I'm not entirely sure what the right behavior should be here. Maybe we could add a compatibility option, similar to what openai-completions.ts already has. Alternatively, we could enable it automatically for model IDs containing claude, which amazon-bedrocks.ts already use isAnthropicClaudeModel supportsThinkingSignature .
Version
0.74.0
What happened?
Using an Anthropic-compatible provider, the model returned thinking blocks even though thinking was set to off. The saved session JSONL shows those blocks have an empty thinkingSignature.
On the next turn, pi-ai cannot replay them as Anthropic thinking blocks because Anthropic requires a valid signature, so it converts the thinking text into normal content. This causes internal reasoning text to be sent back in the visible conversation context.
This is fine with claude models and anthropic provider, but most anthropic-compatible api don't provide meaningful thinkingSignature
Steps to reproduce
https://www.alibabacloud.com/help/en/model-studio/anthropic-api-messages#ah1a2b3c4d5e8 is an example of such provider which always returned an emptyturns out they fixed itthinkingSignature.I can confirm fireworks's anthropic endpoint also return empty thinkingSignature, though don't have a documented response example.
Expected behavior
I'm not entirely sure what the right behavior should be here. Maybe we could add a compatibility option, similar to what
openai-completions.tsalready has. Alternatively, we could enable it automatically for model IDs containingclaude, which amazon-bedrocks.ts already useisAnthropicClaudeModelsupportsThinkingSignature.Version
0.74.0