Skip to content

Anthropic-compatible replay sends unsigned thinking blocks back as user-visible content #4464

@pandada8

Description

@pandada8

What happened?

Using an Anthropic-compatible provider, the model returned thinking blocks even though thinking was set to off. The saved session JSONL shows those blocks have an empty thinkingSignature.

On the next turn, pi-ai cannot replay them as Anthropic thinking blocks because Anthropic requires a valid signature, so it converts the thinking text into normal content. This causes internal reasoning text to be sent back in the visible conversation context.

This is fine with claude models and anthropic provider, but most anthropic-compatible api don't provide meaningful thinkingSignature

Steps to reproduce

https://www.alibabacloud.com/help/en/model-studio/anthropic-api-messages#ah1a2b3c4d5e8 is an example of such provider which always returned an empty thinkingSignature. turns out they fixed it
I can confirm fireworks's anthropic endpoint also return empty thinkingSignature, though don't have a documented response example.

Expected behavior

I'm not entirely sure what the right behavior should be here. Maybe we could add a compatibility option, similar to what openai-completions.ts already has. Alternatively, we could enable it automatically for model IDs containing claude, which amazon-bedrocks.ts already use isAnthropicClaudeModel supportsThinkingSignature .

Version

0.74.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions