Skip to content

fix(ai): send store: false on Azure OpenAI Responses requests#5524

Closed
Jaxkr wants to merge 1 commit into
earendil-works:mainfrom
Jaxkr:fix/azure-responses-store-false
Closed

fix(ai): send store: false on Azure OpenAI Responses requests#5524
Jaxkr wants to merge 1 commit into
earendil-works:mainfrom
Jaxkr:fix/azure-responses-store-false

Conversation

@Jaxkr

@Jaxkr Jaxkr commented Jun 8, 2026

Copy link
Copy Markdown

To maintainers: This is a THREE LINE change which fixes a nasty bug. Also I opened this before I read CONTRIBUTING.md telling me not to open PRs. Sorry! Please don't ban me!

Summary

packages/ai/src/providers/azure-openai-responses.ts requests include: ["reasoning.encrypted_content"] (the stateless reasoning-replay path) but never sets store, so Azure runs in its default stateful mode. The sibling openai-responses and openai-codex-responses providers both set store: false alongside the same include; Azure is the only Responses provider that omits it.

Symptom

Intermittent 400 Item with id 'rs_...' not found on multi-turn conversations, which clears if you retry the same request.

Root cause

When a transcript is replayed, convertResponsesMessages re-emits each prior assistant reasoning item with its original rs_* id inlined in input (the item is round-tripped through thinkingSignature). In stateful mode Azure tries to resolve that rs_* id against its server-side store rather than using the inlined encrypted_content. Azure's Responses endpoint is load-balanced and stored items are tied to a specific response/backend with a retention window, so whether the lookup succeeds is non-deterministic — replays that land on a backend without the item fail, and retrying eventually routes to one that resolves (or regenerates), which is why "try again" clears it.

This bites hardest when the transcript is rebuilt from persisted messages each turn (no previous_response_id threading), so the replayed reasoning ids reference responses the serving backend no longer has.

Fix

Set store: false in buildParams. With store: false + include: ["reasoning.encrypted_content"], every request is self-contained: the reasoning travels inline as encrypted content and Azure never does a store lookup, so the call is deterministic. This matches the existing behavior of openai-responses and openai-codex-responses.

 	const params: ResponseCreateParamsStreaming = {
 		model: deploymentName,
 		input: messages,
 		stream: true,
 		prompt_cache_key: clampOpenAIPromptCacheKey(options?.sessionId),
+		store: false,
 	};

Test

Extended packages/ai/test/azure-openai-base-url.test.ts to assert the provider sends store: false. npm run check and the full packages/ai test suite pass (323 passed, e2e/keyed tests skipped).

The azure-openai-responses provider sets include: ["reasoning.encrypted_content"] to replay reasoning statelessly, but never set store, so Azure ran in its default stateful mode. Replayed transcripts re-send prior reasoning items with their rs_* ids, which Azure tries to resolve against its server-side store; on a rebuilt transcript those ids are usually absent, producing an intermittent 400 "Item with id 'rs_...' not found" that clears on retry. Set store: false so each request is self-contained via encrypted_content, matching openai-responses and openai-codex-responses.
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

This PR was auto-closed. Only contributors approved with lgtm can open PRs. Open an issue first.

Maintainers review auto-closed issues daily. Issues that do not meet the quality bar in CONTRIBUTING.md will not be reopened or receive a reply.

If a maintainer replies lgtmi, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open.

See CONTRIBUTING.md.

@github-actions github-actions Bot closed this Jun 8, 2026
input: messages,
stream: true,
prompt_cache_key: clampOpenAIPromptCacheKey(options?.sessionId),
// Reasoning is replayed statelessly via reasoning.encrypted_content (set below),

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed these comments on my latest commit in my branch.

@Jaxkr Jaxkr changed the title fix(ai): send store: false on Azure Responses requests fix(ai): send store: false on Azure OpenAI Responses requests Jun 8, 2026
@Jaxkr

Jaxkr commented Jun 8, 2026

Copy link
Copy Markdown
Author

I also made it unconditionally include reasoning.encrypted_content which is the correct implementation. GPT 5 can reason even when you put effortLevel = null.

The latest commits on my branch have this too (this PR isn't updating due to it being closed)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant