Skip to content

Python: fix background=True + tools infinite-retrieve loop#5462

Merged
eavanvalkenburg merged 1 commit intomicrosoft:mainfrom
he-yufeng:fix/background-tool-loop-continuation-leak
May 4, 2026
Merged

Python: fix background=True + tools infinite-retrieve loop#5462
eavanvalkenburg merged 1 commit intomicrosoft:mainfrom
he-yufeng:fix/background-tool-loop-continuation-leak

Conversation

@he-yufeng
Copy link
Copy Markdown
Contributor

Fixes #5394.

The bug

When a caller combines background=True with local function tools, the agent gets stuck retrieving the same completed response on every tool-loop iteration and tool results are never POSTed. After max_iterations the loop exits with empty text.

Root cause

FunctionInvocationLayer.get_response() builds a single mutable_options = dict(options) dict and threads it through every iteration of the tool loop via super_get_response(options=mutable_options). RawOpenAIChatClient._inner_get_response reads continuation_token from that dict and, when present, takes the responses.retrieve(response_id) branch.

After the very first retrieve returns a completed background response, continuation_token is still sitting in mutable_options. Every subsequent iteration therefore re-enters the retrieve branch and hits the same completed response again — never the responses.create/parse path that would POST the tool results.

The HTTP trace from the issue confirms it: 1 POST followed by ~40 GETs of the same response_id, and no tool-result POSTs.

Fix

In _inner_get_response, after the non-streaming retrieve completes, check whether the returned ChatResponse still carries a continuation_token. If it doesn't (i.e. the background operation is no longer in progress), pop continuation_token and background from the caller's options dict in place. FunctionInvocationLayer passes the same dict reference across iterations, so the mutation makes the next iteration fall through to _prepare_requestresponses.create(...) and the tool results flow normally.

This is the same change the reporter (@Laende) verified as a runtime monkeypatch, lifted into _chat_client.py.

Scope

  • Non-streaming path only. The streaming continuation branch (responses.retrieve(stream=True)) yields chunks inside an async generator and isn't exercised by the tool loop in the same way; leaving that out until someone sees a real reproduction.
  • No public API change; the dict being mutated is the options argument that's already owned and re-used by the caller.

Test plan

  • Reporter's repro in Python: [Bug]: background=True causes infinite tool-call loop, tool-result submissions inherit background mode #5394 (agent with background=True + function tools): after this fix the HTTP trace should match the monkeypatched trace in the issue — 1 POST, a few GETs while the background response is in progress, then the tool-result POSTs that let the model produce a final answer.
  • Existing agent/chat-client tests pass with ruff check / ruff format green locally; happy to add a regression test if a reviewer points me at the right mock scaffolding for client.responses.retrieve.

Comment thread python/packages/openai/agent_framework_openai/_chat_client.py Outdated
… tool loop

Fixes microsoft#5394.

When `background=True` is combined with local function tools,
`FunctionInvocationLayer` calls `_inner_get_response(options=mutable_options)`
repeatedly with the same dict reference across loop iterations. Once the
first poll retrieves a completed background response, `continuation_token`
stays in `mutable_options`, so every subsequent iteration takes the
`continuation_token is not None` branch and `GET`s the same completed
response instead of `POST`ing the tool results. The loop exits after
`max_iterations` with empty text and the model never sees any tool output.

After the retrieve, if the returned `ChatResponse.continuation_token` is
`None` (the background response is no longer in progress), pop
`continuation_token` and `background` from the shared options dict in
place. The next loop iteration then falls through to the normal
`responses.create`/`parse` path and posts tool results.

The diagnosis and a verified runtime monkeypatch are in the issue; this
is the same fix moved in-tree.
@he-yufeng he-yufeng force-pushed the fix/background-tool-loop-continuation-leak branch from 1ed0f6a to 8bfa009 Compare April 25, 2026 06:00
@he-yufeng
Copy link
Copy Markdown
Contributor Author

@eavanvalkenburg good catch — kept background so subsequent iterations still POST as background. Pushed 8bfa0098 with that change and updated the comment to make the rationale clearer.

@he-yufeng
Copy link
Copy Markdown
Contributor Author

Friendly ping @eavanvalkenburg — pushed 8bfa0098 per your suggestion (kept background, only drop the stale continuation_token). Mind a quick re-look when you get a chance?

@moonbox3
Copy link
Copy Markdown
Contributor

moonbox3 commented May 4, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/openai/agent_framework_openai
   _chat_client.py91912486%522–525, 529–530, 536–537, 572–578, 585–587, 608, 616, 639, 757, 856, 915, 917, 919, 921, 987, 1001, 1081, 1091, 1096, 1139, 1255, 1436, 1441, 1445–1447, 1451–1452, 1518, 1547, 1553, 1563, 1569, 1574, 1580, 1585–1586, 1605, 1695, 1717–1718, 1733–1734, 1752–1753, 1796, 1962, 2000–2001, 2017, 2019, 2099–2107, 2137, 2247, 2282, 2297, 2317–2327, 2340, 2351–2355, 2369, 2383–2394, 2403, 2435–2438, 2448–2449, 2460–2462, 2476–2478, 2488–2489, 2495, 2510
TOTAL29868348288% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
6023 30 💤 0 ❌ 0 🔥 1m 37s ⏱️

@eavanvalkenburg eavanvalkenburg enabled auto-merge May 4, 2026 10:30
@he-yufeng
Copy link
Copy Markdown
Contributor Author

Thanks @eavanvalkenburg @moonbox3 for the reviews. CI is green, branch is mergeable — happy to have anyone with merge rights pull the trigger when convenient.

@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue May 4, 2026
Merged via the queue into microsoft:main with commit 330d3d7 May 4, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: background=True causes infinite tool-call loop, tool-result submissions inherit background mode

3 participants