fix(gateway): only mark final response sent when split-overflow chunks actually land by teknium1 · Pull Request #23420 · NousResearch/hermes-agent

teknium1 · 2026-05-10T21:25:01Z

Summary

Telegram streaming used to silently drop the final answer after tool calls — users saw only the "working on it" tool-progress bubbles. The split-overflow path was claiming "final delivered" based on a cumulative flag that didn't reflect whether the current chunked send actually landed.

What was wrong

_send_or_edit in gateway/stream_consumer.py runs a chunked-send loop when accumulated text exceeds the platform limit and there's no message to edit. On the done frame it was copying self._already_sent into self._final_response_sent:

for chunk in chunks:
    await self._send_new_chunk(chunk, self._message_id)  # return value ignored
...
if got_done:
    self._final_response_sent = self._already_sent  # WRONG — doesn't reflect *these* chunks
    return

_already_sent goes True on any successful prior edit (e.g. tool-progress display) and on fallback-mode promotion when an edit fails. Neither proves the chunked send above delivered the final answer. When all chunks failed, the consumer told the gateway "final delivered" and the gateway's fallback path in run.py got suppressed. The answer never reached the user.

Fix

Track per-chunk success locally. _send_new_chunk returns the new message_id on success or the passed-in reply_to unchanged on failure, so a returned id that differs from the input means the chunk landed:

chunks_delivered = False
reply_to = self._message_id
for chunk in chunks:
    new_id = await self._send_new_chunk(chunk, reply_to)
    if new_id is not None and new_id != reply_to:
        chunks_delivered = True
...
if got_done:
    self._final_response_sent = chunks_delivered
    return

If no chunks landed → _final_response_sent stays False → gateway runs its independent fallback.

Note on the issue's second cited site

The reporter cited two locations. The CancelledError handler at L417-418 was already fixed by commit 3b5572ded on 2026-04-16 (fix(stream-consumer): only confirm final delivery on successful best-effort send). Only the split-overflow site remained.

Validation

tests/gateway/test_stream_consumer.py: 86/86 pass (2 new + 84 existing).
New TestFinalResponseDeliveryGuard class with two tests:
- test_split_overflow_failed_send_does_not_mark_final_sent — primes _already_sent=True, makes every send fail, asserts _final_response_sent stays False.
- test_split_overflow_partial_send_marks_final_sent — happy path, asserts _final_response_sent goes True.

Closes #10748.

…s actually land The split-overflow path in _send_or_edit (gateway/stream_consumer.py) was copying the cumulative _already_sent flag into _final_response_sent on the done frame. _already_sent goes True on any successful prior edit (tool progress) or on fallback-mode promotion when an edit fails — neither proves the *current* chunked send delivered the final answer. When the chunked send actually fails (network error, flood control), the consumer would wrongly claim 'final delivered' and the gateway's independent fallback delivery in run.py would be suppressed. User saw only tool-progress bubbles and never got the answer. Now we track per-chunk success locally: _send_new_chunk returns the new message_id on success or returns the passed-in reply_to unchanged on failure. If at least one returned id differs, chunks_delivered = True; otherwise stays False, gateway fallback runs. Adds two regression tests: - test_split_overflow_failed_send_does_not_mark_final_sent — primes _already_sent=True, then makes every send fail; asserts _final_response_sent stays False. - test_split_overflow_partial_send_marks_final_sent — happy path, asserts _final_response_sent goes True. Note: the companion bug at the CancelledError handler (issue cited lines 417-418) was already fixed by 3b5572d on 2026-04-16. Closes #10748

github-actions · 2026-05-10T21:26:09Z

🔎 Lint report: `salvage/issue-10748` vs `origin/main`

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 8074 on HEAD, 8074 on base (➖ 0)

🆕 New issues (3):

Rule	Count
`invalid-argument-type`	3

First entries

run_agent.py:7160: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`
run_agent.py:13287: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown, Unknown] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:13284: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`

✅ Fixed issues (3):

Rule	Count
`invalid-argument-type`	3

First entries

run_agent.py:13287: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:13284: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown | str, Unknown | str | dict[str, str]] | Any | ... omitted 3 union elements`
run_agent.py:7160: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown | str, Unknown | str | dict[str, str]] | Any | ... omitted 3 union elements`

Unchanged: 4251 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

…s actually land (NousResearch#23420) The split-overflow path in _send_or_edit (gateway/stream_consumer.py) was copying the cumulative _already_sent flag into _final_response_sent on the done frame. _already_sent goes True on any successful prior edit (tool progress) or on fallback-mode promotion when an edit fails — neither proves the *current* chunked send delivered the final answer. When the chunked send actually fails (network error, flood control), the consumer would wrongly claim 'final delivered' and the gateway's independent fallback delivery in run.py would be suppressed. User saw only tool-progress bubbles and never got the answer. Now we track per-chunk success locally: _send_new_chunk returns the new message_id on success or returns the passed-in reply_to unchanged on failure. If at least one returned id differs, chunks_delivered = True; otherwise stays False, gateway fallback runs. Adds two regression tests: - test_split_overflow_failed_send_does_not_mark_final_sent — primes _already_sent=True, then makes every send fail; asserts _final_response_sent stays False. - test_split_overflow_partial_send_marks_final_sent — happy path, asserts _final_response_sent goes True. Note: the companion bug at the CancelledError handler (issue cited lines 417-418) was already fixed by 3b5572d on 2026-04-16. Closes NousResearch#10748

…s actually land (NousResearch#23420) The split-overflow path in _send_or_edit (gateway/stream_consumer.py) was copying the cumulative _already_sent flag into _final_response_sent on the done frame. _already_sent goes True on any successful prior edit (tool progress) or on fallback-mode promotion when an edit fails — neither proves the *current* chunked send delivered the final answer. When the chunked send actually fails (network error, flood control), the consumer would wrongly claim 'final delivered' and the gateway's independent fallback delivery in run.py would be suppressed. User saw only tool-progress bubbles and never got the answer. Now we track per-chunk success locally: _send_new_chunk returns the new message_id on success or returns the passed-in reply_to unchanged on failure. If at least one returned id differs, chunks_delivered = True; otherwise stays False, gateway fallback runs. Adds two regression tests: - test_split_overflow_failed_send_does_not_mark_final_sent — primes _already_sent=True, then makes every send fail; asserts _final_response_sent stays False. - test_split_overflow_partial_send_marks_final_sent — happy path, asserts _final_response_sent goes True. Note: the companion bug at the CancelledError handler (issue cited lines 417-418) was already fixed by c64f9c7 on 2026-04-16. Closes NousResearch#10748

…s actually land (NousResearch#23420) The split-overflow path in _send_or_edit (gateway/stream_consumer.py) was copying the cumulative _already_sent flag into _final_response_sent on the done frame. _already_sent goes True on any successful prior edit (tool progress) or on fallback-mode promotion when an edit fails — neither proves the *current* chunked send delivered the final answer. When the chunked send actually fails (network error, flood control), the consumer would wrongly claim 'final delivered' and the gateway's independent fallback delivery in run.py would be suppressed. User saw only tool-progress bubbles and never got the answer. Now we track per-chunk success locally: _send_new_chunk returns the new message_id on success or returns the passed-in reply_to unchanged on failure. If at least one returned id differs, chunks_delivered = True; otherwise stays False, gateway fallback runs. Adds two regression tests: - test_split_overflow_failed_send_does_not_mark_final_sent — primes _already_sent=True, then makes every send fail; asserts _final_response_sent stays False. - test_split_overflow_partial_send_marks_final_sent — happy path, asserts _final_response_sent goes True. Note: the companion bug at the CancelledError handler (issue cited lines 417-418) was already fixed by 3b5572d on 2026-04-16. Closes NousResearch#10748

…s actually land (NousResearch#23420) The split-overflow path in _send_or_edit (gateway/stream_consumer.py) was copying the cumulative _already_sent flag into _final_response_sent on the done frame. _already_sent goes True on any successful prior edit (tool progress) or on fallback-mode promotion when an edit fails — neither proves the *current* chunked send delivered the final answer. When the chunked send actually fails (network error, flood control), the consumer would wrongly claim 'final delivered' and the gateway's independent fallback delivery in run.py would be suppressed. User saw only tool-progress bubbles and never got the answer. Now we track per-chunk success locally: _send_new_chunk returns the new message_id on success or returns the passed-in reply_to unchanged on failure. If at least one returned id differs, chunks_delivered = True; otherwise stays False, gateway fallback runs. Adds two regression tests: - test_split_overflow_failed_send_does_not_mark_final_sent — primes _already_sent=True, then makes every send fail; asserts _final_response_sent stays False. - test_split_overflow_partial_send_marks_final_sent — happy path, asserts _final_response_sent goes True. Note: the companion bug at the CancelledError handler (issue cited lines 417-418) was already fixed by d52cb31 on 2026-04-16. Closes NousResearch#10748

…s actually land (NousResearch#23420) The split-overflow path in _send_or_edit (gateway/stream_consumer.py) was copying the cumulative _already_sent flag into _final_response_sent on the done frame. _already_sent goes True on any successful prior edit (tool progress) or on fallback-mode promotion when an edit fails — neither proves the *current* chunked send delivered the final answer. When the chunked send actually fails (network error, flood control), the consumer would wrongly claim 'final delivered' and the gateway's independent fallback delivery in run.py would be suppressed. User saw only tool-progress bubbles and never got the answer. Now we track per-chunk success locally: _send_new_chunk returns the new message_id on success or returns the passed-in reply_to unchanged on failure. If at least one returned id differs, chunks_delivered = True; otherwise stays False, gateway fallback runs. Adds two regression tests: - test_split_overflow_failed_send_does_not_mark_final_sent — primes _already_sent=True, then makes every send fail; asserts _final_response_sent stays False. - test_split_overflow_partial_send_marks_final_sent — happy path, asserts _final_response_sent goes True. Note: the companion bug at the CancelledError handler (issue cited lines 417-418) was already fixed by 3b5572d on 2026-04-16. Closes NousResearch#10748

teknium1 merged commit 6636fec into main May 10, 2026
15 of 18 checks passed

teknium1 deleted the salvage/issue-10748 branch May 10, 2026 22:13

This was referenced May 10, 2026

fix(gateway): stream consumer first message drops thread context (salvage of #13077) #23437

Merged

fix(gateway): stream consumer first message drops thread context #13077

Closed

spiky02plateau mentioned this pull request Jun 2, 2026

fix(gateway): require all overflow chunks before final delivery #37638

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(gateway): only mark final response sent when split-overflow chunks actually land#23420

fix(gateway): only mark final response sent when split-overflow chunks actually land#23420
teknium1 merged 1 commit into
mainfrom
salvage/issue-10748

teknium1 commented May 10, 2026

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

teknium1 commented May 10, 2026

Summary

What was wrong

Fix

Note on the issue's second cited site

Validation

Uh oh!

github-actions Bot commented May 10, 2026

🔎 Lint report: salvage/issue-10748 vs origin/main

ruff

ty (type checker)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

🔎 Lint report: `salvage/issue-10748` vs `origin/main`