test(tui_gateway): stop reloading server module in fixture teardown#34217
Merged
Conversation
tui_gateway.server registers two atexit hooks at module load time:
ThreadPoolExecutor shutdown (line 170) and _shutdown_sessions (line 336).
Three test files reloaded the module on each fixture teardown to reset
per-test state. Each reload re-runs module-level code, including the
atexit registrations — duplicates accumulate across the test session.
At pytest interpreter shutdown the duplicated atexit hooks race the
stderr buffer flush:
Fatal Python error: _enter_buffered_busy: could not acquire lock
for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown,
possibly due to daemon threads
pytest reports 'tests passed but the slice exited non-zero', and the
shard turns red on CI. Surfaced today on PR #34193's test slice 1
(204 files, 3572 tests passed, then Fatal Python error during exit).
Fix: drop importlib.reload(mod) from the three fixtures that have it.
Per-test reset is handled by clearing the mutable session dicts
(_sessions, _pending, _answers). _methods is also no longer cleared —
it's populated at module import time and would only be re-populated by
a reload, so clearing it without reload broke session.resume /
command.dispatch / slash.exec method registration across tests.
Affected fixtures:
- tests/tui_gateway/test_goal_command.py
- tests/tui_gateway/test_protocol.py
- tests/tui_gateway/test_review_summary_callback.py
The second reload in test_protocol.py at line 211 (reload of
tui_gateway.transport) is preserved — transport.py has no atexit hooks
or threads, so reload is safe there.
Tests: 84/84 in tests/tui_gateway/ pass cleanly with exit code 0; no
Fatal Python error at interpreter shutdown.
Contributor
🔎 Lint report:
|
Bryce-huang
pushed a commit
to wbkunlun/hermes-agent
that referenced
this pull request
May 29, 2026
…ousResearch#34217) tui_gateway.server registers two atexit hooks at module load time: ThreadPoolExecutor shutdown (line 170) and _shutdown_sessions (line 336). Three test files reloaded the module on each fixture teardown to reset per-test state. Each reload re-runs module-level code, including the atexit registrations — duplicates accumulate across the test session. At pytest interpreter shutdown the duplicated atexit hooks race the stderr buffer flush: Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads pytest reports 'tests passed but the slice exited non-zero', and the shard turns red on CI. Surfaced today on PR NousResearch#34193's test slice 1 (204 files, 3572 tests passed, then Fatal Python error during exit). Fix: drop importlib.reload(mod) from the three fixtures that have it. Per-test reset is handled by clearing the mutable session dicts (_sessions, _pending, _answers). _methods is also no longer cleared — it's populated at module import time and would only be re-populated by a reload, so clearing it without reload broke session.resume / command.dispatch / slash.exec method registration across tests. Affected fixtures: - tests/tui_gateway/test_goal_command.py - tests/tui_gateway/test_protocol.py - tests/tui_gateway/test_review_summary_callback.py The second reload in test_protocol.py at line 211 (reload of tui_gateway.transport) is preserved — transport.py has no atexit hooks or threads, so reload is safe there. Tests: 84/84 in tests/tui_gateway/ pass cleanly with exit code 0; no Fatal Python error at interpreter shutdown. #AI commit#
KKT-OPT
pushed a commit
to KKT-OPT/hermes-agent
that referenced
this pull request
May 31, 2026
…ousResearch#34217) tui_gateway.server registers two atexit hooks at module load time: ThreadPoolExecutor shutdown (line 170) and _shutdown_sessions (line 336). Three test files reloaded the module on each fixture teardown to reset per-test state. Each reload re-runs module-level code, including the atexit registrations — duplicates accumulate across the test session. At pytest interpreter shutdown the duplicated atexit hooks race the stderr buffer flush: Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads pytest reports 'tests passed but the slice exited non-zero', and the shard turns red on CI. Surfaced today on PR NousResearch#34193's test slice 1 (204 files, 3572 tests passed, then Fatal Python error during exit). Fix: drop importlib.reload(mod) from the three fixtures that have it. Per-test reset is handled by clearing the mutable session dicts (_sessions, _pending, _answers). _methods is also no longer cleared — it's populated at module import time and would only be re-populated by a reload, so clearing it without reload broke session.resume / command.dispatch / slash.exec method registration across tests. Affected fixtures: - tests/tui_gateway/test_goal_command.py - tests/tui_gateway/test_protocol.py - tests/tui_gateway/test_review_summary_callback.py The second reload in test_protocol.py at line 211 (reload of tui_gateway.transport) is preserved — transport.py has no atexit hooks or threads, so reload is safe there. Tests: 84/84 in tests/tui_gateway/ pass cleanly with exit code 0; no Fatal Python error at interpreter shutdown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stops three tui_gateway test fixtures from reloading
tui_gateway.serverin teardown. The reloads were re-registering the module's atexit hooks on every test, accumulating duplicates across the session, and racing the stderr buffer at interpreter shutdown — which surfaces as a CI flake where "all tests pass" but the slice exits non-zero.The flake
Affected slices fail with:
Surfaced today on PR #34193 test slice 1 — same shape was hitting the docker integration tests earlier in the day too. The exit-code-non-zero turns the slice red even though every test reported PASS.
Root cause
tui_gateway/server.pyregisters two atexit hooks at module load time:Three test fixtures called
importlib.reload(mod)at teardown to reset per-test session state. Each reload re-runs the module-level code, including the atexit registrations. After N tests using the fixture, the atexit registry has N+1 copies of each hook. At pytest interpreter shutdown all N+1 try to fire in parallel and race for the stderr buffer flush.Fix
Drop
importlib.reload(mod)from the three fixtures. Per-test reset is handled by clearing the mutable session dicts (_sessions,_pending,_answers)._methodsis also no longer cleared — it's populated at module import time and would only repopulate via reload, so clearing it without the reload brokesession.resume/command.dispatch/slash.execmethod registration across tests.tests/tui_gateway/test_review_summary_callback.pytests/tui_gateway/test_goal_command.pytests/tui_gateway/test_protocol.pyThe second reload in
test_protocol.pyat line 211 (reload oftui_gateway.transportfor theHERMES_TUI_GATEWAY_NO_FLUSHenv-var test) is preserved —transport.pyhas no atexit hooks or threads, so reload is safe there.Validation
tests/tui_gateway/full directory: 84/84 pass, exit code 0, no Fatal Python error at interpreter shutdown.test_session_resume_returns_hydrated_messages,test_slash_exec_*,test_command_dispatch_*): all pass in order-shuffled session.Origin
This PR is a side-fix from triaging an unrelated CI failure on PR #34193 — the kanban checkpoint reminder feature. The atexit-duplicate issue is independent of any kanban work; it's a test-infrastructure bug that benefits every PR. Splitting it into its own PR so it lands cleanly regardless of whether #34193 ships.
Infographic