test(conftest): isolate CODEX_HOME so token-refresh writeback doesn't leak between tests#10927
Closed
luigileap wants to merge 1 commit into
Closed
test(conftest): isolate CODEX_HOME so token-refresh writeback doesn't leak between tests#10927luigileap wants to merge 1 commit into
luigileap wants to merge 1 commit into
Conversation
…into Path.home()
Production ``_refresh_codex_auth_tokens`` writes refreshed OAuth tokens
back to ``$CODEX_HOME/auth.json`` (defaulting to ``Path.home() /
".codex" / "auth.json"``) so the Codex CLI / VS Code extension stay in
sync. When a test triggers a refresh without explicitly setting
``CODEX_HOME``, the writeback lands in the test process's *real* HOME.
Under the project's standard ``HOME=$(mktemp -d)`` test wrapper that
HOME directory is shared by every test in the run, so the file persists
for the remainder of the session. Any later test that calls
``load_pool("openai-codex")`` then re-imports those stale tokens via
``_seed_from_singletons`` -> ``_import_codex_cli_tokens``, and ends up
with an extra phantom credential in the pool.
Concretely this caused four tests to fail intermittently under xdist
when ``test_try_refresh_current_updates_only_current_entry`` ran on the
same worker before any of:
- tests/hermes_cli/test_auth_commands.py::test_auth_remove_accepts_label_target
- tests/hermes_cli/test_auth_commands.py::test_auth_remove_prefers_exact_numeric_label_over_index
- tests/hermes_cli/test_opencode_go_in_model_list.py::test_opencode_go_appears_when_api_key_set
- tests/hermes_cli/test_overlay_slug_resolution.py::test_kimi_for_coding_overlay_uses_hermes_slug
Pin ``CODEX_HOME`` to a per-test scratch dir from the autouse
``_isolate_hermes_home`` fixture so writebacks are always contained to
``tmp_path`` and cleaned up between tests. Mirrors the existing
HERMES_HOME treatment. Add a regression test that exercises
``pool.try_refresh_current()`` for ``openai-codex`` and asserts
``Path.home() / ".codex" / "auth.json"`` is not created.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
Thanks for the submission @luigileap. Closing as superseded — the token-refresh writeback that caused cross-test leakage was removed by #12360; CODEX_HOME is no longer touched during refresh. Hermes's Codex auth design was reworked in #12360 ("Hermes owns its own Codex auth; stop touching The valid adjacent fixes from this batch (error parsing, fallback chain on auth failure, reauth UX) landed together in #15104. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Four tests fail intermittently under
pytest-xdistwhenever theopenai-codextoken-refresh test happens to run on the same workerbefore them:
Each one observes an extra phantom
openai-codexcredential it neverseeded. For example:
Root cause
Production
_refresh_codex_auth_tokenswrites refreshed OAuth tokensback to
$CODEX_HOME/auth.json(defaulting toPath.home() / ".codex" / "auth.json") so the Codex CLI / VS Code extension stay insync with Hermes:
tests/agent/test_credential_pool.py::test_try_refresh_current_updates_only_current_entrymocks the inner
refresh_codex_oauth_purebut not the outer_refresh_codex_auth_tokens, so the writeback runs for real. The testonly sets
HERMES_HOME, neverCODEX_HOME, so_write_codex_cli_tokensfalls back toPath.home() / ".codex" / "auth.json".Under the project's standard
HOME=$(mktemp -d)test wrapper that HOMEdirectory is shared by every test in the run. The leaked
auth.jsontherefore persists for the rest of the session, and anylater test that calls
load_pool("openai-codex")re-imports thosestale tokens via
_seed_from_singletons→_import_codex_cli_tokens, ending up with the phantom credential.Fix
Pin
CODEX_HOMEto a per-test scratch dir from the autouse_isolate_hermes_homefixture, mirroring the existingHERMES_HOMEtreatment:
The writeback now lands inside
tmp_path, which pytest cleans upbetween tests, so no later test ever sees the file. Add a regression
test in
tests/agent/test_credential_pool.pythat exercisespool.try_refresh_current()foropenai-codexand assertsPath.home() / ".codex" / "auth.json"is not created.Tests that need a specific
CODEX_HOMEcontinue to override it viamonkeypatch.setenv(...)as before.Verification
(All four originally failing tests pass under both serial and xdist
runs; the new regression test passes; nothing in the rest of the
suite regresses.)