Skip to content

test: isolate gateway system service identity in unit tests#10

Closed
DIZ-admin wants to merge 5 commits into
mainfrom
fix/gateway-system-service-tests-clean
Closed

test: isolate gateway system service identity in unit tests#10
DIZ-admin wants to merge 5 commits into
mainfrom
fix/gateway-system-service-tests-clean

Conversation

@DIZ-admin

@DIZ-admin DIZ-admin commented May 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • stub system service identity in the affected gateway unit tests so root shells do not trip the runtime guard
  • keep the fix test-only and limited to the two systemd unit generation cases
  • avoid any production gateway behavior changes

Test Plan

  • pytest -q tests/hermes_cli/test_gateway_service.py

Summary by CodeRabbit

  • Chores

    • Updated Hermes Agent package to 0.14.0
    • Enhanced release changelog generation with additional author mapping
  • Tests

    • Improved system service unit tests with deterministic identity handling
    • Strengthened test isolation by unsetting custom base-URL environment variables in fixtures and a specific auxiliary client test

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 17, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Version 0.14.0 of the Hermes Agent package is pinned in the registry config. Release tooling recognizes hermes-agent commits. Tests are hardened: systemd unit generation tests use a deterministic system-service identity and env vars for custom base URLs are explicitly unset.

Changes

Hermes Agent Release Support

Layer / File(s) Summary
Version bump and release author mapping
acp_registry/agent.json, scripts/release.py
Agent version updated from 0.13.00.14.0 with matching distribution.uvx.package pin; AUTHOR_MAP gains "hermes-agent@users.noreply.github.com": "hermes-agent".
Test hermeticity and deterministic systemd units
tests/hermes_cli/test_gateway_service.py, tests/agent/test_auxiliary_client.py, tests/conftest.py
Two systemd unit generation tests monkeypatch gateway_cli._system_service_identity to a fixed user/home for deterministic PATH resolution; CUSTOM_BASE_URL and OPENROUTER_BASE_URL are explicitly removed in affected tests and the autouse hermetic fixture.

Possibly related PRs

  • DIZ-admin/hermes-agent#4: Both PRs monkeypatch _system_service_identity in test_gateway_service.py to make systemd unit generation tests deterministic.
  • DIZ-admin/hermes-agent#3: Both PRs update test hermeticization and deterministic identity handling for gateway unit tests and environment cleanup.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 I hopped the version line to fourteen,
Mapped the author so it's clearly seen,
Tests scrubbed clean, identity set right,
Deterministic units sleep through the night.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'test: isolate gateway system service identity in unit tests' accurately reflects the primary change in the pull request: the main focus is on monkeypatching gateway_cli._system_service_identity in gateway service tests for hermeticity. However, the PR also includes updates to hermes-agent package version, AUTHOR_MAP configuration, and environment variable isolation in multiple test files. The title captures the most significant code change but doesn't reflect the full scope of the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/gateway-system-service-tests-clean

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented May 17, 2026

Copy link
Copy Markdown

🔎 Lint report: fix/gateway-system-service-tests-clean vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 8331 on HEAD, 8331 on base (➖ 0)

🆕 New issues (45):

Rule Count
invalid-argument-type 35
unresolved-attribute 6
unsupported-operator 4
First entries
run_agent.py:13478: [invalid-argument-type] invalid-argument-type: Argument to function `normalize_usage` is incorrect: Expected `str | None`, found `(str & ~Literal["codex_app_server"]) | (Unknown & ~Literal["codex_app_server"]) | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:9531: [invalid-argument-type] invalid-argument-type: Argument to function `get_transport` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:3501: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:2622: [invalid-argument-type] invalid-argument-type: Argument to function `ensure_lmstudio_model_loaded` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:6177: [unresolved-attribute] unresolved-attribute: Attribute `split` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]]`, `int`, `dict[Unknown, Unknown]` in union `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
tests/run_agent/test_provider_attribution_headers.py:223: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache"]` and `Unknown | str | dict[str, str] | ... omitted 3 union elements`
run_agent.py:12003: [unresolved-attribute] unresolved-attribute: Attribute `strip` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy`, `int & ~AlwaysFalsy`, `dict[Unknown, Unknown] & ~AlwaysFalsy` in union `(str & ~AlwaysFalsy) | (Unknown & ~AlwaysFalsy) | (dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy) | ... omitted 3 union elements`
tests/agent/test_codex_cloudflare_headers.py:181: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["originator"]` and `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:14518: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:13523: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:6177: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["/"]` and `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:3501: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
cli.py:9213: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:9338: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:13525: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | dict[str, str]`
run_agent.py:9338: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str & ~AlwaysFalsy`, `int & ~AlwaysFalsy` in union `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:5009: [invalid-argument-type] invalid-argument-type: Argument to function `save_trajectory` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:9984: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_profile` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:14034: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown, Unknown] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 3 union elements`
tests/run_agent/test_provider_attribution_headers.py:158: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | dict[str, str]`
run_agent.py:2716: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:14298: [invalid-argument-type] invalid-argument-type: Argument to function `_pool_may_recover_from_rate_limit` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:10159: [invalid-argument-type] invalid-argument-type: Argument to function `github_model_reasoning_efforts` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
... and 20 more

✅ Fixed issues (49):

Rule Count
invalid-argument-type 38
unresolved-attribute 7
unsupported-operator 4
First entries
run_agent.py:6177: [unresolved-attribute] unresolved-attribute: Attribute `split` is not defined on `dict[Unknown, Unknown]`, `int`, `dict[Unknown | str, Unknown | str | dict[str, str]]` in union `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:9531: [invalid-argument-type] invalid-argument-type: Argument to function `get_transport` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str & ~AlwaysFalsy`, `int & ~AlwaysFalsy` in union `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:14298: [invalid-argument-type] invalid-argument-type: Argument to function `_pool_may_recover_from_rate_limit` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
tests/agent/test_codex_cloudflare_headers.py:181: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["originator"]` and `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:8182: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
tests/run_agent/test_provider_attribution_headers.py:158: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | Divergent | dict[str, str]`
tests/run_agent/test_provider_attribution_headers.py:224: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache-TTL"]` and `Unknown | str | dict[str, str] | ... omitted 4 union elements`
run_agent.py:12768: [invalid-argument-type] invalid-argument-type: Argument to function `apply_anthropic_cache_control` is incorrect: Expected `bool`, found `int | Divergent | Unknown | ... omitted 3 union elements`
run_agent.py:9338: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str`, found `Divergent | Unknown | str | ... omitted 3 union elements`
run_agent.py:13523: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:3501: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:7547: [invalid-argument-type] invalid-argument-type: Argument to function `_codex_cloudflare_headers` is incorrect: Expected `str`, found `Unknown | str | dict[str, str] | ... omitted 4 union elements`
run_agent.py:3501: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:7736: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown | str, Unknown | str | dict[str, str]] | Any | ... omitted 4 union elements`
run_agent.py:9967: [invalid-argument-type] invalid-argument-type: Argument to function `_get_anthropic_max_output` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:8182: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:5756: [invalid-argument-type] invalid-argument-type: Argument to function `parse_rate_limit_headers` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:2767: [invalid-argument-type] invalid-argument-type: Argument to function `get_model_context_length` is incorrect: Expected `str`, found `str | dict[str, str] | Any | ... omitted 4 union elements`
cli.py:9213: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:2622: [invalid-argument-type] invalid-argument-type: Argument to function `ensure_lmstudio_model_loaded` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:13478: [invalid-argument-type] invalid-argument-type: Argument to function `normalize_usage` is incorrect: Expected `str | None`, found `(str & ~Literal["codex_app_server"]) | (Unknown & ~Literal["codex_app_server"]) | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:13562: [invalid-argument-type] invalid-argument-type: Argument to bound method `SessionDB.update_token_counts` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:4355: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:14031: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown | str, Unknown | str | dict[str, str]] | Any | ... omitted 4 union elements`
... and 24 more

Unchanged: 4312 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/agent/test_auxiliary_client.py (1)

34-42: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Duplicate fixture definition: _clean_env is defined twice.

The _clean_env fixture is defined at both line 34 and line 2487. Python will use only the second definition (lines 2487-2494), silently ignoring the first. This is confusing for maintainers who might update the wrong definition.

🔧 Consolidate or remove the duplicate

If both sets of env vars need clearing, merge them:

 `@pytest.fixture`(autouse=True)
 def _clean_env(monkeypatch):
     """Strip provider env vars so each test starts clean."""
     for key in (
         "OPENROUTER_API_KEY", "OPENAI_BASE_URL", "OPENAI_API_KEY",
         "OPENAI_MODEL", "LLM_MODEL", "NOUS_INFERENCE_BASE_URL",
         "ANTHROPIC_API_KEY", "ANTHROPIC_TOKEN", "CLAUDE_CODE_OAUTH_TOKEN",
+        "NVIDIA_API_KEY", "NVIDIA_BASE_URL",
     ):
         monkeypatch.delenv(key, raising=False)

Then delete lines 2487-2494.

Also applies to: 2487-2494

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/agent/test_auxiliary_client.py` around lines 34 - 42, The _clean_env
fixture is defined twice which causes the first definition to be ignored;
consolidate the environment variable lists into a single _clean_env fixture and
remove the duplicate definition so tests run with a single, authoritative
fixture; locate both fixtures named _clean_env, merge any differing env var
tuples into one comprehensive tuple inside that fixture (keeping the
autouse=True behavior and monkeypatch.delenv calls), and delete the redundant
fixture definition.
🧹 Nitpick comments (2)
tests/conftest.py (1)

355-355: 💤 Low value

Consider adding CUSTOM_BASE_URL to _CREDENTIAL_NAMES for consistency.

The explicit cleanup here is correct, but for consistency with other base URLs (OPENROUTER_BASE_URL, OPENAI_BASE_URL, etc. at lines 142-150), you could add CUSTOM_BASE_URL to the _CREDENTIAL_NAMES frozenset instead. That way all base URLs are handled uniformly by the credential filter loop at lines 301-303.

♻️ Alternative: add to _CREDENTIAL_NAMES
     "BROWSER_USE_API_KEY",
     "CUSTOM_API_KEY",
+    "CUSTOM_BASE_URL",
     "GATEWAY_PROXY_URL",
     "GEMINI_BASE_URL",

Then remove line 355 (explicit cleanup no longer needed).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/conftest.py` at line 355, The test teardown explicitly calls
monkeypatch.delenv("CUSTOM_BASE_URL") but CUSTOM_BASE_URL is not included in the
_CREDENTIAL_NAMES frozenset, causing inconsistency; add "CUSTOM_BASE_URL" to the
_CREDENTIAL_NAMES declaration so the existing credential-filter loop (referenced
around the credential-cleanup logic on the test file) will automatically remove
it, and then remove the explicit monkeypatch.delenv("CUSTOM_BASE_URL") call at
the teardown location to avoid duplicate cleanup.
tests/agent/test_auxiliary_client.py (1)

666-667: 💤 Low value

Cleanup now redundant after conftest.py change.

After the addition of CUSTOM_BASE_URL cleanup in tests/conftest.py line 355, this explicit cleanup is redundant—the _hermetic_environment autouse fixture already clears it globally.

Similarly, OPENROUTER_BASE_URL is already in _CREDENTIAL_NAMES (conftest.py line 144), so it has always been cleared by the hermetic environment.

The redundancy is harmless and makes the test's intent more explicit, but you can simplify if desired:

♻️ Remove now-redundant cleanup
     def test_returns_none_when_nothing_available(self, monkeypatch):
         monkeypatch.delenv("OPENAI_BASE_URL", raising=False)
         monkeypatch.delenv("OPENAI_API_KEY", raising=False)
         monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
-        monkeypatch.delenv("CUSTOM_BASE_URL", raising=False)
-        monkeypatch.delenv("OPENROUTER_BASE_URL", raising=False)
         with patch("agent.auxiliary_client._read_nous_auth", return_value=None), \
              patch("agent.auxiliary_client._read_codex_access_token", return_value=None), \
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/agent/test_auxiliary_client.py` around lines 666 - 667, Remove the
now-redundant explicit environment cleanup lines that call
monkeypatch.delenv("CUSTOM_BASE_URL", raising=False) and
monkeypatch.delenv("OPENROUTER_BASE_URL", raising=False) from the test; these
are already handled by the autouse fixture _hermetic_environment which clears
variables listed in _CREDENTIAL_NAMES, so delete those two monkeypatch.delenv
calls in test_auxiliary_client.py to simplify the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@tests/agent/test_auxiliary_client.py`:
- Around line 34-42: The _clean_env fixture is defined twice which causes the
first definition to be ignored; consolidate the environment variable lists into
a single _clean_env fixture and remove the duplicate definition so tests run
with a single, authoritative fixture; locate both fixtures named _clean_env,
merge any differing env var tuples into one comprehensive tuple inside that
fixture (keeping the autouse=True behavior and monkeypatch.delenv calls), and
delete the redundant fixture definition.

---

Nitpick comments:
In `@tests/agent/test_auxiliary_client.py`:
- Around line 666-667: Remove the now-redundant explicit environment cleanup
lines that call monkeypatch.delenv("CUSTOM_BASE_URL", raising=False) and
monkeypatch.delenv("OPENROUTER_BASE_URL", raising=False) from the test; these
are already handled by the autouse fixture _hermetic_environment which clears
variables listed in _CREDENTIAL_NAMES, so delete those two monkeypatch.delenv
calls in test_auxiliary_client.py to simplify the test.

In `@tests/conftest.py`:
- Line 355: The test teardown explicitly calls
monkeypatch.delenv("CUSTOM_BASE_URL") but CUSTOM_BASE_URL is not included in the
_CREDENTIAL_NAMES frozenset, causing inconsistency; add "CUSTOM_BASE_URL" to the
_CREDENTIAL_NAMES declaration so the existing credential-filter loop (referenced
around the credential-cleanup logic on the test file) will automatically remove
it, and then remove the explicit monkeypatch.delenv("CUSTOM_BASE_URL") call at
the teardown location to avoid duplicate cleanup.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fc696d6f-0373-4951-a6f5-6adbbee1f58a

📥 Commits

Reviewing files that changed from the base of the PR and between 3b1d831 and a3df6ab.

📒 Files selected for processing (2)
  • tests/agent/test_auxiliary_client.py
  • tests/conftest.py

@DIZ-admin

Copy link
Copy Markdown
Owner Author

Superseded by #13. The two system-service test patches from this branch were carried forward there together with the remaining hermetic cleanup so the red cluster is fixed in one clean PR.

@DIZ-admin

Copy link
Copy Markdown
Owner Author

Closing as superseded by #13.

@DIZ-admin DIZ-admin closed this May 18, 2026
@DIZ-admin DIZ-admin deleted the fix/gateway-system-service-tests-clean branch May 18, 2026 10:41
DIZ-admin pushed a commit that referenced this pull request May 26, 2026
Four findings from Copilot's review on PR NousResearch#22891, all in the AX
elements-array cap added by 22fa1ed:

1. The truncation note ("response truncated to N of M elements") was
   appended unconditionally — including in the som/vision multimodal
   path, whose response carries a screenshot rather than an `elements`
   array. The note described a payload field that wasn't present.
   Moved the note into the AX-text branch where the array actually
   appears.

2. `_format_elements(cap.elements)` ran on the full untrimmed list with
   its own `max_lines=40` cap, so a caller passing `max_elements=10`
   would see summary lines referencing `#11..NousResearch#40` even though the JSON
   `elements` array only held #1..#10. Format on `visible_elements`
   instead so the summary indices always exist in the response.

3. `_coerce_max_elements` enforced a lower bound but no upper bound,
   so `max_elements=10_000_000` silently disabled the safeguard and
   reintroduced the original context-blow-up. Added a hard cap
   (`_MAX_ALLOWED_MAX_ELEMENTS = 1000`) that clamps oversized values.

4. The schema string said "Default 100" but the property carried no
   `default` field, and claimed `max_elements` had no effect on som/
   vision while the image-missing fallback path can still return an
   elements array. Added `"default": 100`, `"maximum": 1000`, and
   clarified the fallback-path wording.

Each finding gets a regression test:

- test_capture_ax_clamps_oversized_max_elements_to_hard_cap
- test_capture_ax_summary_indices_match_returned_elements
- test_capture_multimodal_summary_omits_truncation_note
- test_schema_max_elements_documents_default_and_upper_bound

Verified with `pytest tests/tools/test_computer_use.py` (53 passed,
including the 5 new cases). Confirmed each new test fails on the
pre-fix code path before applying the production change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants