Skip to content

fix(docker): tee supervised gateway stdout to docker logs#33621

Merged
benbarclay merged 1 commit into
mainfrom
gateway-logs-to-docker-logs
May 28, 2026
Merged

fix(docker): tee supervised gateway stdout to docker logs#33621
benbarclay merged 1 commit into
mainfrom
gateway-logs-to-docker-logs

Conversation

@benbarclay

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #33583 (the gateway run auto-supervise redirect).

Before this fix, the supervised gateway's stdout was swallowed by s6-log into the rotated file at ${HERMES_HOME}/logs/gateways/<profile>/current and never reached docker logs. Operational signal lived in two places, depending on which fd a given log line went to:

Stream Where it went today Result
Python logging (stderr) docker logs warnings/errors visible
Rich-console banners, print(), raw stdout writes rotated file only ✗ hidden from docker logs

Users running docker run … gateway run against the s6 image would see partial output in docker logs (warnings yes, the iconic ⚕ Hermes Gateway Starting banner no), conclude something was broken, and have to docker exec + tail the file to find the missing pieces.

Fix

Add the 1 s6-log action directive before the file destination so each line is forwarded to s6-log's stdout — which propagates up through the s6-supervise pipeline to /init's stdout = container stdout = docker logs. The file destination is preserved as a second destination, so the rotated log (with ISO 8601 timestamps) still exists for hermes logs and for survival across container restarts.

- exec s6-setuidgid hermes s6-log    n10 s1000000 T "$log_dir"
+ exec s6-setuidgid hermes s6-log  1 n10 s1000000 T "$log_dir"

Trade-off: timestamps

T is non-sticky in s6-log — it only prefixes lines for the next action directive. Putting T between 1 and the file (not before 1) means:

  • docker logs shows raw lines. Python's logging formatter already has its own timestamps, and docker logs --timestamps adds another layer when desired. No double-stamping in the common reading path.
  • The rotated file gets s6-log's ISO 8601 timestamp so even output that lacked a Python-logger timestamp (rich banners, third-party raw prints) is correlatable in current.

Verification

Unit test

New assertion in test_service_manager.py locks the s6-log 1 directive into the rendered run-script. Mutation-tested: reverted the source to the pre-fix script (no 1); the assert caught it cleanly with a clear failure message naming the missing directive.

Docker harness test

New test_supervised_gateway_stdout_reaches_docker_logs (in tests/docker/test_gateway_run_supervised.py):

  • Builds the image
  • Runs docker run -d <image> gateway run
  • Asserts the unique banner glyph (or "Hermes Gateway Starting" text) reaches docker logs
  • Cross-asserts the same banner is also in the rotated log file (so we haven't lost the file destination)

End-to-end mutation-tested: built a deliberately-broken image without the 1 directive; the test failed exactly as designed, with the diagnostic showing the banner present in current but absent from docker logs.

Regression sweep

  • tests/hermes_cli/ — 5,561 tests pass, 0 fail
  • tests/docker/ — 27 tests pass (5 existing in test_gateway_run_supervised.py + the new one, plus 21 in the other 7 docker test files), 0 fail

Docs

website/docs/user-guide/docker.md gains a new :::note Where gateway logs go admonition documenting both destinations explicitly (the supervised stdout → docker logs, the rotated file → hermes logs reads it, persisted via volume mount) plus the audit log at ${HERMES_HOME}/logs/container-boot.log. The existing :::tip Gateway runs supervised admonition's claim that "docker logs shows the supervised gateway's output" — which was previously aspirational — is now actually correct.

Files

File Change
hermes_cli/service_manager.py One-character fix on the s6-log invocation (1 insertion) + expanded docstring explaining the routing
tests/hermes_cli/test_service_manager.py New assertion locking the 1 directive
tests/docker/test_gateway_run_supervised.py New harness test verifying end-to-end stdout flow
website/docs/user-guide/docker.md New :::note Where gateway logs go admonition

Follow-up to #33583 (the gateway-run-supervised redirect).

Before this fix, the supervised gateway's stdout (most visibly the
"Hermes Gateway Starting…" rich-console banner) was swallowed by
`s6-log` into the rotated file at
`${HERMES_HOME}/logs/gateways/<profile>/current` and never reached
`docker logs`. Operational signal lived in two places:

  * **docker logs** — saw stderr (Python `logging` defaults to
    stderr), so warnings/errors were visible.
  * **the rotated file** — saw stdout (rich banners, `print()`
    output, third-party libs that wrote to fd 1).

This was surprising for users coming from the pre-s6 image, where
`docker run … gateway run` produced a single unified stream in
`docker logs`. They'd see partial output, conclude something was
broken, and dig around for the missing pieces.

Fix: add the `1` s6-log action directive before the file destination
so each line is forwarded to s6-log's stdout — which propagates up
the s6-supervise pipeline to /init's stdout = container stdout =
`docker logs`. The file destination is preserved as a second
destination, so the rotated log (with ISO 8601 timestamps) still
exists for `hermes logs` and for survival across container restarts.

Trade-off considered: timestamps. Putting `T` between `1` and the
file destination (not before `1`) means:

  * docker logs sees raw lines — Python's logging formatter has its
    own timestamps, and `docker logs --timestamps` adds another
    layer when desired. No double-stamping in the common reading
    path.
  * The persisted file gets s6-log's ISO 8601 timestamp so even
    output that lacked a Python-logger timestamp (rich banners,
    third-party raw prints) is correlatable in `current`.

Verification:

  * New unit-test assertion in `test_service_manager.py` locks the
    `s6-log 1` directive into the rendered run-script. Mutation-
    tested by reverting to the pre-fix script (no `1`); the assert
    catches it cleanly.
  * New docker-harness test `test_supervised_gateway_stdout_reaches_docker_logs`
    builds the image, runs `docker run … gateway run`, and asserts
    the unique `⚕` banner glyph reaches `docker logs`. Also verifies
    the rotated file still contains the banner (no regression on
    the existing file destination). Mutation-tested end-to-end: built
    a deliberately-broken image without the `1` directive and the
    test failed exactly as designed, citing the banner present in
    `current` but absent from `docker logs`.
  * `website/docs/user-guide/docker.md` gains a new `:::note Where
    gateway logs go` admonition documenting both destinations and
    the audit-log file at `${HERMES_HOME}/logs/container-boot.log`.

Existing functionality preserved: every other docker-harness test
still passes against the new image. Unit-test sweep across
`tests/hermes_cli/` (5561 tests) is green.
@github-actions

Copy link
Copy Markdown
Contributor

🔎 Lint report: gateway-logs-to-docker-logs vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 9519 on HEAD, 9519 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 5015 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery area/docker Docker image, Compose, packaging P2 Medium — degraded but workaround exists labels May 28, 2026
@benbarclay benbarclay merged commit b345323 into main May 28, 2026
26 checks passed
@benbarclay benbarclay deleted the gateway-logs-to-docker-logs branch May 28, 2026 03:18
r266-tech added a commit to r266-tech/hermes-agent that referenced this pull request May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docker Docker image, Compose, packaging comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants