What does this PR do?

Fixes the multi-process log-rotation split reported in #27649 without adding a new runtime dependency.

When several Hermes processes share ~/.hermes/logs/agent.log (e.g. the gateway plus several hermes --tui sessions and their tui_gateway.entry / slash_worker subprocesses) the stdlib RotatingFileHandler keeps an open file descriptor pointing at the active inode. If another process rotates the file -- renaming agent.log -> agent.log.1 and creating a fresh agent.log -- the original fd still resolves to the renamed inode, so:

1. Writes from the older processes silently land in agent.log.1 forever.
2. Newer processes write to the new agent.log.
3. The live log stream gets split across files; tools that follow only agent.log miss current activity.
4. Worse, once the older process eventually triggers its own rollover, the rename overwrites the newer agent.log that a sibling just created.

hermes_logging._ManagedRotatingFileHandler now detects external rotation by comparing the open stream's (st_dev, st_ino) against baseFilename on disk, and reopens the active file when they disagree. The fix is stdlib-only -- no concurrent-log-handler dependency.

Related Issue

Fixes #27649.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Security fix
• Documentation update
• Tests (adding or improving test coverage)
• Refactor (no behavior change)
• New skill (bundled or hub)

Changes Made

hermes_logging.py -- _ManagedRotatingFileHandler is now inode-aware:

• New class attribute _REOPEN_CHECK_INTERVAL = 16 controls how often the inode check runs (the first emit always checks, then every Nth thereafter).
• New _stream_inode_key() / _baseFilename_inode_key() helpers return (st_dev, st_ino) and treat st_ino == 0 (Windows + Python combos that don't expose stable inode numbers) and OSError as "unknown" rather than triggering spurious reopens.
• New _reopen_if_rotated_externally() closes the stale stream and reopens baseFilename when the inode-comparison mismatches; returns True if a reopen happened.
• emit() invokes the check before delegating to the base class; failures in the check path are swallowed so a broken logging path never kills the calling code (fail-open).
• doRollover() short-circuits when an external rotation already happened, reopening baseFilename instead of renaming it -- preventing the older process from clobbering the newer agent.log a sibling just created.
• Managed-mode chmod 0o660 is re-applied on the reopen path so sibling-rotated files don't get stuck at the process umask (0644).

tests/test_hermes_logging_multiprocess_rotation.py (new file, 15 regression tests):

• TestExternalRotationReopen (4 tests) -- direct [Bug]: Multiprocess logging can keep writing to rotated agent.log.N files #27649 repro plus steady-state guards.
• TestDoRolloverShortCircuit (2 tests) -- the rollover-clobber regression.
• TestThrottleAndCorners (7 tests) -- first-emit-always-checks, throttle-bounded-window, closed-stream safety, st_ino == 0 Windows compat, missing-baseFilename handling, OSError swallowing during close, fail-open of the inode-check path.
• TestManagedModeChmodAfterReopen (1 test, POSIX-only) -- chmod survives the reopen.
• TestIntegrationWithSetupLogging (1 test) -- setup_logging() still wires the managed handler.

How to Test

1. Check out the branch and set up the venv:

   python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[all,dev]"
   

2. Run the new regression suite alone:

   scripts/run_tests.sh tests/test_hermes_logging_multiprocess_rotation.py -v
   

   Expected: 15 passed.

3. Run the full logging-related suite to confirm no cross-file regression:

   scripts/run_tests.sh tests/test_hermes_logging.py tests/test_hermes_logging_multiprocess_rotation.py
   

   Expected: 65 passed (50 pre-existing + 15 new).

4. (Optional) Reproduce the [Bug]: Multiprocess logging can keep writing to rotated agent.log.N files #27649 scenario manually:

   python -c "
   import logging, os
   from hermes_logging import _ManagedRotatingFileHandler
   
   h = _ManagedRotatingFileHandler('/tmp/x.log', maxBytes=1024, backupCount=3)
   h._REOPEN_CHECK_INTERVAL = 1
   logging.getLogger('demo').addHandler(h); logging.getLogger('demo').setLevel(logging.INFO)
   
   logging.getLogger('demo').info('pre-rotation')
   os.rename('/tmp/x.log', '/tmp/x.log.1')
   open('/tmp/x.log','w').close()
   logging.getLogger('demo').info('post-rotation')
   print('live:', open('/tmp/x.log').read())
   print('backup:', open('/tmp/x.log.1').read())
   "
   

   Expected: post-rotation shows up in /tmp/x.log, not in /tmp/x.log.1.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (2 commits: fix(logging), test(logging))
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix (no unrelated commits)
• I've run scripts/run_tests.sh tests/test_hermes_logging.py tests/test_hermes_logging_multiprocess_rotation.py and all tests pass
• I've added tests for my changes (15 new regression tests)
• I've tested on my platform: macOS 15.2 (Darwin 24.6.0), Python 3.12

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) -- the class-level docstring on _ManagedRotatingFileHandler now documents the external-rotation safety contract and the throttle interval. No user-facing docs are affected (the handler is internal).
• I've updated cli-config.yaml.example if I added/changed config keys -- N/A (no config-key changes).
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows -- N/A.
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide -- the inode helpers treat st_ino == 0 and OSError as "unknown" (Windows-friendly fallback), and the managed-mode chmod test is gated by @pytest.mark.skipif(sys.platform.startswith("win")).
• I've updated tool descriptions/schemas if I changed tool behavior -- N/A.

Screenshots / Logs

$ scripts/run_tests.sh tests/test_hermes_logging_multiprocess_rotation.py -v
4 workers [15 items]
...............                                                          [100%]
============================== 15 passed in 1.17s ==============================

$ scripts/run_tests.sh tests/test_hermes_logging.py tests/test_hermes_logging_multiprocess_rotation.py
4 workers [65 items]
.................................................................        [100%]
============================== 65 passed in 1.18s ==============================

Limitations (per the issue's "no strong prescription from me" note)

This is a minimal stdlib-only fix:

• The inode check is throttled to every 16 records by default. Up to 15 records emitted between checks immediately after an external rotation may still land in the rotated backup. Tests pin this as a bounded contract rather than promising zero loss.
• True multi-process rotation safety (file locking around the rename) is out of scope -- two processes rotating simultaneously can still race. The short-circuit in doRollover prevents the worst symptom (a slow process overwriting a fresh agent.log written by a faster sibling) but doesn't synchronize the rename itself.
• No new dependency is added (concurrent-log-handler was considered and rejected to keep the install footprint minimal).