Skip to content

fix(gateway): drain in-flight work before restart#7503

Merged
teknium1 merged 5 commits into
mainfrom
hermes/hermes-dc4200fd
Apr 11, 2026
Merged

fix(gateway): drain in-flight work before restart#7503
teknium1 merged 5 commits into
mainfrom
hermes/hermes-dc4200fd

Conversation

@teknium1

@teknium1 teknium1 commented Apr 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Salvage of PR #7290 by @bobashopcashier (Kenny Xie). Fixes #7286.

Adds graceful drain-before-restart to the gateway. Previously, /restart from Discord immediately interrupted all running agents mid-response. Now the gateway:

  1. Sets a drain flag and stops accepting new work
  2. Waits for in-flight agents to finish (configurable timeout, default 60s)
  3. Interrupts stragglers after timeout
  4. Cleans up tool resources and disconnects adapters
  5. Exits with code 75 for systemd/launchd to auto-restart

Key changes

  • /restart slash command — gateway-only command that triggers graceful drain
  • busy_input_mode: queue wired into gateway — was CLI-only; now queues messages during drain
  • SIGUSR1 signal handler — service managers can request graceful restart
  • agent.restart_drain_timeout — new config.yaml setting (default 60s, 0 = immediate interrupt)
  • systemd/launchd integration — ExecReload, RestartForceExitStatus=75, dynamic TimeoutStopSec
  • Extracted merge_pending_message_event() — shared photo-burst merge helper, reduces duplication
  • gateway/restart.py — shared constants and timeout parsing

Salvage fixes (on top of contributor commits)

  • Restored agent.close() call in _finalize_shutdown_agents() — the refactored stop() dropped per-agent tool resource cleanup (terminal sandboxes, browser daemons, httpx clients), which would cause zombie processes on every restart
  • Preserved global cleanup (process_registry, environments, browsers) in _stop_impl() during conflict resolution
  • Fixed /restart CommandDef category — moved from "Info" to "Session" to match /stop and /status

Test results

  • 88 targeted tests pass (restart drain, shutdown, session race guard, session boundary hooks, gateway service)
  • 11 E2E tests pass (config loading, drain flow, idempotency, cleanup verification)
  • All 20 gateway failures are pre-existing on main (approval buttons, telegram conflict, feishu, signal)

Closes #7286. Credit to @bobashopcashier for the design and implementation.

bobashopcashier and others added 5 commits April 10, 2026 18:54
@bobashopcashier @teknium1
fix(gateway): drain in-flight work before restart
ef120b5
@bobashopcashier @teknium1
fix(gateway): tolerate partial runner construction
b592853
@bobashopcashier @teknium1
fix(gateway): self-request service restarts when invoked in-process
b4cb803
@bobashopcashier @teknium1
fix(gateway): address restart review feedback
eb3f021
@teknium1
fix: restore agent.close() cleanup and correct /restart category
c4c3a57 
- Add agent.close() call to _finalize_shutdown_agents() to prevent
  zombie processes (terminal sandboxes, browser daemons, httpx clients)
- Global cleanup (process_registry, environments, browsers) preserved
  in _stop_impl() during conflict resolution
- Move /restart CommandDef from 'Info' to 'Session' category to match
  /stop and /status
@teknium1 teknium1 merged commit 9605195 into main Apr 11, 2026
3 of 6 checks passed
@teknium1 teknium1 mentioned this pull request Apr 11, 2026
Closed
18 tasks
@philmossman philmossman mentioned this pull request Apr 12, 2026
Closed
teknium1 added a commit that referenced this pull request Apr 13, 2026
@teknium1
fix: add contributor audit script + fix missed contributors
4180b1f 
Three problems fixed:

1. bobashopcashier missing from v0.9.0 contributor list despite
   authoring the gateway drain PR (#7290, salvaged into #7503).
   Their email (kennyx102@gmail.com) was missing from AUTHOR_MAP.

2. release.py only scanned git commit authors, missing Co-authored-by
   trailers. Now parse_coauthors() extracts trailers from commit bodies.

3. No mechanism to detect contributors from salvaged PRs (where original
   author only appears in PR description, not git log).

Changes:
- scripts/release.py: add kennyx102@gmail.com to AUTHOR_MAP, enhance
  get_commits() to parse Co-authored-by trailers, filter AI assistants
  (Claude, Copilot, Cursor Agent) from co-author lists
- scripts/contributor_audit.py: new script that cross-references git
  authors, co-author trailers, and salvaged PR descriptions. Reports
  unknown emails and contributors missing from release notes.
- RELEASE_v0.9.0.md: add bobashopcashier to community contributors

Usage:
  python scripts/contributor_audit.py --since-tag v2026.4.8
  python scripts/contributor_audit.py --since-tag v2026.4.8 --release-file RELEASE_v0.9.0.md
@teknium1 teknium1 mentioned this pull request Apr 13, 2026
Merged
teknium1 added a commit that referenced this pull request Apr 13, 2026
@teknium1
fix: add contributor audit script + fix missed contributors (#9264)
def8b95 
Three problems fixed:

1. bobashopcashier missing from v0.9.0 contributor list despite
   authoring the gateway drain PR (#7290, salvaged into #7503).
   Their email (kennyx102@gmail.com) was missing from AUTHOR_MAP.

2. release.py only scanned git commit authors, missing Co-authored-by
   trailers. Now parse_coauthors() extracts trailers from commit bodies.

3. No mechanism to detect contributors from salvaged PRs (where original
   author only appears in PR description, not git log).

Changes:
- scripts/release.py: add kennyx102@gmail.com to AUTHOR_MAP, enhance
  get_commits() to parse Co-authored-by trailers, filter AI assistants
  (Claude, Copilot, Cursor Agent) from co-author lists
- scripts/contributor_audit.py: new script that cross-references git
  authors, co-author trailers, and salvaged PR descriptions. Reports
  unknown emails and contributors missing from release notes.
- RELEASE_v0.9.0.md: add bobashopcashier to community contributors

Usage:
  python scripts/contributor_audit.py --since-tag v2026.4.8
  python scripts/contributor_audit.py --since-tag v2026.4.8 --release-file RELEASE_v0.9.0.md
@github-actions github-actions Bot mentioned this pull request Apr 15, 2026
Merged
ulasbilgen pushed a commit to ulasbilgen/hermes-adhd-agent that referenced this pull request May 1, 2026
@teknium1
fix: add contributor audit script + fix missed contributors (NousRese…
6c54770 
…arch#9264)

Three problems fixed:

1. bobashopcashier missing from v0.9.0 contributor list despite
   authoring the gateway drain PR (NousResearch#7290, salvaged into NousResearch#7503).
   Their email (kennyx102@gmail.com) was missing from AUTHOR_MAP.

2. release.py only scanned git commit authors, missing Co-authored-by
   trailers. Now parse_coauthors() extracts trailers from commit bodies.

3. No mechanism to detect contributors from salvaged PRs (where original
   author only appears in PR description, not git log).

Changes:
- scripts/release.py: add kennyx102@gmail.com to AUTHOR_MAP, enhance
  get_commits() to parse Co-authored-by trailers, filter AI assistants
  (Claude, Copilot, Cursor Agent) from co-author lists
- scripts/contributor_audit.py: new script that cross-references git
  authors, co-author trailers, and salvaged PR descriptions. Reports
  unknown emails and contributors missing from release notes.
- RELEASE_v0.9.0.md: add bobashopcashier to community contributors

Usage:
  python scripts/contributor_audit.py --since-tag v2026.4.8
  python scripts/contributor_audit.py --since-tag v2026.4.8 --release-file RELEASE_v0.9.0.md
aj-nt pushed a commit to aj-nt/hermes-agent that referenced this pull request May 1, 2026
@teknium1
fix: add contributor audit script + fix missed contributors (NousRese…
6f5449d 
…arch#9264)

Three problems fixed:

1. bobashopcashier missing from v0.9.0 contributor list despite
   authoring the gateway drain PR (NousResearch#7290, salvaged into NousResearch#7503).
   Their email (kennyx102@gmail.com) was missing from AUTHOR_MAP.

2. release.py only scanned git commit authors, missing Co-authored-by
   trailers. Now parse_coauthors() extracts trailers from commit bodies.

3. No mechanism to detect contributors from salvaged PRs (where original
   author only appears in PR description, not git log).

Changes:
- scripts/release.py: add kennyx102@gmail.com to AUTHOR_MAP, enhance
  get_commits() to parse Co-authored-by trailers, filter AI assistants
  (Claude, Copilot, Cursor Agent) from co-author lists
- scripts/contributor_audit.py: new script that cross-references git
  authors, co-author trailers, and salvaged PR descriptions. Reports
  unknown emails and contributors missing from release notes.
- RELEASE_v0.9.0.md: add bobashopcashier to community contributors

Usage:
  python scripts/contributor_audit.py --since-tag v2026.4.8
  python scripts/contributor_audit.py --since-tag v2026.4.8 --release-file RELEASE_v0.9.0.md
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
@teknium1
fix: add contributor audit script + fix missed contributors (NousRese…
9dfd565 
…arch#9264)

Three problems fixed:

1. bobashopcashier missing from v0.9.0 contributor list despite
   authoring the gateway drain PR (NousResearch#7290, salvaged into NousResearch#7503).
   Their email (kennyx102@gmail.com) was missing from AUTHOR_MAP.

2. release.py only scanned git commit authors, missing Co-authored-by
   trailers. Now parse_coauthors() extracts trailers from commit bodies.

3. No mechanism to detect contributors from salvaged PRs (where original
   author only appears in PR description, not git log).

Changes:
- scripts/release.py: add kennyx102@gmail.com to AUTHOR_MAP, enhance
  get_commits() to parse Co-authored-by trailers, filter AI assistants
  (Claude, Copilot, Cursor Agent) from co-author lists
- scripts/contributor_audit.py: new script that cross-references git
  authors, co-author trailers, and salvaged PR descriptions. Reports
  unknown emails and contributors missing from release notes.
- RELEASE_v0.9.0.md: add bobashopcashier to community contributors

Usage:
  python scripts/contributor_audit.py --since-tag v2026.4.8
  python scripts/contributor_audit.py --since-tag v2026.4.8 --release-file RELEASE_v0.9.0.md
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
@teknium1
fix: add contributor audit script + fix missed contributors (NousRese…
ab807d7 
…arch#9264)

Three problems fixed:

1. bobashopcashier missing from v0.9.0 contributor list despite
   authoring the gateway drain PR (NousResearch#7290, salvaged into NousResearch#7503).
   Their email (kennyx102@gmail.com) was missing from AUTHOR_MAP.

2. release.py only scanned git commit authors, missing Co-authored-by
   trailers. Now parse_coauthors() extracts trailers from commit bodies.

3. No mechanism to detect contributors from salvaged PRs (where original
   author only appears in PR description, not git log).

Changes:
- scripts/release.py: add kennyx102@gmail.com to AUTHOR_MAP, enhance
  get_commits() to parse Co-authored-by trailers, filter AI assistants
  (Claude, Copilot, Cursor Agent) from co-author lists
- scripts/contributor_audit.py: new script that cross-references git
  authors, co-author trailers, and salvaged PR descriptions. Reports
  unknown emails and contributors missing from release notes.
- RELEASE_v0.9.0.md: add bobashopcashier to community contributors

Usage:
  python scripts/contributor_audit.py --since-tag v2026.4.8
  python scripts/contributor_audit.py --since-tag v2026.4.8 --release-file RELEASE_v0.9.0.md
@lost9999 lost9999 mentioned this pull request Jun 3, 2026
Open
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
@teknium1
fix: add contributor audit script + fix missed contributors (NousRese…
4513e07 
…arch#9264)

Three problems fixed:

1. bobashopcashier missing from v0.9.0 contributor list despite
   authoring the gateway drain PR (NousResearch#7290, salvaged into NousResearch#7503).
   Their email (kennyx102@gmail.com) was missing from AUTHOR_MAP.

2. release.py only scanned git commit authors, missing Co-authored-by
   trailers. Now parse_coauthors() extracts trailers from commit bodies.

3. No mechanism to detect contributors from salvaged PRs (where original
   author only appears in PR description, not git log).

Changes:
- scripts/release.py: add kennyx102@gmail.com to AUTHOR_MAP, enhance
  get_commits() to parse Co-authored-by trailers, filter AI assistants
  (Claude, Copilot, Cursor Agent) from co-author lists
- scripts/contributor_audit.py: new script that cross-references git
  authors, co-author trailers, and salvaged PR descriptions. Reports
  unknown emails and contributors missing from release notes.
- RELEASE_v0.9.0.md: add bobashopcashier to community contributors

Usage:
  python scripts/contributor_audit.py --since-tag v2026.4.8
  python scripts/contributor_audit.py --since-tag v2026.4.8 --release-file RELEASE_v0.9.0.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

bug(gateway): busy_input_mode=queue is not honored during Discord self-restart

2 participants

@teknium1 @bobashopcashier