Skip to content

feat(gateway): wire clarify tool with inline keyboard buttons on Telegram#24199

Merged
teknium1 merged 1 commit into
mainfrom
feat/clarify-gateway-buttons
May 12, 2026
Merged

feat(gateway): wire clarify tool with inline keyboard buttons on Telegram#24199
teknium1 merged 1 commit into
mainfrom
feat/clarify-gateway-buttons

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Wires clarify end-to-end on the gateway with inline keyboard buttons on Telegram. Closes #24191.

The clarify tool returned "not available in this execution context" for every gateway-mode agent because gateway/run.py never passed clarify_callback into the AIAgent constructor. The schema actively encouraged calling it; users never saw the question.

Changes

  • tools/clarify_gateway.py (new) — event-based primitive mirroring tools/approval.py. Per-session FIFO + threading.Event with 1s heartbeat slices so the inactivity watchdog keeps ticking. clear_session for boundary cleanup.
  • gateway/platforms/base.py — abstract send_clarify with a numbered-text fallback so every adapter (Discord, Slack, WhatsApp, Signal, Matrix, etc.) gets a working clarify. Plus an active-session guard bypass for non-command messages when there's a pending text-awaiting clarify, so the user's typed answer reaches the resolver instead of being queued and triggering an interrupt. Same shape as the /approve deadlock fix from PR fix(gateway): bypass active-session guard for /approve and /deny commands #4926.
  • gateway/platforms/telegram.py — concrete send_clarify renders one button per choice plus ✏️ Other (type answer). cl: callback handler resolves numeric choices immediately, flips to text-capture mode for Other, with the same authorization guards as exec/slash approvals.
  • gateway/run.pyclarify_callback wired at the cached-agent per-turn callback assignment site (only the user-facing agent path; cron and hygiene-compress agents have no human attached, so wiring clarify there would block forever). Bridges sync→async via run_coroutine_threadsafe, blocks with the configured timeout, returns [user did not respond within Xm] sentinel on timeout. Text-intercept added in _handle_message (skips slash commands). clear_session in the run's finally cancels orphan entries.
  • hermes_cli/config.pyagent.clarify_timeout default 600.
  • website/docs/user-guide/messaging/telegram.md — Interactive Prompts section.

Validation

Before After
Telegram clarify (multi-choice) "not available in this execution context" Buttons render, button click resolves agent thread
Telegram clarify (open-ended) "not available in this execution context" Question renders, next message resolves
Discord/Slack/Matrix/etc clarify "not available in this execution context" Numbered text fallback, next message resolves
Agent thread on user no-show (n/a — never delivered) Unblocks at agent.clarify_timeout (default 10m) with sentinel string
Session boundary (interrupt, /new, shutdown) (n/a) clear_session cancels orphan entries, blocked threads unwind

Tests

  • tests/tools/test_clarify_gateway.py — 14 tests, full primitive coverage (button resolve, open-ended auto-await, Other flip, timeout None, unknown-id idempotency, clear_session cancellation, FIFO ordering, notify register/unregister, config default).
  • tests/gateway/test_telegram_clarify_buttons.py — 12 tests, render paths (multi-choice / open-ended / long-label / HTML-escape / not-connected) + callback dispatch (numeric / Other flip / already-resolved / unauthorized / invalid-token) + base-adapter text fallback.

Targeted: 26/26 new, 40/40 adjacent approval tests pass. Full tests/gateway/: 5371 passed (+14 new), 7 skipped, 7 pre-existing failures unrelated to this change (verified via stash + rerun on origin/main).

Out of scope

Tracked separately for now: bot-to-bot (Bot API 10.0, needs PTB ≥23), guest mode, checklists (Premium-business-account-only, narrow audience), poll media, live photos, document scanner, member tags, login-with-Telegram, business accounts, suggested posts, gifts, blockchain gifts, Stars billing.

Design notes

  • "Other" path = Option A (issue-resolved): always include an "Other (type answer)" button when choices are provided, mirroring CLI behavior and matching the documented schema contract. Open-ended clarify (no choices) skips buttons entirely and captures the next message directly.
  • Wired only at the cached-agent path in gateway/run.py. The hygiene-compress, cron worker, and /compress tmp_agent AIAgent sites are background tasks with no user attached — wiring clarify there would block forever. The original issue text overstated this as "wire at all 4 AIAgent sites"; only one is correct.
  • agent.clarify_timeout, not gateway.clarify_timeout — kept under the existing agent.* block alongside gateway_timeout (inactivity), gateway_timeout_warning, restart_drain_timeout, etc., for consistency.

@github-actions

github-actions Bot commented May 12, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: feat/clarify-gateway-buttons vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 8253 on HEAD, 8241 on base (🆕 +12)

🆕 New issues (5):

Rule Count
unresolved-import 2
unused-type-ignore-comment 1
invalid-method-override 1
invalid-assignment 1
First entries
gateway/platforms/telegram.py:2841: [unused-type-ignore-comment] unused-type-ignore-comment: Unused blanket `type: ignore` directive
tests/gateway/test_telegram_clarify_buttons.py:14: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/gateway/test_telegram_clarify_buttons.py:434: [invalid-method-override] invalid-method-override: Invalid override of method `send`: Definition is incompatible with `BasePlatformAdapter.send`
tools/clarify_gateway.py:121: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to `def touch_activity_if_due(state: dict[Unknown, Unknown], label: str) -> None`
tests/tools/test_clarify_gateway.py:15: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`

✅ Fixed issues: none

Unchanged: 4333 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/feature New feature or request P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter comp/agent Core agent loop, run_agent.py, prompt builder area/config Config system, migrations, profiles labels May 12, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Supersedes #22532 and #21517 — same clarify-on-Telegram feature with a more comprehensive implementation. Closes #24191 and #18301.

…gram

The clarify tool returned 'not available in this execution context' for
every gateway-mode agent because gateway/run.py never passed
clarify_callback into the AIAgent constructor. Schema actively encouraged
calling it; users never saw the question.

Changes:

- tools/clarify_gateway.py — new event-based primitive mirroring
  tools/approval.py: register/wait_for_response/resolve_gateway_clarify
  with per-session FIFO, threading.Event blocking with 1s heartbeat
  slices (so the inactivity watchdog keeps ticking), and
  clear_session for boundary cleanup.

- gateway/platforms/base.py — abstract send_clarify with a numbered-text
  fallback so every adapter (Discord, Slack, WhatsApp, Signal, Matrix,
  etc.) gets a working clarify out of the box. Plus an active-session
  bypass: when the agent is blocked on a text-awaiting clarify, the next
  non-command message routes inline to the runner's intercept instead
  of being queued + triggering an interrupt. Same shape as the /approve
  deadlock fix from PR #4926.

- gateway/platforms/telegram.py — concrete send_clarify renders one
  inline button per choice plus '✏️ Other (type answer)'. cl: callback
  handler resolves numeric choices immediately, flips to text-capture
  mode for Other, with the same authorization guards as exec/slash
  approvals.

- gateway/run.py — clarify_callback wired at the cached-agent per-turn
  callback assignment site (only the user-facing agent path; cron and
  hygiene-compress agents have no human attached). Bridges sync→async
  via run_coroutine_threadsafe, blocks with the configured timeout, and
  returns a '[user did not respond within Xm]' sentinel on timeout so
  the agent adapts rather than pinning the running-agent guard. Text-
  intercept added to _handle_message before slash-confirm intercept
  (skipping slash commands). clear_session called in the run's finally
  to cancel any orphan entries.

- hermes_cli/config.py — agent.clarify_timeout default 600s.

- website/docs/user-guide/messaging/telegram.md — Interactive Prompts
  section.

Tests:

- tests/tools/test_clarify_gateway.py (14 tests) — full primitive
  coverage: button resolve, open-ended auto-await, Other flip, timeout
  None, unknown-id idempotency, clear_session cancellation, FIFO
  ordering, register/unregister notify, config default.

- tests/gateway/test_telegram_clarify_buttons.py (12 tests) — render
  paths (multi-choice/open-ended/long-label/HTML-escape/not-connected),
  callback dispatch (numeric resolve/Other flip/already-resolved/
  unauthorized/invalid-token), and base-adapter text fallback.

Out of scope: bot-to-bot, guest mode, checklists, poll media, live
photos. Closes #24191.
@teknium1 teknium1 force-pushed the feat/clarify-gateway-buttons branch from 869065a to 3dca4b7 Compare May 12, 2026 23:33
@teknium1 teknium1 merged commit 29d7c24 into main May 12, 2026
15 of 16 checks passed
@teknium1 teknium1 deleted the feat/clarify-gateway-buttons branch May 12, 2026 23:33
sunJose pushed a commit to sunJose/hermes-agent that referenced this pull request May 14, 2026
…gram (NousResearch#24199)

The clarify tool returned 'not available in this execution context' for
every gateway-mode agent because gateway/run.py never passed
clarify_callback into the AIAgent constructor. Schema actively encouraged
calling it; users never saw the question.

Changes:

- tools/clarify_gateway.py — new event-based primitive mirroring
  tools/approval.py: register/wait_for_response/resolve_gateway_clarify
  with per-session FIFO, threading.Event blocking with 1s heartbeat
  slices (so the inactivity watchdog keeps ticking), and
  clear_session for boundary cleanup.

- gateway/platforms/base.py — abstract send_clarify with a numbered-text
  fallback so every adapter (Discord, Slack, WhatsApp, Signal, Matrix,
  etc.) gets a working clarify out of the box. Plus an active-session
  bypass: when the agent is blocked on a text-awaiting clarify, the next
  non-command message routes inline to the runner's intercept instead
  of being queued + triggering an interrupt. Same shape as the /approve
  deadlock fix from PR NousResearch#4926.

- gateway/platforms/telegram.py — concrete send_clarify renders one
  inline button per choice plus '✏️ Other (type answer)'. cl: callback
  handler resolves numeric choices immediately, flips to text-capture
  mode for Other, with the same authorization guards as exec/slash
  approvals.

- gateway/run.py — clarify_callback wired at the cached-agent per-turn
  callback assignment site (only the user-facing agent path; cron and
  hygiene-compress agents have no human attached). Bridges sync→async
  via run_coroutine_threadsafe, blocks with the configured timeout, and
  returns a '[user did not respond within Xm]' sentinel on timeout so
  the agent adapts rather than pinning the running-agent guard. Text-
  intercept added to _handle_message before slash-confirm intercept
  (skipping slash commands). clear_session called in the run's finally
  to cancel any orphan entries.

- hermes_cli/config.py — agent.clarify_timeout default 600s.

- website/docs/user-guide/messaging/telegram.md — Interactive Prompts
  section.

Tests:

- tests/tools/test_clarify_gateway.py (14 tests) — full primitive
  coverage: button resolve, open-ended auto-await, Other flip, timeout
  None, unknown-id idempotency, clear_session cancellation, FIFO
  ordering, register/unregister notify, config default.

- tests/gateway/test_telegram_clarify_buttons.py (12 tests) — render
  paths (multi-choice/open-ended/long-label/HTML-escape/not-connected),
  callback dispatch (numeric resolve/Other flip/already-resolved/
  unauthorized/invalid-token), and base-adapter text fallback.

Out of scope: bot-to-bot, guest mode, checklists, poll media, live
photos. Closes NousResearch#24191.
jsboige pushed a commit to jsboige/hermes-agent that referenced this pull request May 14, 2026
…gram (NousResearch#24199)

The clarify tool returned 'not available in this execution context' for
every gateway-mode agent because gateway/run.py never passed
clarify_callback into the AIAgent constructor. Schema actively encouraged
calling it; users never saw the question.

Changes:

- tools/clarify_gateway.py — new event-based primitive mirroring
  tools/approval.py: register/wait_for_response/resolve_gateway_clarify
  with per-session FIFO, threading.Event blocking with 1s heartbeat
  slices (so the inactivity watchdog keeps ticking), and
  clear_session for boundary cleanup.

- gateway/platforms/base.py — abstract send_clarify with a numbered-text
  fallback so every adapter (Discord, Slack, WhatsApp, Signal, Matrix,
  etc.) gets a working clarify out of the box. Plus an active-session
  bypass: when the agent is blocked on a text-awaiting clarify, the next
  non-command message routes inline to the runner's intercept instead
  of being queued + triggering an interrupt. Same shape as the /approve
  deadlock fix from PR NousResearch#4926.

- gateway/platforms/telegram.py — concrete send_clarify renders one
  inline button per choice plus '✏️ Other (type answer)'. cl: callback
  handler resolves numeric choices immediately, flips to text-capture
  mode for Other, with the same authorization guards as exec/slash
  approvals.

- gateway/run.py — clarify_callback wired at the cached-agent per-turn
  callback assignment site (only the user-facing agent path; cron and
  hygiene-compress agents have no human attached). Bridges sync→async
  via run_coroutine_threadsafe, blocks with the configured timeout, and
  returns a '[user did not respond within Xm]' sentinel on timeout so
  the agent adapts rather than pinning the running-agent guard. Text-
  intercept added to _handle_message before slash-confirm intercept
  (skipping slash commands). clear_session called in the run's finally
  to cancel any orphan entries.

- hermes_cli/config.py — agent.clarify_timeout default 600s.

- website/docs/user-guide/messaging/telegram.md — Interactive Prompts
  section.

Tests:

- tests/tools/test_clarify_gateway.py (14 tests) — full primitive
  coverage: button resolve, open-ended auto-await, Other flip, timeout
  None, unknown-id idempotency, clear_session cancellation, FIFO
  ordering, register/unregister notify, config default.

- tests/gateway/test_telegram_clarify_buttons.py (12 tests) — render
  paths (multi-choice/open-ended/long-label/HTML-escape/not-connected),
  callback dispatch (numeric resolve/Other flip/already-resolved/
  unauthorized/invalid-token), and base-adapter text fallback.

Out of scope: bot-to-bot, guest mode, checklists, poll media, live
photos. Closes NousResearch#24191.
bladeJumper added a commit to bladeJumper/hermes-agent that referenced this pull request May 21, 2026
Implements the clarify tool on Feishu with interactive card buttons,
mirroring the Telegram pattern from NousResearch#24199.

- send_clarify: renders question + choice buttons as Feishu interactive card
- _handle_clarify_card_action: routes button callbacks via hermes_clarify_action
  - choice: resolves immediately with choice text
  - other: flips to text-capture mode via mark_awaiting_text
- Already-resolved guard prevents double-click / stale-button issues
- Authorization check reuses _is_interactive_operator_authorized
- Choice text lookup: stored state → clarify entry → fallback index

Known limitation: Feishu shows both the tool-progress bubble (❓ clarify: ...)
and the interactive card because Feishu lacks delete_message support.
This can be revisited once Feishu implements message deletion.

Closes NousResearch#12573
Closes NousResearch#21032
Supersedes NousResearch#23740 (which included framework code now merged in NousResearch#24199)
Related NousResearch#24199, NousResearch#21893, NousResearch#503
AlexFoxD pushed a commit to AlexFoxD/hermes-agent that referenced this pull request May 21, 2026
…gram (NousResearch#24199)

The clarify tool returned 'not available in this execution context' for
every gateway-mode agent because gateway/run.py never passed
clarify_callback into the AIAgent constructor. Schema actively encouraged
calling it; users never saw the question.

Changes:

- tools/clarify_gateway.py — new event-based primitive mirroring
  tools/approval.py: register/wait_for_response/resolve_gateway_clarify
  with per-session FIFO, threading.Event blocking with 1s heartbeat
  slices (so the inactivity watchdog keeps ticking), and
  clear_session for boundary cleanup.

- gateway/platforms/base.py — abstract send_clarify with a numbered-text
  fallback so every adapter (Discord, Slack, WhatsApp, Signal, Matrix,
  etc.) gets a working clarify out of the box. Plus an active-session
  bypass: when the agent is blocked on a text-awaiting clarify, the next
  non-command message routes inline to the runner's intercept instead
  of being queued + triggering an interrupt. Same shape as the /approve
  deadlock fix from PR NousResearch#4926.

- gateway/platforms/telegram.py — concrete send_clarify renders one
  inline button per choice plus '✏️ Other (type answer)'. cl: callback
  handler resolves numeric choices immediately, flips to text-capture
  mode for Other, with the same authorization guards as exec/slash
  approvals.

- gateway/run.py — clarify_callback wired at the cached-agent per-turn
  callback assignment site (only the user-facing agent path; cron and
  hygiene-compress agents have no human attached). Bridges sync→async
  via run_coroutine_threadsafe, blocks with the configured timeout, and
  returns a '[user did not respond within Xm]' sentinel on timeout so
  the agent adapts rather than pinning the running-agent guard. Text-
  intercept added to _handle_message before slash-confirm intercept
  (skipping slash commands). clear_session called in the run's finally
  to cancel any orphan entries.

- hermes_cli/config.py — agent.clarify_timeout default 600s.

- website/docs/user-guide/messaging/telegram.md — Interactive Prompts
  section.

Tests:

- tests/tools/test_clarify_gateway.py (14 tests) — full primitive
  coverage: button resolve, open-ended auto-await, Other flip, timeout
  None, unknown-id idempotency, clear_session cancellation, FIFO
  ordering, register/unregister notify, config default.

- tests/gateway/test_telegram_clarify_buttons.py (12 tests) — render
  paths (multi-choice/open-ended/long-label/HTML-escape/not-connected),
  callback dispatch (numeric resolve/Other flip/already-resolved/
  unauthorized/invalid-token), and base-adapter text fallback.

Out of scope: bot-to-bot, guest mode, checklists, poll media, live
photos. Closes NousResearch#24191.
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…gram (NousResearch#24199)

The clarify tool returned 'not available in this execution context' for
every gateway-mode agent because gateway/run.py never passed
clarify_callback into the AIAgent constructor. Schema actively encouraged
calling it; users never saw the question.

Changes:

- tools/clarify_gateway.py — new event-based primitive mirroring
  tools/approval.py: register/wait_for_response/resolve_gateway_clarify
  with per-session FIFO, threading.Event blocking with 1s heartbeat
  slices (so the inactivity watchdog keeps ticking), and
  clear_session for boundary cleanup.

- gateway/platforms/base.py — abstract send_clarify with a numbered-text
  fallback so every adapter (Discord, Slack, WhatsApp, Signal, Matrix,
  etc.) gets a working clarify out of the box. Plus an active-session
  bypass: when the agent is blocked on a text-awaiting clarify, the next
  non-command message routes inline to the runner's intercept instead
  of being queued + triggering an interrupt. Same shape as the /approve
  deadlock fix from PR NousResearch#4926.

- gateway/platforms/telegram.py — concrete send_clarify renders one
  inline button per choice plus '✏️ Other (type answer)'. cl: callback
  handler resolves numeric choices immediately, flips to text-capture
  mode for Other, with the same authorization guards as exec/slash
  approvals.

- gateway/run.py — clarify_callback wired at the cached-agent per-turn
  callback assignment site (only the user-facing agent path; cron and
  hygiene-compress agents have no human attached). Bridges sync→async
  via run_coroutine_threadsafe, blocks with the configured timeout, and
  returns a '[user did not respond within Xm]' sentinel on timeout so
  the agent adapts rather than pinning the running-agent guard. Text-
  intercept added to _handle_message before slash-confirm intercept
  (skipping slash commands). clear_session called in the run's finally
  to cancel any orphan entries.

- hermes_cli/config.py — agent.clarify_timeout default 600s.

- website/docs/user-guide/messaging/telegram.md — Interactive Prompts
  section.

Tests:

- tests/tools/test_clarify_gateway.py (14 tests) — full primitive
  coverage: button resolve, open-ended auto-await, Other flip, timeout
  None, unknown-id idempotency, clear_session cancellation, FIFO
  ordering, register/unregister notify, config default.

- tests/gateway/test_telegram_clarify_buttons.py (12 tests) — render
  paths (multi-choice/open-ended/long-label/HTML-escape/not-connected),
  callback dispatch (numeric resolve/Other flip/already-resolved/
  unauthorized/invalid-token), and base-adapter text fallback.

Out of scope: bot-to-bot, guest mode, checklists, poll media, live
photos. Closes NousResearch#24191.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent loop, run_agent.py, prompt builder comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(gateway): wire clarify tool end-to-end with inline keyboard buttons on Telegram

2 participants