Skip to content

feat(matrix): add scoped native tools and reaction controls#18506

Open
nepenth wants to merge 10 commits into
NousResearch:mainfrom
nepenth:feature/matrix-tools-interactions
Open

feat(matrix): add scoped native tools and reaction controls#18506
nepenth wants to merge 10 commits into
NousResearch:mainfrom
nepenth:feature/matrix-tools-interactions

Conversation

@nepenth

@nepenth nepenth commented May 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds the Matrix tools/interactions layer now that the Matrix foundation work from #18505 has merged into main.

Scope

  • Matrix-native tools and Matrix toolset registration.
  • Current-room authorization for Matrix tools.
  • Explicit admin gates for redaction, invites, and room creation.
  • Reaction-based dangerous-command approvals and /model selection.
  • Collapsible Matrix tool-activity messages with raw thinking hidden by default.

Review Notes

This PR is now the active Matrix review target. It was rebuilt on top of current main after #18505 merged, so the PR commit list contains only this tools/interactions layer.

Matrix does not provide a portable native button surface equivalent to Slack Block Kit, Discord components, or Telegram inline keyboards. This PR keeps the upstream-safe path reaction-based so it works on ordinary Matrix homeservers and clients.

Validation

  • Local Matrix/browser subset: 375 passed, 1 skipped.
  • GitHub CI is green.

@alt-glitch alt-glitch added type/feature New feature or request P2 Medium — degraded but workaround exists platform/matrix Matrix adapter (E2EE) comp/gateway Gateway runner, session dispatch, delivery comp/tools Tool registry, model_tools, toolsets labels May 1, 2026
@nepenth nepenth marked this pull request as ready for review May 1, 2026 19:35
@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch 3 times, most recently from 5c51c4f to a75c78d Compare May 7, 2026 19:59
@nepenth

nepenth commented May 7, 2026

Copy link
Copy Markdown
Contributor Author

May 7, 2026 Update

This PR has been refreshed on top of the updated Matrix foundation branch after the v2026.5.7 release.

Branch Refresh

  • Rebased onto refreshed PR 1 / feature/matrix-foundation.
  • Current head: a75c78d78.
  • Base head observed during refresh: 04193cf71.

Matrix UX Fix

  • Increased Matrix tool-progress preview length from the inherited compact default to a Matrix-specific default of 160 characters.
  • Rationale: Matrix uses editable progress panes, so slightly longer bounded previews are useful and do not create the same channel noise as permanent one-message-per-tool platforms.
  • The value remains bounded and can still be overridden through display.platforms.matrix.tool_preview_length.

Local Validation

Passed:

scripts/run_tests.sh \
  tests/tools/test_matrix_tools.py \
  tests/gateway/test_matrix_exec_approval.py \
  tests/gateway/test_matrix_mention.py \
  tests/gateway/test_matrix_project_context_isolation.py \
  tests/agent/test_prompt_builder.py::TestPromptBuilderConstants::test_platform_hints_matrix \
  -q -o 'addopts='

Additional Matrix/display validation passed as part of the final stack check:

338 passed, 1 skipped

Also passed:

python -m py_compile \
  gateway/platforms/matrix.py \
  gateway/run.py \
  tools/matrix_tools.py \
  toolsets.py \
  gateway/display_config.py

git diff --check

@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch 2 times, most recently from 13f944d to 6735a7f Compare May 14, 2026 12:35
@nepenth

nepenth commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

Update: Matrix output defaults

This PR now defaults Matrix chat output to final answers only.

Why:

  • Matrix clients often treat edited/replaced progress messages as read-marker-relevant events.
  • Live tool/thinking panes can cause clients to jump around the room after returning to the channel.
  • Global gateway streaming/progress settings should not accidentally enable that behavior for Matrix.

Behavior after this update:

  • Matrix suppresses live tool/thinking progress and interim assistant streaming by default.
  • Final responses are still delivered normally.
  • Operators can explicitly opt back in with:
display:
  platforms:
    matrix:
      tool_progress: new
      streaming: true
      interim_assistant_messages: true

Validation:

  • Focused Matrix/gateway test set passes locally: 297 passed.

@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch from 6735a7f to 9cf1aea Compare May 14, 2026 12:54
@nepenth

nepenth commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

History cleanup

Rebased and cleaned commit history for reviewability.

  • No scope change.
  • Final tree is equivalent to the previous branch state.
  • Focused Matrix validation still passes locally: 297 passed.

@nepenth

nepenth commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

Update: Matrix tool activity output

Adjusted the Matrix output defaults after local client testing.

Behavior now:

  • Matrix shows live tool activity by default (tool_progress: new).
  • Matrix still suppresses raw reasoning/thinking panes unless display.platforms.matrix.show_reasoning: true is explicitly set.
  • Matrix response token streaming remains off unless display.platforms.matrix.streaming: true is set.
  • Tool activity is sent with a Matrix formatted-body details/summary block plus a plain-text fallback body, so clients that support collapsible HTML can hide/show details while other clients still show usable text.

Validation:

  • Focused Matrix stack passed locally: 323 passed.
  • py_compile and git diff --check passed.

@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch from 8a94d1b to 6951dbd Compare May 16, 2026 19:04
@nepenth

nepenth commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

v2026.5.16 refresh

Rebased this Matrix PR stack onto current upstream main after the v2026.5.16 / v0.14.0 release.

Notes:

  • The v2026.5.16 release did not include this Matrix parity work; these PRs remain open and unmerged.
  • Checked Matrix/gateway-adjacent upstream changes from the release window.
  • Focused Matrix validation passed locally: 323 passed.
  • py_compile and git diff --check passed.

@nepenth nepenth marked this pull request as draft May 16, 2026 19:45
@nepenth

nepenth commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

Draft stacked preview

This PR is intentionally marked draft while #18505 is the active review target. It is kept open so maintainers and external reviewers can see the next Matrix layer without treating it as an independent merge candidate yet.

Depends on

Scope

  • Matrix-native tools and Matrix toolset registration.
  • Current-room authorization for Matrix tools.
  • Explicit admin gates for redaction, invites, and room creation.
  • Reaction-based dangerous-command approvals and /model selection.
  • Collapsible Matrix tool-activity messages with raw thinking hidden by default.

Design note
Matrix does not provide a portable native button surface equivalent to Slack Block Kit, Discord components, or Telegram inline keyboards. This PR keeps the upstream-safe path reaction-based so it works on ordinary Matrix homeservers and clients. Optional richer metadata remains isolated in #23815.

I will move this out of draft once #18505 has maintainer direction.

@nepenth

nepenth commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

Update: Matrix tool activity preview width

Pushed a focused fix for Matrix tool-activity rendering.

What changed

  • Matrix default tool_preview_length is now 1000 instead of 320.
  • Matrix no longer falls back to the generic 40-character chat preview cap when global display.tool_preview_length: 0 is present.
  • Added regression coverage proving Matrix tool activity preserves long command previews in the collapsible <details><pre><code> block.

This keeps the single edited tool-activity pane readable without enabling raw thinking or response streaming. The downstream stacked branches (#18507 and #23815) have the same fix.

@nepenth

nepenth commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

Correction: restored Matrix tool activity preview scope

I backed out the broader preview-width change from the previous update.

Current state

  • Matrix tool_preview_length is back to 320.
  • The extra gateway/run.py fallback logic was removed.
  • The wider 1000 default and its regression test were removed.

This keeps the PR closer to the existing display configuration model and avoids expanding the review surface. For deployments that see short Matrix tool lines because global display.tool_preview_length: 0 is present, the intended configuration is an explicit Matrix platform override:

display:
  platforms:
    matrix:
      tool_progress: new
      tool_preview_length: 320

The local all-stack test branch has the same rollback.

@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch 3 times, most recently from 92e8b10 to 31d8bd0 Compare May 31, 2026 11:16
@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch 3 times, most recently from 09e0c96 to b1b9fa0 Compare June 6, 2026 13:19
@nepenth

nepenth commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Rebased this PR onto current main (56236b16e) after the v0.16.0 / v2026.6.5 release.

Notes:

The custom Synapse/AgentFirst metadata PR (#23815) has been retired from the active stack to keep this series focused on upstreamable core Matrix functionality.

@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch from b1b9fa0 to 17ec09b Compare June 10, 2026 13:44
@nepenth

nepenth commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Jun 10, 2026 stacked preview refresh

Rebased onto current upstream main via the refreshed foundation stack (#18505).

Item Value
Base (main) a72bb037
Branch head f9fc0705a
Commits ahead of main 18 (11 incremental atop #18505)
Status draft stacked preview — do not merge before #18505

Incremental scope (review commits after #18505 branch point)

  • tools/matrix_tools.py — room-local Matrix-native tools with layered authorization:
    • session-platform gate (Matrix-only at runtime)
    • current-room default; MATRIX_TOOLS_ALLOW_CROSS_ROOM / _DESTRUCTIVE opt-ins
    • per-action admin gates for redaction, invites, room creation
    • MATRIX_ALLOWED_ROOMS allowlist
  • Reaction-based dangerous-command approval + requester-bound validation
  • Reaction-based /model picker
  • Thinking + tool-activity progress panes (gateway/run.py)
  • i18n catalog parity for Matrix slash-command keys (cherry-picked from foundation)

Local validation

.venv/bin/python -m pytest \
  tests/tools/test_matrix_tools.py \
  tests/gateway/test_matrix_exec_approval.py \
  tests/gateway/test_matrix_mention.py \
  tests/gateway/test_run_progress_topics.py \
  tests/gateway/test_matrix_project_context_isolation.py -q

Result: passes locally on refreshed branch (part of 529-test Matrix suite run today)

Review note

Until #18505 merges, the Files changed tab includes foundation commits. For incremental review, filter to the 11 commits after the foundation branch point (from 7424b0aa4 feat(matrix): add scoped native tools and reactions through f9fc0705a).

Leaving this as draft per prior agreement — will mark ready for review once maintainers signal on #18505.

@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch 2 times, most recently from 7137639 to 3fbf4b2 Compare June 11, 2026 02:46
@nepenth

nepenth commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Jun 11 stacked refresh

Restacked this PR onto the updated #18505 foundation branch.

Validation:

  • GitHub CI is green on this PR.

@nepenth nepenth force-pushed the feature/matrix-tools-interactions branch from 3fbf4b2 to 172154f Compare June 11, 2026 11:58
@nepenth

nepenth commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Jun 11 update: #18505 merged, branch rebuilt on main

#18505 has landed in main, so this PR has been rebuilt on top of the merged foundation commit.

  • Commit list now contains only this PR's 10 Matrix tools/interactions commits.
  • Old foundation commits from the previous 3-PR stack are no longer part of this PR.
  • feature/matrix-full-stack in the fork was refreshed to the cleaned top of stack.
  • This PR is now marked ready for review as the next active Matrix layer.

Validation:

  • Local Matrix/browser subset: 375 passed, 1 skipped.
  • GitHub CI is green.

@nepenth nepenth marked this pull request as ready for review June 11, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists platform/matrix Matrix adapter (E2EE) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants