Skip to content

feat(matrix): add optional AgentFirstModule metadata support#23815

Closed
nepenth wants to merge 22 commits into
NousResearch:mainfrom
nepenth:feature/matrix-agentfirst-metadata
Closed

feat(matrix): add optional AgentFirstModule metadata support#23815
nepenth wants to merge 22 commits into
NousResearch:mainfrom
nepenth:feature/matrix-agentfirst-metadata

Conversation

@nepenth

@nepenth nepenth commented May 11, 2026

Copy link
Copy Markdown
Contributor

Summary

This is PR 4 in the Matrix gateway parity series and should be reviewed after
#18505, #18506, and #18507.

It adds optional support for structured Matrix agent metadata emitted by the
companion Synapse module
AgentFirstModule.

The integration is intentionally optional. Standard Matrix homeservers do not
emit this metadata, and the Matrix adapter falls back to existing behavior when
the fields are absent.

Why

The first three Matrix PRs add the foundation, native Matrix tools/interactions,
and Matrix rendering/media/E2EE/diagnostics hardening.

This PR adds the optional enhancement layer for deployments that want richer
agent-native Matrix behavior from their homeserver:

  • structured session scope hints
  • room identity hints
  • tool status metadata
  • approval request metadata
  • tool call/result metadata
  • typing/status metadata

Changes

Matrix adapter

  • Parses optional AgentFirstModule metadata from Matrix event unsigned.
  • Exposes parsed metadata on inbound MessageEvent.agent.
  • Applies safe room/session metadata to SessionSource.
  • Keeps behavior unchanged when metadata is missing.

Session context

  • Adds optional SessionSource.agent_metadata.
  • Adds SessionSource.apply_agent_metadata().
  • Applies session_scope=room by clearing synthetic thread scope for the turn.
  • Applies room_identity only when the metadata room ID matches the actual Matrix room ID.
  • Adds a concise Matrix prompt note when AgentFirstModule metadata is present.

Agent UI helpers

  • Adds gateway/agent_ui.py with small rendering helpers for approval request and tool status payloads.
  • No new dependencies.

Docs/tests

  • Documents optional AgentFirstModule metadata support in the Matrix guide.
  • Adds focused tests for parsing, inbound handler propagation, room identity safety, session key behavior, prompt notes, UI helpers, and unchanged behavior when AgentFirst metadata is absent.

Security / Compatibility

  • No behavior change on standard Matrix homeservers.
  • Durable routing still uses Matrix room IDs, not mutable room names.
  • AgentFirst room identity metadata cannot override the event's actual Matrix room ID.
  • No Honcho or memory-provider behavior is changed.
  • No new dependencies are introduced.

Validation

Passed locally:

python -m py_compile \
  gateway/platforms/matrix.py \
  gateway/session.py \
  gateway/platforms/base.py \
  gateway/agent_ui.py

scripts/run_tests.sh \
  tests/gateway/test_matrix_agentfirst_metadata.py \
  tests/gateway/test_matrix.py \
  tests/gateway/test_matrix_project_context_isolation.py \
  tests/gateway/test_matrix_exec_approval.py \
  tests/tools/test_matrix_tools.py \
  -q -o 'addopts='

git diff --check

Result:

259 passed

Stack

Builds on:

This PR should be reviewed after the first three Matrix PRs.

@daimon-nous daimon-nous Bot added type/feature New feature or request P3 Low — cosmetic, nice to have platform/matrix Matrix adapter (E2EE) comp/gateway Gateway runner, session dispatch, delivery labels May 11, 2026
@nepenth nepenth force-pushed the feature/matrix-agentfirst-metadata branch 2 times, most recently from 40062e8 to 5584503 Compare May 11, 2026 15:33
@nepenth nepenth changed the title feat(matrix): add optional AgentFirst metadata support feat(matrix): add optional AgentFirstModule metadata support May 11, 2026
@nepenth nepenth marked this pull request as ready for review May 11, 2026 15:35
@nepenth nepenth requested a review from a team May 12, 2026 13:11
@nepenth nepenth force-pushed the feature/matrix-agentfirst-metadata branch from 9d55a3c to b8a722e Compare May 14, 2026 12:30
@nepenth nepenth requested a review from a team May 14, 2026 12:30
@nepenth nepenth force-pushed the feature/matrix-agentfirst-metadata branch 2 times, most recently from fa06076 to 3f9cf63 Compare May 14, 2026 12:54
@nepenth

nepenth commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

History cleanup

Rebased and cleaned commit history for reviewability.

  • No scope change.
  • Final tree is equivalent to the previous branch state.
  • Focused Matrix validation still passes locally: 297 passed.

@nepenth nepenth force-pushed the feature/matrix-agentfirst-metadata branch 3 times, most recently from 34f8545 to 18f395e Compare May 16, 2026 19:04
@nepenth

nepenth commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

v2026.5.16 refresh

Rebased this Matrix PR stack onto current upstream main after the v2026.5.16 / v0.14.0 release.

Notes:

  • The v2026.5.16 release did not include this Matrix parity work; these PRs remain open and unmerged.
  • Checked Matrix/gateway-adjacent upstream changes from the release window.
  • Preserved upstream MessageEvent.channel_context while retaining optional AgentFirst metadata.
  • Focused Matrix validation passed locally: 323 passed.
  • py_compile and git diff --check passed.

@nepenth

nepenth commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

Draft optional enhancement

This PR is intentionally marked draft while the core Matrix stack (#18505, #18506, #18507) is reviewed first. It is optional and should not be treated as required for the standard Matrix gateway functionality.

Depends on

Scope

  • Parse optional unsigned.agent_metadata from AgentFirstModule.
  • Preserve standard Matrix behavior when no metadata is present.
  • Validate room_identity against the actual Matrix event room ID.
  • Expose structured metadata on MessageEvent.agent and SessionSource.agent_metadata.

Non-goal
This does not add a hard dependency on the companion homeserver module. Ordinary Matrix homeservers continue to work without behavior changes. The standard upstream interaction path remains reaction-based because Matrix has no portable native button API comparable to Slack/Discord/Telegram buttons.

@nepenth nepenth force-pushed the feature/matrix-agentfirst-metadata branch 3 times, most recently from 45c6b64 to 7851604 Compare May 31, 2026 11:16
@nepenth nepenth force-pushed the feature/matrix-agentfirst-metadata branch 2 times, most recently from 5a3daea to 3db09ec Compare June 2, 2026 01:16
@nepenth nepenth force-pushed the feature/matrix-agentfirst-metadata branch from 3db09ec to bff674c Compare June 2, 2026 01:23
@nepenth

nepenth commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Retiring this PR from the active Matrix stack for now. The custom Synapse/AgentFirst metadata integration adds non-standard homeserver-side coupling and increases review surface for the core Matrix work. We are keeping the upstreamable Hermes Matrix stack focused on the foundation, native Matrix tools/interactions, and media/E2EE/diagnostics PRs, and can reassess AgentFirst/Synapse metadata later if we pursue a cleaner upstream Synapse/MSC path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have platform/matrix Matrix adapter (E2EE) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant