Add peer-to-peer federation#38
Merged
Merged
Conversation
Lay the groundwork for full content federation where servers trust each
other directly via a configured peer list (no central directory). This
phase establishes server identity, peer trust, and the signed transport
envelope end-to-end; content application (messages, membership, reactions)
follows in later phases.
What's included:
- Ed25519 server identity (src/federation/identity.rs), persisted 0600
beside master_server_id and published at GET /.well-known/accord-federation.
- HTTP-signature signing/verification over method+path+host+Date+SHA-256
body digest, with a clock-skew replay bound (src/federation/signatures.rs).
- Qualified-ID helpers and the FederationEnvelope type (src/federation/mapping.rs).
- Authority chokepoint enforcing origin==signer and space-homed-on-signer,
plus the no-local-overwrite guard (src/federation/authority.rs).
- Inbound POST /federation/v1/inbox pipeline: verify -> authority -> trust
gate -> dedup -> apply (src/federation/inbox.rs). Handles m.ping; rejects
unknown event types pending later phases.
- Durable outbound queue + signed delivery with exponential backoff and a
dead-letter cap (src/federation/sender.rs), spawned from main.
- Peer .well-known discovery with an SSRF guard (src/federation/peers.rs).
- DB layer for peers, inbox dedup, and the outbox (src/db/federation.rs).
- Admin peer management API (GET/POST /admin/federation/peers,
PATCH/DELETE /admin/federation/peers/{domain}).
- Migrations 027 (sqlite) / 012 (postgres): users.origin column and the
federation_peers / federation_inbox_dedup / federation_outbox tables.
- Config (FEDERATION_DOMAIN/PUBLIC_URL/ENABLED) and AppState wiring; the
feature is inert unless configured.
Tests: 11 unit + 9 integration tests covering signed-ping acceptance and
the negative security cases (bad signature, unknown/pending peer, origin
spoofing, missing signature, dedup idempotency). Full suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
…tion Phase 1 (remote identity): - db::users::upsert_remote_user stores a federated user as a normal `users` row keyed by qualified ID, with the fully-qualified handle in `username` (collision-free against local bare usernames) and origin set. - snowflake::timestamp_of tolerates qualified IDs (strips @Domain). Phase 2 (inbound content injection seam): - Migration 028/013: `origin` on spaces/channels/roles/messages/members and spaces.federation_enabled; indexes on members/spaces/messages origin. - Replica upserts in db::federation (space/channel/role/member) keyed by qualified IDs with origin set; db::messages::insert_remote_message stores a mirrored message idempotently; db::federation::interested_servers computes the fanout target set (distinct member origins). - src/federation/apply.rs applies m.message.create: re-checks authority (S1: message/author/channel homed on the signing peer), enforces input caps (S6), ignores events for unmirrored channels, upserts the remote author, stores the replica message, and injects a message.create at the gateway_tx seam — delivery-only, never re-fanning out (S7 loop prevention). - Inbox dispatches non-ping events through the applier. - MessageRow/models gain an `origin` field. Tests: 12 federation integration tests (added inbound message stored+broadcast, spoofed-author rejected, unmirrored-channel ignored). Full suite green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
When a message is created in a locally-homed space, fan it out to every peer that has a member there: - db::federation::space_origin (am I authoritative?) + interested_servers (distinct member origins) gate and target the fanout. - src/federation/outbound.rs builds an m.message.create envelope with all IDs qualified to our domain (including author profile so peers can cache it) and enqueues it via the durable sender to interested peers only. - Wired into routes/messages.rs create_message after the local gateway broadcast; non-fatal and a no-op unless federation is on and the space is local with >=1 remote member. Guards: fanout is skipped for remote-homed spaces (we are not authoritative; the forward path will handle those) and for DMs. Combined with the inbound applier's inject-only behaviour, this preserves S7 loop prevention. Tests: +2 (fans out a qualified envelope to interested peers; no fanout without remote members). 14 federation tests; full suite green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
A user on server A can now join a space homed on server B:
- src/federation/handshake.rs implements both sides:
- Home side (POST /federation/v1/join): verifies the signature (shared
verify::verify_signed), binds the joining user to the signing peer (S1),
requires the space to be federation_enabled (S9) and the user un-banned
(reuses db::bans::get_ban), adds the remote user as a member, and returns a
snapshot (space, channels, roles, members, recent messages) with all IDs
qualified to the home domain.
- Joining side (apply_snapshot): mirrors the snapshot as local replica rows
(origin = home domain), refusing any entity not homed on the peer (S2),
and records the local joining user as a member so the mirrored space shows
up in their space list / gateway delivery.
- Extracted shared inbound signature+trust verification into
src/federation/verify.rs; inbox now reuses it.
- db::federation: space_federation_enabled / set_space_federation_enabled,
space_origin, replica upserts already added in the prior commit.
- mapping::handle centralises qualified-handle construction (idempotent),
used by the applier and snapshot builder.
Tests: +4 (home serves join & records remote member; refuses non-federated
space; joiner applies snapshot as replica incl. local membership; joiner
rejects a snapshot targeting a local space). 18 federation tests; full suite
green; clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
…trip
Completes bidirectional content federation: a user on A can join a space homed
on B and post into it, with messages flowing both ways.
- src/federation/forward.rs:
- handle_send (POST /federation/v1/send, home side): a peer forwards one of
its members' message actions; the home server re-runs permissions from its
OWN DB via require_channel_permission with a synthetic AuthUser (authority
stays home, never trusting the request), persists the canonical message,
broadcasts locally, and fans out to interested peers.
- forward_message / initiate_join (replica side): synchronous signed S2S
calls used by local routes.
- routes/messages.rs create_message now forwards to the home server for
remote-homed spaces instead of persisting locally; the authoritative message
returns via the inbox fanout.
- New authed REST trigger POST /api/v1/federation/spaces/join
(spaces::join_federated_space) initiates a join and applies the snapshot.
- sender::request_signed (synchronous signed POST) + sender::deliver_due_once
(drain the outbox once; also used by the background loop).
- outbound::message_payload extracted and shared by fanout and the /send reply.
Authority refinement: the home server is authoritative for its space's whole
message stream, so an inbound/snapshot message must have its id/channel/space
homed on the signing peer, but its AUTHOR may be a member from another server
(relayed). Members in a join snapshot likewise may come from several servers;
only the space's own state must be homed on the home server. The spoofed-author
test is repurposed to assert a foreign message/channel home is rejected.
Tests: +1 two-server round-trip over real HTTP (join handshake -> remote-homed
post forwarded to B -> fanout back to A), using ACCORD_FEDERATION_ALLOW_INSECURE
for loopback. 19 federation tests; full suite green; clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
…anout
Extends live federation beyond message.create so replicas stay current:
- Reactions, bidirectional:
- Inbox appliers m.reaction.add / m.reaction.remove: bind message/channel to
the signing peer (author/reactor may be remote), upsert the reactor, apply
to the replica reactions table idempotently, and rebroadcast.
- Home POST /federation/v1/react: re-runs add_reactions / membership from our
own DB, applies, broadcasts, and fans out to interested peers.
- routes/reactions.rs add_reaction & remove_own_reaction now forward to the
home server for remote-homed spaces and fan out for locally-homed ones.
- Message edit/delete, home -> replica:
- Inbox appliers m.message.update / m.message.delete mutate only replica rows
homed on the signing peer (S2) and rebroadcast message.update/.delete.
- routes/messages.rs update_message / delete_message fan the change out to
interested peers for locally-homed spaces.
- Generalised outbound::fanout_to_space (reused by messages/reactions/edits/
deletes) and added reaction_payload; new db helpers edit_remote_message,
add_reaction, remove_reaction; MessageRow already carries origin.
Tests: two-server e2e now also round-trips a reaction over real HTTP
(alice reacts in a remote-homed space -> forwarded to B -> fanned back to A);
added an inbound edit+delete test. 20 federation tests; full suite green; clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
- S7: per-peer inbound rate limiting. FederationContext carries a token-bucket DashMap keyed by peer domain (300 req/60s); enforced centrally in verify::verify_signed after the trust gate, so all signed endpoints (inbox, join, send, react) are covered. Exhaustion returns 429. - S3: bounded dedup table. db::federation::cleanup_dedup deletes federation_inbox_dedup rows older than the retention window; the sender loop prunes ~hourly (24h retention). Tests: +1 unit (bucket exhausts then blocks; independent per peer) and +1 integration (cleanup prunes old rows, fresh survives). Full suite green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
… live Closes the last core liveness gap: replica member lists now track joins and leaves on the home server, and a local user can actually leave a remote-homed space (previously only their replica row was dropped while the home still listed them). - Inbox appliers m.member.join / m.member.leave (src/federation/apply.rs): bound to the signing peer's space, upsert/add or remove the member on the replica, and rebroadcast member.join/.leave. - Home -> replica fanout at the local member-change sites for locally-homed spaces (reusing outbound::fanout_to_space): - join: routes/spaces.rs join_public_space, routes/invites.rs accept_invite. - leave/kick: routes/members.rs leave_space, kick_member. - outbound::member_join_payload / member_leave_payload qualify IDs. - Remote-homed leave forward: new POST /federation/v1/leave (forward::handle_leave -> serve_leave re-checks membership from our DB, removes, broadcasts, fans out) and forward::forward_leave; leave_space now forwards to the home for remote-homed spaces then drops the local replica row. Kick on a remote-homed space remains home-authority-only (deferred). Tests: +1 inbound member join+leave applied; two-server e2e extended with a leave round-trip (alice leaves -> forwarded -> B drops her -> a.test no longer an interested server). 22 federation tests; full suite green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
… complete Closes the remaining bidirectional gaps so every core chat action federates both ways: - Remote-homed message edit/delete forward: new POST /federation/v1/edit and /federation/v1/delete (forward::handle_edit/handle_delete -> serve_* re-run the author-or-manage check from our own DB, apply, broadcast, fan out, and return the authoritative result). routes/messages.rs update_message and delete_message now forward to the home for remote-homed spaces instead of failing the local author check (a forwarded message's author is the qualified remote id). - Typing indicators: POST /federation/v1/typing (home re-broadcasts + fans out to interested peers) and inbox m.typing applier (ephemeral rebroadcast, no DB). routes/messages.rs typing_indicator forwards for remote-homed spaces and fans out for locally-homed ones. - forward::actor_ref centralises the qualified-actor object used by the send/react/leave/edit/delete/typing forwards. Tests: the two-server e2e now round-trips the full lifecycle over real HTTP — join, post, react, edit, typing, delete, leave. Full suite green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
Security review found that the no-local-overwrite guard (S2) was bypassable: the inbound appliers passed author/reactor/member user IDs straight into upsert_remote_user without requiring them to be qualified. A trusted peer could send an event whose author/reactor/member ID was a BARE local snowflake (e.g. "42"); domain_of returns None so origin defaulted to the peer, and INSERT ... ON CONFLICT(id="42") DO UPDATE overwrote the LOCAL user's username/display_name/avatar and flipped its origin — identity takeover of any local user (incl. admins) across the trust boundary. Fix: call authority::require_remote_target (reject unqualified IDs) on every inbound user-ID path before upsert_remote_user — apply_message_create (author), apply_reaction (reactor), apply_member_join (member), and apply_snapshot (members + message authors). Qualified IDs can never collide with local bare snowflakes, so legitimate relayed/remote users are unaffected. The space/ channel/message/role paths were already guarded with require_homed_on. Tests: +2 regression tests (inbound message and reaction with a bare local author/reactor ID are rejected 403 and the local user row is untouched). Full suite green; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4N5r2u3KcGouc7Nfnq9cP
Adds Phase 1 (1:1) cross-server direct messages and addresses the findings from the federation review. Cross-server DMs (src/federation/dm.rs): - Deterministic home anchoring (smaller qualified user id) so both servers agree on a single home without duplicating the channel. - dm/open + dm/announce handshake with consent (block list) enforced by the recipient's own server; dm/send forward; m.dm.message.create applier; participant-addressed fanout. - Gateway delivery for DMs now targets participant ids instead of the buggy (None, None) global broadcast. - Wiring: remote-recipient detection in create_dm_channel, DM forward/fanout in create_message, S2S endpoints, apply dispatch. Review hardening fixes: - Member-leave fanout captures interested servers before removal so the departing member's home server is still notified (kick/leave/serve_leave). - Inbox rolls back the dedup record when apply fails so a peer retry can re-apply instead of being dropped. - SSRF: resolve peer hosts and reject names mapping to private IPs. - Signature replay cache on the synchronous forward endpoints (inbox excluded; it dedups by event id). - Sender signs the receiver's INBOX_PATH constant for path consistency; serve_react verifies the target message exists and belongs to the channel. Tests: cross_server_dm_round_trip (home-agnostic); serialise the two env-var-mutating two-server tests to avoid a global-state race. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced Jun 19, 2026
Addresses the remaining findings from the federation security review: - Profile spoofing/clobbering (S2): only a user's own home server may overwrite its cached profile. Add db::users::ensure_remote_user (insert without overwrite) and route message authors, member joins, reactions and join-snapshot users through it unless the signing peer is the user's home. Reactions carry no profile, so they no longer wipe an existing display name/avatar. - SSRF via redirects (S5): build the federation HTTP client with redirects disabled and a 15s timeout, so a peer cannot 3xx to an unvalidated internal address after validate_peer_url_resolved. - Identity key: load_or_create only generates on NotFound; a present but unreadable/corrupt key is now a hard error instead of silently rotating the server identity and breaking established peer trust. - S2S pre-auth cost: reject missing/stale Date before the peer DB lookup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…factor Federate custom emoji for remote-homed spaces: included in the join snapshot and propagated live via m.emoji.create/update/delete fanout. Mirrored emoji use qualified IDs (origin = home domain) and reference the home server's image by absolute URL rather than mirroring bytes. Emoji mutations on a mirrored space are rejected — they must run on the home server. Inbound appliers re-check authority, input caps, and only ever touch replica rows homed on the signing peer (S2). Also folds in the in-progress S2S refactor this builds on: a shared verify::prepare helper for the signed endpoints, and RemoteUserRef relocated to mapping with an optional username + username_or_id(). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced Jun 19, 2026
Open
Three Postgres-only failures in the federation suite: - audit_log.changes was JSONB in the Postgres migration but TEXT in SQLite, and the code binds it as text — even a NULL bind is rejected with "column is of type jsonb but expression is of type text". Align the Postgres column to TEXT to match SQLite and the Rust type. - upsert_remote_space bound the bare remote slug, which collides with the globally-unique idx_spaces_slug. Qualify the mirrored slug with the home domain so it stays globally unique. - Postgres test harness did not truncate the federation tables between tests, leaking state across cases. Add federation_peers, federation_inbox_dedup, and federation_outbox to the truncate list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Inbound appliers scoped their gateway broadcast on the peer-supplied space_id rather than the resource's own space, letting a trusted peer misroute (or globally fan out) events to spaces the resource does not belong to: - apply_message_create: store and broadcast on the mirrored channel's own space_id, not payload.space_id. - apply_emoji_delete: read the emoji's space_id before deletion and broadcast on that, not payload.space_id; drop the now-unused field. Member/emoji-upsert/message-update/delete/reaction paths already route on authority-checked or DB-sourced values, so they are unaffected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Security hardening for the federation feature: - SSRF (S5): fold IPv4-mapped/compatible IPv6 back to V4 checks and add ULA/link-local/CGNAT/broadcast/multicast coverage in is_private; add a connect-time SsrfGuardResolver wired into the HTTP client to close the DNS-rebinding/TOCTOU gap. - DM consent (S2): bind the announced opener to the signing peer and enforce block-lists for each local participant against every other participant, not just the opener. - Identity (S2/S3): mirror_user resolves @our_domain refs to a real local account and caches the real local profile instead of peer-supplied values. - Input caps (S6): cap remote embed byte size, reaction emoji length, and author display name; require http(s) + length-bounded avatar/emoji URLs to block stored-XSS via javascript:/data: schemes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements peer-to-peer federation for Accord, enabling multiple independent servers to exchange content and allow users to join spaces homed on remote servers. This adds the full federation pipeline: signature verification, authority binding, trust gates, deduplication, join handshakes, message/reaction/member forwarding, and durable outbound delivery with exponential backoff.
Federation is opt-in: inactive unless
FEDERATION_DOMAINis set, and per-space unlessfederation_enabledis turned on.Key Changes
Federation infrastructure (
src/federation/)identity.rs: Ed25519 keypair generation and persistence (0600) for server signingsignatures.rs: HTTP signature signing/verification over a canonical method/path/host/date/SHA-256-digest string, with a 5-minute clock-skew replay boundmapping.rs: Qualified ID helpers (<snowflake>@<domain>) and the federation event envelopeauthority.rs: Authority invariant enforcement — the signing peer must be the home server of any mutated resource, and federation input may never target a local entityverify.rs: Shared inbound verification (parse signature → resolve peer → verify → trust gate → per-peer rate limit) for all S2S endpointsInbound pipeline
inbox.rs:POST /federation/v1/inbox— verify → authority → trust → dedup → applyapply.rs: Appliers form.message.create/update/delete,m.reaction.add/remove,m.member.join/leave, andm.typing, with loop prevention (inbound never triggers outbound fanout) and input capswellknown.rs:GET /.well-known/accord-federationmetadata for peer discoveryOutbound pipeline
sender.rs: Durable outbound queue drained by a background task (up to 12 attempts, 1-hour backoff cap, dead-letter, periodic dedup pruning)outbound.rs: Fanout builder for locally-homed spaces (qualifies IDs at the boundary)Join handshake & request forwarding
handshake.rs:POST /federation/v1/join— home side validates federation opt-in + bans and returns a snapshot (space, channels, roles, members, recent messages); replica side applies it asorigin = homeforward.rs:/send,/react,/leave,/edit,/delete,/typingso replica servers forward user actions to the authoritative home, which re-runs permissions locallyPeer management
peers.rs: Peer metadata fetching with SSRF guards (https-only, no loopback/private IP literals unlessACCORD_FEDERATION_ALLOW_INSECURE=1)src/db/federation.rs: DB layer for peer state, inbound dedup, and the outbound queuesrc/routes/admin.rs: Admin peer endpoints (add, list, update trust, refresh, delete)Database & models
migrations/027_federation.sql+028_federation_replica.sql(and PostgreSQL variants):federation_peers,federation_inbox_dedup,federation_outboxtables; nullableorigincolumn on users/spaces/channels/roles/messages/members; per-spacefederation_enabledflagUser,MessageRow,Spacegain an optionaloriginfield;db/users.rsanddb/messages.rsget remote upsert/insert helpersIntegration points
routes/messages.rs,routes/reactions.rs,routes/members.rs,routes/invites.rs,routes/spaces.rs: forward writes for remote-homed spaces, fan out writes for locally-homed spacesconfig.rs:FederationConfig(domain, public URL);state.rs:federation: Option<FederationContext>;main.rs: init whenFEDERATION_DOMAINis setroutes/mod.rs: S2S endpoints mounted outside/api/v1(signature-authed, not bearer/rate-limited)Security Model
origin = <peer>— federation can never create or overwrite a local (bare-ID) row.trustedpeers; per-peer token-bucket rate limiting bounds inbound load.Remaining Tasks
Findings from review that should be addressed before this is production-ready:
kick_member,leave_space, andforward::serve_leave,remove_memberruns beforefanout_to_space;interested_serversthen no longer lists the departed user's origin when they were the last member from that server, so their home replica keeps a stale membership. Snapshot interested servers before removal, or always include the affected user's domain.dedup_first_seenmarks(event_id, origin)seen beforeapply_event; a transient failure (e.g. DB busy) returns an error, but the peer's retry is then deduped (200 OK) and the event is lost. Record dedup only after a successful apply, or distinguish transient vs permanent errors.validate_peer_urlonly rejects IP-literal private/loopback hosts; a peer domain resolving to an internal address (e.g. cloud metadata) still passes. Pin a resolver and re-validate the resolved IPs (already flagged as a follow-up inpeers.rs).event_id, but/send,/react,/leave,/edit,/delete, and/typingrely only on the 5-minute Date skew window (i.e. on TLS) to prevent replay. Add per-request nonces / idempotency keys.sender::deliversigns the real inbox URL path while the receiver verifies against the constantINBOX_PATH; only matters for non-root inbox paths, but should be made consistent.m.emoji.create/m.emoji.update/m.emoji.deletefanout. Images are referenced by an absolute home-server CDN URL rather than mirrored (depends on attachments/CDN below). Emoji mutations on a remote-homed (mirrored) space are rejected — they must be performed on the home server.permission_overwritesand post-join role/permission changes do not propagate to replicas).🤖 Generated with Claude Code