-
-
Notifications
You must be signed in to change notification settings - Fork 96
Description
Problem
ActivityPub activities can be delivered out of order when they relate to the same object, causing federation issues. Specifically, when a post is created and quickly deleted, the Delete activity can arrive at remote instances before the Create activity, resulting in “zombie posts” that should have been deleted.
This was observed in production: hackers-pub/hackerspub#162 (comment).
Scenario
- User creates a post →
Create(Note)activity is enqueued - User immediately deletes the post →
Delete(Note)activity is enqueued - Both activities are sent to remote instances independently
- Due to parallel workers and lack of ordering guarantees,
Delete(Note)may arrive first - Remote instance ignores the delete (no post exists yet)
- Remote instance later receives and persists
Create(Note) - Result: “zombie post” exists on remote instance but not on origin
Root cause
The MessageQueue interface doesn't provide:
- Ordering guarantees — messages for the same object can be processed out of order
- Cancellation mechanism — can't cancel a queued
Createwhen aDeleteis enqueued shortly after - Dependency tracking — no way to express that one activity depends on or supersedes another
While PostgresMessageQueue uses ORDER BY created, parallel workers can still process messages out of order. This issue affects all MessageQueue implementations (Postgres, Redis, SQLite, AMQP, Deno KV, etc.).
Proposed solutions
Option A: Add message cancellation API
interface MessageQueue {
enqueue(message: any, options?: MessageQueueEnqueueOptions & { key?: string }): Promise<void>;
cancel(key: string): Promise<boolean>; // Returns true if cancelled
}Users could assign keys to messages and cancel pending ones:
await queue.enqueue(createActivity, { key: `create:${noteId}` });
// Later, if deleted quickly:
if (await queue.cancel(`create:${noteId}`)) {
// Create was cancelled, no need to send Delete
} else {
// Create already sent, send Delete
await queue.enqueue(deleteActivity);
}Pros: Flexible, allows application-level logic
Cons: Requires checking if cancellation succeeded; complex to implement in distributed queues
Option B: Per-object ordering
interface MessageQueueEnqueueOptions {
delay?: Temporal.Duration;
orderingKey?: string; // Messages with same key are processed in order
}Activities for the same object would be guaranteed to arrive in order:
await queue.enqueue(createActivity, { orderingKey: noteId });
await queue.enqueue(deleteActivity, { orderingKey: noteId });
// Delete always arrives after CreatePros: Matches ActivityPub's causal ordering semantics; works well with existing queue backends
Cons: May reduce parallelism; requires implementation changes in all queue backends
Option C: Activity batching
Allow grouping related activities so they're sent atomically or cancelled together:
interface MessageQueue {
enqueueGroup(messages: any[], options?: { atomicCancel?: boolean }): Promise<string>;
cancelGroup(groupId: string): Promise<boolean>;
}Pros: Handles complex multi-message scenarios
Cons: Most complex to implement; may not fit all use cases; batching semantics unclear for ActivityPub
Additional context
This is a framework-level issue that affects any Fedify application with rapid Create/Delete operations. A proper fix would benefit the entire ecosystem.