Skip to content

Collaboration: Replace post meta storage with dedicated database table#11256

Open
josephfusco wants to merge 46 commits intoWordPress:trunkfrom
josephfusco:collaboration/single-table
Open

Collaboration: Replace post meta storage with dedicated database table#11256
josephfusco wants to merge 46 commits intoWordPress:trunkfrom
josephfusco:collaboration/single-table

Conversation

@josephfusco
Copy link
Copy Markdown

@josephfusco josephfusco commented Mar 14, 2026

The real-time collaboration sync layer currently stores messages as post meta, which creates side effects at scale. This moves it to a single dedicated wp_collaboration table purpose-built for the workload.

Table Definition

CREATE TABLE wp_collaboration (
  id bigint(20) unsigned NOT NULL auto_increment,
  room varchar(191) NOT NULL default '',
  type varchar(32) NOT NULL default '',
  client_id varchar(32) NOT NULL default '',
  user_id bigint(20) unsigned NOT NULL default '0',
  data longtext NOT NULL,
  date_gmt datetime NOT NULL default '0000-00-00 00:00:00',
  PRIMARY KEY  (id),
  KEY type_client_id (type, client_id),
  KEY room (room, id),
  KEY date_gmt (date_gmt)
);

Testing

npm run env:cli -- core update-db
npm run test:php -- --filter WP_Test_REST_Collaboration_Server
npm run test:e2e -- tests/e2e/specs/collaboration/

References

Trac ticket: https://core.trac.wordpress.org/ticket/64696
PR with prior work and feedback (2 table approach): #11068

Use of AI Tools

Co-authored with Claude Code (Opus 4.6), used to synthesize discussion across related tickets and PRs into a single implementation. All code was reviewed and tested before submission.


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

@josephfusco josephfusco changed the title Collaboration: Replace post meta storage with dedicated database table - wp_collaboration Collaboration: Replace post meta storage with dedicated database table Mar 14, 2026
@josephfusco josephfusco changed the title Collaboration: Replace post meta storage with dedicated database table Collaboration: Replace post meta storage with dedicated database table Mar 14, 2026
@github-actions
Copy link
Copy Markdown

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

@josephfusco josephfusco force-pushed the collaboration/single-table branch from c08b703 to 693c813 Compare March 16, 2026 16:10
Introduces the wp_collaboration table for storing real-time editing
data (document states, awareness info, undo history) and the
WP_Collaboration_Table_Storage class that implements all CRUD
operations against it. Bumps the database schema version to 61840.
Replaces WP_HTTP_Polling_Sync_Server with
WP_HTTP_Polling_Collaboration_Server using the wp-collaboration/v1
REST namespace. Switches to string-based client IDs, fixes the
compaction race condition, adds a backward-compatible wp-sync/v1
route alias, and uses UPDATE-then-INSERT for awareness data.
Deletes WP_Sync_Post_Meta_Storage and WP_Sync_Storage interface,
and removes the wp_sync_storage post type registration from post.php.
These are superseded by the dedicated collaboration table.
Adds wp_is_collaboration_enabled() gate, injects the collaboration
setting into the block editor, registers cron event for cleaning up
stale collaboration data, and updates require/include paths for the
new storage and server classes.
Adds 67 PHPUnit tests for WP_HTTP_Polling_Collaboration_Server covering
document sync, awareness, undo/redo, compaction, permissions, cursor
mechanics, race conditions, cron cleanup, and the backward-compatible
wp-sync/v1 route. Adds E2E tests for 3-user presence, sync, and
undo/redo. Removes the old sync server tests. Updates REST schema
setup and fixtures for the new collaboration endpoints.
@josephfusco josephfusco force-pushed the collaboration/single-table branch from 87fc57a to 886f0b1 Compare March 16, 2026 17:03
Adds a cache-first read path to get_awareness_state() following the
transient pattern: check the persistent object cache, fall back to
the database on miss, and prime the cache with the result.

set_awareness_state() updates the cached entries in-place after the
DB write rather than invalidating, so the cache stays warm for the
next reader in the room. This is application-level deduplication:
the shared collaboration table cannot carry a UNIQUE KEY on
(room, client_id) because sync rows need multiple entries per
room+client pair.

Sites without a persistent cache see no behavior change — the
in-memory WP_Object_Cache provides no cross-request benefit but
keeps the code path identical.
Restore the `wp_client_side_media_processing_enabled` filter and the
`finalize` route that were accidentally removed from the REST schema
test. Add the `collaboration` table to the list of tables expected to
be empty after multisite site creation.
The connectors API key entries in wp-api-generated.js were
incorrectly carried over during the trunk merge. Trunk does not
include them in the generated fixtures since the settings are
dynamically registered and not present in the CI test context.
@josephfusco josephfusco force-pushed the collaboration/single-table branch from 5140e44 to 09d0b86 Compare March 16, 2026 19:52
@josephfusco josephfusco marked this pull request as ready for review March 16, 2026 20:11
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 16, 2026

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props joefusco, peterwilsoncc, czarate, paulkevan.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Rename the `update_value` column to `data` in the collaboration table
storage class and tests, and fix array arrow alignment to satisfy PHPCS.

The shorter name is consistent with WordPress meta tables and avoids
confusion with the `update_value()` method in `WP_REST_Meta_Fields`.
Add a composite index on (type, client_id) to the collaboration table
to speed up awareness upserts, which filter on both columns.

Bump $wp_db_version from 61840 to 61841 so existing installations
pick up the schema change via dbDelta on upgrade.
@josephfusco josephfusco force-pushed the collaboration/single-table branch from 1a44948 to d4e27d4 Compare March 17, 2026 02:10
@josephfusco
Copy link
Copy Markdown
Author

josephfusco commented Mar 17, 2026

Carrying over props from original PR:

Props joefusco, peterwilsoncc, mindctrl, westonruter, paulkevan, dd32, czarate.

Introduce MAX_BODY_SIZE (16 MB), MAX_ROOMS_PER_REQUEST (50), and
MAX_UPDATE_DATA_SIZE (1 MB) constants to cap request payloads.

Wire a validate_callback on the route to reject oversized request
bodies with a 413, add maxItems to the rooms schema, and replace
the hardcoded maxLength with the new constant.
Reject non-numeric object IDs early in
can_user_collaborate_on_entity_type(). Verify that a post's actual
type matches the room's claimed entity name before granting access.

For taxonomy rooms, confirm the term exists in the specified taxonomy
and simplify the capability check to use assign_term with the
term's object ID.
Cover oversized request body (413), exceeding max rooms (400),
non-numeric object ID, post type mismatch, nonexistent taxonomy
term, and term in the wrong taxonomy.
Copy link
Copy Markdown
Contributor

@peterwilsoncc peterwilsoncc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few notes from my first pass are inline. There will probably be more passes as I continuing reviewing the code and testing the functionality.

/* 
 * For multi-line comments the WordPress Coding Standard
 * is to write them like this rather than with consecutive lines
 * beginning with a `//`.
 *
 * This applies in a few places so I haven't dropped an inline comment.
 */

…rage

Convert consecutive single-line comments to block comment style per
WordPress coding standards, replace forward slashes with colons in
cache keys to avoid ambiguity, hoist `global $wpdb` above the cache
check in `get_awareness_state()`, and clarify the `$cursor` param
docblock in `remove_updates_before_cursor()`.
When collaboration is disabled, run both DELETE queries (sync and
awareness rows) before unscheduling the cron hook so leftover data
is removed. Hoist `global $wpdb` to the top of the function so the
disabled branch can use it. Add a comment noting future persistent
types may also need exclusion from the sync cleanup query.
Copy link
Copy Markdown
Contributor

@peterwilsoncc peterwilsoncc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in a few more notes.

josephfusco and others added 4 commits March 19, 2026 07:33
Co-authored-by: Peter Wilson <519727+peterwilsoncc@users.noreply.github.com>
…rage.php

Co-authored-by: Peter Wilson <519727+peterwilsoncc@users.noreply.github.com>
Co-authored-by: Peter Wilson <519727+peterwilsoncc@users.noreply.github.com>
@josephfusco josephfusco moved this from 🆕 Backlog to Community In Review in Headless OSS Mar 23, 2026
Comment on lines +192 to +195
room varchar($max_index_length) NOT NULL default '',
type varchar(32) NOT NULL default '',
client_id varchar(32) NOT NULL default '',
user_id bigint(20) unsigned NOT NULL default '0',
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we pass in an empty room, type, client_id, and/or user_id via add_update or similar, what do we expect to happen? How will that be used? What use cases do we have for any/all of these having empty values? Should we make these have some kind of real value and return errors in the helper functions if not provided?

'type' => array( 'object', 'null' ),
),
'client_id' => array(
'minimum' => 1,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar question here, is there a reason to not require client_id be a valid non-empty value? You removed the min check and just curious why.

$is_compactor = false;
if ( count( $merged_awareness ) > 0 ) {
$is_compactor = min( array_keys( $merged_awareness ) ) === $client_id;
$is_compactor = (string) min( array_keys( $merged_awareness ) ) === $client_id;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (string) coercion (only on the left-side of the comparison) prevents compaction from taking place. $is_compactor is always false.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a unit test to assert compaction.

chriszarate added a commit to WordPress/gutenberg that referenced this pull request Mar 30, 2026
Backport of WordPress/wordpress-develop#11256.

Replaces WP_Sync_Post_Meta_Storage / WP_Sync_Storage / WP_HTTP_Polling_Sync_Server
with WP_Collaboration_Table_Storage / WP_HTTP_Polling_Collaboration_Server backed
by a dedicated `wp_collaboration` table.

Key changes:
- New `wp_collaboration` table created via dbDelta in lib/upgrade.php
- Table creation also exposed as `gutenberg_create_collaboration_table` action
  hook for WP-CLI usage
- Storage uses per-client awareness rows (eliminates race condition)
- Awareness reads served from persistent object cache with DB fallback
- REST namespace changed to wp-collaboration/v1 with wp-sync/v1 alias
- Payload limits: 16 MB body, 50 rooms/request, 1 MB per update
- Permission hardening: post type mismatch check, non-numeric ID rejection
- Compaction insert-before-delete to close new-client race window
- Cron cleanup for stale data (daily, 7-day sync / 60-second awareness)
chriszarate added a commit to WordPress/gutenberg that referenced this pull request Mar 30, 2026
Backport of WordPress/wordpress-develop#11256.

Replaces WP_Sync_Post_Meta_Storage / WP_Sync_Storage / WP_HTTP_Polling_Sync_Server
with WP_Collaboration_Table_Storage / WP_HTTP_Polling_Collaboration_Server backed
by a dedicated `wp_collaboration` table.

Key changes:
- New `wp_collaboration` table created via dbDelta in lib/upgrade.php
- Table creation also exposed as `gutenberg_create_collaboration_table` action
  hook for WP-CLI usage
- Storage uses per-client awareness rows (eliminates race condition)
- Awareness reads served from persistent object cache with DB fallback
- REST namespace changed to wp-collaboration/v1 with wp-sync/v1 alias
- Payload limits: 16 MB body, 50 rooms/request, 1 MB per update
- Permission hardening: post type mismatch check, non-numeric ID rejection
- Compaction insert-before-delete to close new-client race window
- Cron cleanup for stale data (daily, 7-day sync / 60-second awareness)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants