RTC: Add separate doc persistence endpoint by alecgeatches · Pull Request #78891 · WordPress/gutenberg

alecgeatches · 2026-06-02T19:56:30Z

What?

Add a new wp-sync/v1/save endpoint for persisting a CRDT doc outside of a regular post save.

Why?

@jsnajdr called out two undesirable behaviors with the current CRDT persistence setup in #78799:

When entities are loaded, we may determine the server-side CRDT document is out of sync with editor content during getEntityRecord(), and then trigger a subsequent saveEntityRecord() in order to update the CRDT post meta. This get operation side effect is unexpected, and has additional side effects both in the editor and on the server:

saveEntityRecord, the REST endpoint, and the server-side PHP wp_update_post have the semantics that the user really saved a new version of the post, and modified it in some meaningful way. But saving the CRDT document is just a RTC bookkeeping, the user is not really saving anything. It's triggered even by purely read-only actions.

In wp-data, saveEntityRecord makes the UI think that the post is really saving, and it disables the "Publish" button for a moment, for example.

wp_update_post calls many popular filters and actions that plugins hook into, and the assumption is that we're saving a post.

It also changes a post's modified date, which can be incorrect when post content was not saved and only the CRDT meta was updated. We just want to update a persisted CRDT doc, not all of these other actions to occur.
We send no-op { id } save for non-persisted entities like categories due to a bug:
It seems there is a case where the payload is too small and doesn't even include the CRDT doc. Sometimes when creating a new post I see a request to update a category, where the payload includes only the id:
```
POST /wp/v2/categories/1 { id: 1 }
```
This is triggered by a getEntityRecord( 'taxonomy', 'category' ) read.

How?

When we're just saving CRDT document snapshots, use the new dedicated wp-sync/v1/save REST endpoint and not the saveEntityRecord save flow. Directly updating meta avoids wp_update_post(), post-save hooks, editor save UI state, and post_modified changes for bookkeeping-only CRDT repairs.

The sync config has a new supportsPersistence flag so we don't need to guess at which entities should receive an update. Unsupported entity rooms such as categories are rejected for CRDT doc persistence and also skipped client side.

On the client side, saveCRDTDoc() serializes the current Yjs document via the sync manager and posts it to /wp-sync/v1/save. Saves are queued per room so a slower, older serialized snapshot cannot complete after a newer one and overwrite it.

An important note: User-initiated post saves (e.g. clicking the "Save" button) still include the CRDT doc in the normal post save payload as before. This is explained in the existing prePersistPostType() call:

gutenberg/packages/core-data/src/entities.js

Lines 309 to 313 in cb9c492

    
           // Add meta for the persisted CRDT document during real post saves so the 
        
           // saved post and CRDT snapshot are committed in the same request. We don't 
        
           // want a post save to fail but a CRDT update to succeed or vice versa. 
        
           // CRDT repair uses /wp-sync/v1/save to avoid post-save side effects. 
        
           if ( persistedRecord ) {

Change to `waitForConvergence()` in E2E test

A small change in behavior worth calling out. Before, on every CRDT save (user-facing or side-effect caused), every user would re-fetch the _crdt_document meta locally. This no longer happens when a side-effect CRDT repair hits /wp-sync/v1/save.

The e2e utility function waitForConvergence() was added as part of fuzzer testing in #77666, and previously had an includeCrdtDocument key that was causing a test failure in collaboration-title-reload.spec.ts. The test failure was specifically due to the includeCrdtDocument: true flag only used in this test, and was failing because the page was checking for stale record?.meta?._crdt_document convergence.

This was removed. Previously this worked because a CRDT doc repair generated a "Save" event, which caused all other peers to re-fetch the document from the server. I briefly tried a separate persistedAt flag to make all other users refetch in the background, but after some investigation it's unnecessary.

After the background CRDT repair persists through /wp-sync/v1/save, the server-side document is correct and users have no need to refetch. The _crdt_document meta is otherwise unused in an already active session. The new CRDT doc is still available when a post is loaded from scratch, and explicit post saves still persist the CRDT doc through normal post meta. But once a browser has an active Yjs document, the in-browser source of truth is the Yjs document and editor state, not the local entity record's _crdt_document field. So we don't need to force-refresh _crdt_document as we did before.

Testing Instructions

General testing instructions. This will affect a few processes, so some all-around RTC testing would be helpful.

Enable real-time collaboration in Settings -> Writing.
Open a post in the editor with network tools enabled.
Make an edit and save. Confirm the post save still succeeds, sending a POST to wp/v2/posts/<post_id>.
Open the same post in another browser session and confirm RTC edits still sync.

CRDT repairs

To test a CRDT repair, keep the post open in two browsers and use WPCLI to append some content to the post. Below, replace the ID in $post_id = <ID>; with the actual post ID:

// Replace <ID> with the post ID
npm run wp-env run cli wp eval '$post_id = <ID>; $p = get_post($post_id); wp_update_post(["ID" => $post_id, "post_content" => $p->post_content . "\n\n<!-- wp:paragraph -->\n<p>Updated content</p>\n<!-- /wp:paragraph -->"]);'

Refresh one of the browsers. After loading, you should see a wp-sync/v1/save request and NOT a request to wp/v2/posts/<post_id>. The modified content from DB should be loaded onto the page. The new content should sync between both browsers:

repair-sync.mov
Refresh the other browser and ensure that the "Updated content" block is still included in post content.

Automated coverage

npm run test:unit -- packages/core-data/src/utils/test/save-crdt-doc.js packages/core-data/src/test/resolvers.js packages/sync/src/test/manager.ts
npm run test:unit:php:base -- phpunit/tests/collaboration/wpSyncSaveServer.php phpunit/tests/collaboration/wpSyncConfig.php phpunit/tests/collaboration/restAutosavesController.php

Use of AI Tools

AI assistance: Yes
Tool(s): Codex
Used for: The whole process.

… in normal saves

…the same entity, tests

…for non-supported entities

github-actions · 2026-06-02T20:05:14Z

Size Change: +171 B (0%)

Total Size: 8.44 MB

📦 View Changed

Filename	Size	Change
`build/scripts/block-editor/index.min.js`	380 kB	+4 B (0%)
`build/scripts/core-data/index.min.js`	31.7 kB	+167 B (+0.53%)

_{compressed-size-action}

…groundCRDTSnapshot' which determine whether other clients show a "X user saved" and re-fetch entities

…meta without notifying other users like a user-facing save

…ver round-trip each time the server-side doc is updated as it isn't necessary other than test assertions

…y saved in auto-draft promotion

jsnajdr · 2026-06-03T10:33:01Z

+
+			if ( get_post_meta( $post_id, self::CRDT_DOC_META_KEY, true ) !== $doc ) {
+				$updated = update_post_meta( $post_id, self::CRDT_DOC_META_KEY, $doc );
+				if ( false === $updated ) {


update_post_meta internally does the check whether old value is equal to the new value, we don't need to do it separately.

The return value is false in case of equality. That also means that the false return value can't be treated as a strict error. An is_wp_error check on the return value is probably better.

I didn't know about that behavior without the $prev_value flag! However, it looks like update_post_meta() and the internal update_metadata() function both return int|bool and not a WP_Error. Overall this makes it a bit confusing, as $updated will be false if the document fails to save, but also if the document is the same as already saved. A WP_Error would be much better.

I pushed a change in 3aa0658 that avoids the get_post_meta() call unless we receive false first and then need to differentiate whether the meta change represents an error or a same-value change.

jsnajdr · 2026-06-03T10:34:07Z

+					__( 'You do not have permission to persist this document.', 'gutenberg' ),
+					array( 'status' => rest_authorization_required_code() )
+				);
+			}


This check is already done in permission_callback, handle_request should only do the business logic.

Good catch, fixed in ee8bfeb.

jsnajdr · 2026-06-03T10:48:21Z

+	} );
+
+	return { promise, resolve, reject };
+}


There is a native Promise.withResolvers helper for this.

I fixed this in b2095bf, but ended up reverting it back in cb9c492. I was seeing test failures related to withResolvers():

https://github.com/WordPress/gutenberg/actions/runs/26975840218/job/79602551098?pr=78891

Error: TypeError: Promise.withResolvers is not a function

This passes locally, but it appears we only run CI JS unit tests on node 20, which doesn't have withResolvers().

Oh, there is Node.js 20 🙁 Let's do it some other day then, when we upgrade Node.

jsnajdr · 2026-06-03T10:54:48Z

+async function flushPromises() {
+	for ( let index = 0; index < 5; index++ ) {
+		await Promise.resolve();
+	}


await new Promise( r => setTimeout( 0, r ) ) is a more reliable way to do this flush. It will resolve only after all microtasks have reliably run, no matter what is the count.

Fixed this and the above withResolvers note in b2095bf, thanks!

jsnajdr · 2026-06-03T11:00:18Z

+		 */
+		public function validate_request( WP_REST_Request $request ) {
+			$body = $request->get_body();
+			if ( is_string( $body ) && strlen( $body ) > self::MAX_BODY_SIZE ) {


The MAX_BODY_SIZE is already declared in the args option of register_rest_route. The default validate_callback implementation should therefore do the check, it doesn't need to be implemented separately.

…licitly about testing non-RTC behavior

…in CI

github-actions · 2026-06-04T20:40:24Z

Flaky tests detected in e481c7e.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/27150061323
📝 Reported issues:

[Flaky Test] Text blocks sync modifications between users #78855 in /test/e2e/specs/editor/collaboration/collaboration-block-gauntlet.spec.ts

github-actions · 2026-06-04T21:53:01Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: alecgeatches <alecgeatches@git.wordpress.org>
Co-authored-by: jsnajdr <jsnajdr@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

alecgeatches · 2026-06-04T21:58:54Z

@danluu Just wanted to note a test change made in this PR from the fuzzer-added tests in #77666, see "Change to waitForConvergence() in E2E test" in the PR description.

In short, in addition to checking the live blocks for convergence, we were checking a local _crdt_document meta value. This PR removes CRDT repairs from the normal post save path, so we don't otherwise require all users to refetch that value as before. I don't think checking that local meta adds any value, as we already update the server copy and bundle the CRDT doc from live state on save. Fixing it requires re-adding the "everybody refetches" logic that is unnecessary otherwise.

alecgeatches · 2026-06-04T21:59:37Z

@jsnajdr Thank you for the review notes above! I've addressed them and finalized the PR and description for review. I'll also want to get a couple of other RTC folks in here to do general testing as well.

jsnajdr

Let's ship This will fix many existing issues.

jsnajdr · 2026-06-08T12:38:23Z

+	} );
+
+	return { promise, resolve, reject };
+}


Oh, there is Node.js 20 🙁 Let's do it some other day then, when we upgrade Node.

jsnajdr · 2026-06-08T12:41:49Z

+			return new WP_REST_Response(
+				array(
+					'room' => $room,
+				),


We don't really use the response, do we? It could be left empty.

Also, wrapping in new WP_REST_Response and 200 status is probably redundant: that's the default behavior. In the default case we only need to return the response array.

jsnajdr · 2026-06-08T12:47:29Z

+			$doc = $request['doc'];
+
+			$updated = update_post_meta( $post_id, self::CRDT_DOC_META_KEY, $doc );
+			if ( false === $updated && get_post_meta( $post_id, self::CRDT_DOC_META_KEY, true ) !== $doc ) {


Looking at usages of update_post_meta in the WordPress codebase, the return value is mostly not checked at all 🤷‍♂️ It's probably quite hard for the update to fail. 🙂

Okay, good to know. I left in the defensive check for now, and ideally we rarely or never hit that code path anyway.

alecgeatches · 2026-06-08T16:31:28Z

@jsnajdr Addressed the WP_REST_Response comment above and merged!

alecgeatches added 8 commits June 2, 2026 12:51

Add WP_Sync_Save_Server class for CRDT persistence

a7b52ee

Persist non-user-CRDT-saves through the /save endpoint

02ab971

Add comment explaining why we still persist the CRDT doc through meta…

a8ea8d3

… in normal saves

Update tests for /save endpoint, add new PHPUnit tests

de2e1a1

Use a per-room queue so we can't fire /save requests out of order to …

f075120

…the same entity, tests

Moved shared room parsing logic to WP_Sync_Config

e2027ba

Add tests for WP_Sync_Config parsing and permissions checks

666d23c

Add client-side supportsPersistence config to avoid saving CRDT docs …

9a9f4ea

…for non-supported entities

alecgeatches self-assigned this Jun 2, 2026

alecgeatches added [Type] Task Issues or PRs that have been broken down into an individual action to take [Feature] Real-time Collaboration Phase 3 of the Gutenberg roadmap around real-time collaboration labels Jun 2, 2026

github-actions Bot added [Package] Core data /packages/core-data [Package] Sync labels Jun 2, 2026

alecgeatches added the No Core Sync Required Indicates that any changes do not need to be synced to WordPress Core label Jun 2, 2026

alecgeatches added 3 commits June 2, 2026 15:26

Distinguish sync manager updates by persistenceEvent 'userSave'/'back…

2f2642e

…groundCRDTSnapshot' which determine whether other clients show a "X user saved" and re-fetch entities

Update tests with persistenceEvent

2016a25

Add persistedAt to notify other users to refetch an updated CRDT doc …

ab7162a

…meta without notifying other users like a user-facing save

alecgeatches mentioned this pull request Jun 2, 2026

Sync editor settings in layout effect (fixes autosave e2e) #78799

Open

alecgeatches added 2 commits June 2, 2026 17:17

Remove persistedAt/persistenceEvent. Don't require other users to ser…

2a2eaf6

…ver round-trip each time the server-side doc is updated as it isn't necessary other than test assertions

Add additional RTC-related autosave tests to ensure CRDT docs are onl…

e38b07a

…y saved in auto-draft promotion

jsnajdr reviewed Jun 3, 2026

View reviewed changes

alecgeatches added 7 commits June 4, 2026 10:20

Merge branch 'trunk' into add/rtc-doc-persistence-endpoint

07ff0c1

Ensure collaboration is enabled on all autosave tests that aren't exp…

84ba0c0

…licitly about testing non-RTC behavior

Check update output before calling get_post_meta() for CRDT doc save

3aa0658

Remove double permissions check in sync save server

ee8bfeb

Use conventional promise handling in save-crdt-doc testing

b2095bf

Rely on maxLength schema for document size limits

7e78ee0

Avoid Promise.withResolvers() because it's not available on node v20 …

cb9c492

…in CI

alecgeatches marked this pull request as ready for review June 4, 2026 21:52

alecgeatches requested review from nerrad and spacedmonkey as code owners June 4, 2026 21:52

alecgeatches requested review from maxschmeling and shekharnwagh June 4, 2026 21:59

jsnajdr mentioned this pull request Jun 7, 2026

Multi-entity save flow appearing incorrectly #78989

Open

jsnajdr approved these changes Jun 8, 2026

View reviewed changes

alecgeatches added 3 commits June 8, 2026 09:53

Return an empty payload on successful sync

6751411

Fix array syntax

81b6d46

Merge branch 'trunk' into add/rtc-doc-persistence-endpoint

e481c7e

alecgeatches merged commit 05bf6da into trunk Jun 8, 2026
41 checks passed

alecgeatches deleted the add/rtc-doc-persistence-endpoint branch June 8, 2026 16:31

github-actions Bot added this to the Gutenberg 23.4 milestone Jun 8, 2026

github-actions Bot mentioned this pull request Jun 9, 2026

Add a scheduled Gutenberg upstream TypeScript watch and triage workflow imjlk/wp-typia#283

Closed

ramonjd added [Type] Enhancement A suggestion for improvement. and removed [Type] Task Issues or PRs that have been broken down into an individual action to take labels Jun 10, 2026

	// Add meta for the persisted CRDT document during real post saves so the
	// saved post and CRDT snapshot are committed in the same request. We don't
	// want a post save to fail but a CRDT update to succeed or vice versa.
	// CRDT repair uses /wp-sync/v1/save to avoid post-save side effects.
	if ( persistedRecord ) {

Conversation

alecgeatches commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What?

Why?

How?

Change to waitForConvergence() in E2E test

Testing Instructions

CRDT repairs

Automated coverage

Use of AI Tools

Uh oh!

github-actions Bot commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 4, 2026

Uh oh!

alecgeatches commented Jun 4, 2026

Uh oh!

alecgeatches commented Jun 4, 2026

Uh oh!

jsnajdr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alecgeatches commented Jun 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alecgeatches commented Jun 2, 2026 •

edited

Loading

Change to `waitForConvergence()` in E2E test

github-actions Bot commented Jun 2, 2026 •

edited

Loading

github-actions Bot commented Jun 4, 2026 •

edited

Loading