Plans for the next major version

So in the last major RxDB versions, the focus was set to improvements of the storage engine. This is done. RxDB has now multiple RxStorage implementations, a better query planner and an improved test suite to ensure everything works correct.
This let to huge improvements in write and query performance and decreased the initial pageload of RxDB based applications.

In the next major version, I want to focus on the replication protocol.
When I first implemented the GraphQL replication a few years ago, I had a specific use case in mind and designed the whole protocol and replication plugins around that use case.

But the time has shown, that the current replication protocol is a big downside of RxDB:

• The replication relies on the backend to solve all conflicts. This was easy to implement into RxDB because the whole responsibility was given away to the person that has to implement a compatible backend.
• In each point in time, the replication does either push or pull documents, but never in parallel. This slows done the whole replication process and makes RxDB not usable for the implementation of features like multi-user-real-time-collaboration or when many read- and write operations have to happen in a short timespan.
• After each push, a pull has to be run to check if the backend had changed the state to solve a conflict.
• The replication protocol does not support attachments.

So I want to replace the whole replication plugins with a new replication protocol. The main goals are:

• Push- and Pull in parallel.
• Use the data in the changestream (optional) to decrease replication latency.
• Implement the conflict resolution into RxDB so that the client resolves it's own conflicts and does not rely on the backend.
• Decrease the complexity for a compatible backend implementation. The new protocol relies on a dumb backend. This will open compatibility with many other use cases like implementing Offline-First in Supabase or using CouchDB but having a faster replication compared to the native CouchDB replication.
• Make it possible to use CRDTs instead of a conflict resolution.

How will the new protocol work?

On the RxDocument level, the replication works like git, where the fork contains all new writes and must be merged with the master before it can push its new state to the master.

• The client pulls the latest state from the master.
• The client does some changes
• The client pushes these changes to the master by sending the latest known master state and the new client state of the document.
• If the master state is equal to the latest master state of the client, the new client state is set as the latest master state.
• If the master also had changes and so the latest master change is different then the one that the client assumes, a conflict arises that is solved by RxDB. The resolved conflict causes a new write to the document which will trigger the push again.

This works great with the already existing RxDB revision handling.
A beta of the whole protocol is already implemented and used in the Memory Synced RxStorage. You can read the code of the replication here.

Will an old RxDB compatible backend work with the new protocol?

No. It will not work out of the box. But there will be ways to easily make it compatible and it will still be possible to resolve conflicts on the server side.

Please discuss.

Other planned changes