Skip to content

storage,compute: rethink reconciliation [PART 1]#13680

Merged
benesch merged 1 commit into
MaterializeInc:mainfrom
benesch:rethink-reconciliation-prefix
Jul 17, 2022
Merged

storage,compute: rethink reconciliation [PART 1]#13680
benesch merged 1 commit into
MaterializeInc:mainfrom
benesch:rethink-reconciliation-prefix

Conversation

@benesch

@benesch benesch commented Jul 17, 2022

Copy link
Copy Markdown
Contributor

This is the prefix of #13526 that is ready for merge.

Motivation

  • This PR refactors existing code.

Testing

  • This PR has adequate test coverage / QA involvement has been duly considered.

Backward compatibility

Release notes

This PR includes the following user-facing behavior changes:

  • n/a

Previously, reconciliation sat on *top* of storage/compute servers, and
attempted to paper over commands and responses so that clients believed
they were talking to a fresh server, while servers always believed they
were talking to the originally-connected client. This introduced a lot
of complicated logic and double bookkeeping.

This commit pushes reconciliation logic into the storaged and computed
servers. The servers are now aware of when a new client connects, and
they simply ditch any state that was related to the old client.

This tees us up for the last reconciliation task: garbage collecting
sources and dataflows that newly-connected clients are not interested
in.

Touches #10549.
Touches #12857.
@benesch benesch requested a review from frankmcsherry July 17, 2022 16:07
@benesch benesch marked this pull request as ready for review July 17, 2022 16:07
@benesch benesch merged commit 5aa4801 into MaterializeInc:main Jul 17, 2022
@benesch benesch deleted the rethink-reconciliation-prefix branch July 17, 2022 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants