Skip to content

kvserver: partial implementation of ReplicasStorage#88606

Closed
sumeerbhola wants to merge 1 commit intocockroachdb:masterfrom
sumeerbhola:rs
Closed

kvserver: partial implementation of ReplicasStorage#88606
sumeerbhola wants to merge 1 commit intocockroachdb:masterfrom
sumeerbhola:rs

Conversation

@sumeerbhola
Copy link
Copy Markdown
Collaborator

The significant missing piece is part of the Init
implementation to handle a RecoveryInconsistentReplica that requires applying committed raft log entries to the state machine. This missing piece will need to wait until #75729 is fixed.

There are multiple TODOs, including related to concurrency, but the implementation is complete enough for the datadriven test to exercise many state transitions. Additionally, the test exercises loss of unsynced state, and fixup of that state in ReplicasStorage.Init, by using vfs.NewStrictMem.

Informs #16624

Release note: None

@sumeerbhola sumeerbhola requested review from nvb and tbg September 23, 2022 19:14
@sumeerbhola sumeerbhola requested review from a team as code owners September 23, 2022 19:14
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@sumeerbhola sumeerbhola force-pushed the rs branch 3 times, most recently from e5557c6 to e2e7c10 Compare September 25, 2022 17:52
@tbg
Copy link
Copy Markdown
Member

tbg commented Sep 26, 2022

cc @cockroachdb/replication

@tbg
Copy link
Copy Markdown
Member

tbg commented Sep 29, 2022

Just chatted with @sumeerbhola - we agreed that the Replication team will push this over the finish line. I'm going to close this PR out, since it's annoying for Sumeer to have the work done on a PR that he opened. The PR can still be checked out (gh pr checkout 888606) after it's closed, and we have tracking issue #88907.

@tbg tbg closed this Sep 29, 2022
@sumeerbhola sumeerbhola reopened this Sep 29, 2022
@sumeerbhola
Copy link
Copy Markdown
Collaborator Author

I have some changes sitting in my local copy that I need to finish and push -- I'm going to do so soon. Is it ok if we delay closing this until someone is actually ready to pick this up -- that allows me to continue pushing changes meanwhile if something occurs to me?

The significant missing piece is part of the Init
implementation to handle a RecoveryInconsistentReplica that
requires applying committed raft log entries to the state
machine. This missing piece will need to wait until cockroachdb#75729
is fixed.

There are multiple TODOs, including related to concurrency,
but the implementation is complete enough for the datadriven
test to exercise many state transitions. Additionally, the
test exercises loss of unsynced state, and fixup of that
state in ReplicasStorage.Init, by using vfs.NewStrictMem.

Informs cockroachdb#16624

Release note: None
@tbg
Copy link
Copy Markdown
Member

tbg commented Sep 30, 2022

Of course! I will hold off on touching this until you confirm that you're ready.

@tbg tbg removed request for a team, nvb and tbg September 30, 2022 06:31
@sumeerbhola
Copy link
Copy Markdown
Collaborator Author

I've pushed my pending changes.

@tbg
Copy link
Copy Markdown
Member

tbg commented Nov 9, 2022

Interesting failure on the PR here, hope this isn't indicative of some storage bug:

sandbox/2316/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/kv/kvserver/kvserver_test_/kvserver_test.runfiles/com_github_cockroachdb_cockroach/pkg/kv/kvserver/datadriven.go:426
            	Error:      	Received unexpected error:
            	            	L6: 000022: file size mismatch (000022.sst): 0 (disk) != 1208 (MANIFEST)
            	            	(1) attached stack trace
            	            	  -- stack trace:
            	            	  | github.com/cockroachdb/pebble/internal/manifest.(*Version).CheckConsistency
            	            	  | 	github.com/cockroachdb/pebble/internal/manifest/external/com_github_cockroachdb_pebble/internal/manifest/version.go:1066
            	            	  | github.com/cockroachdb/pebble.Open
            	            	  | 	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/open.go:258
            	            	  | github.com/cockroachdb/cockroach/pkg/storage.NewPebble
            	            	  | 	github.com/cockroachdb/cockroach/pkg/storage/pebble.go:933
            	            	  | github.com/cockroachdb/cockroach/pkg/storage.Open
            	            	  | 	github.com/cockroachdb/cockroach/pkg/storage/open.go:235
            	            	  | github.com/cockroachdb/cockroach/pkg/kv/kvserver.TestReplicasStorageBasic.func1.3
            	            	  | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replicas_storage_test.go:79
            	            	  | github.com/cockroachdb/cockroach/pkg/kv/kvserver.TestReplicasStorageBasic.func1.4
            	            	  | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replicas_storage_test.go:94
            	            	  | github.com/cockroachdb/cockroach/pkg/kv/kvserver.TestReplicasStorageBasic.func1.5

@tbg
Copy link
Copy Markdown
Member

tbg commented Nov 9, 2022

Closing for #91579, which is a copy of this PR that doesn't spam Sumeer's inbox.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants