Skip to content

kvserver: partial implementation of ReplicasStorage#91579

Closed
tbg wants to merge 3 commits intocockroachdb:masterfrom
tbg:rs
Closed

kvserver: partial implementation of ReplicasStorage#91579
tbg wants to merge 3 commits intocockroachdb:masterfrom
tbg:rs

Conversation

@tbg
Copy link
Copy Markdown
Member

@tbg tbg commented Nov 9, 2022

WIP

sumeerbhola and others added 2 commits November 4, 2022 15:25
The significant missing piece is part of the Init
implementation to handle a RecoveryInconsistentReplica that
requires applying committed raft log entries to the state
machine. This missing piece will need to wait until cockroachdb#75729
is fixed.

There are multiple TODOs, including related to concurrency,
but the implementation is complete enough for the datadriven
test to exercise many state transitions. Additionally, the
test exercises loss of unsynced state, and fixup of that
state in ReplicasStorage.Init, by using vfs.NewStrictMem.

Informs cockroachdb#16624

Release note: None
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @tbg)


pkg/kv/kvserver/replicas_storage.go line 693 at r2 (raw file):

	// contain the term. RangeStorage must simply remove all entries > Hi;
	// additionally it's too late to SingleDelete any existing entries since
	// that write would now apply to the new entries, not the old ones.

This is used for purging the sideloaded files, which include the term in the key, similar to what happens in

if len(rd.Entries) > 0 {
// We may have just overwritten parts of the log which contain
// sideloaded SSTables from a previous term (and perhaps discarded some
// entries that we didn't overwrite). Remove any such leftover on-disk
// payloads (we can do that now because we've committed the deletion
// just above).
firstPurge := rd.Entries[0].Index // first new entry written
purgeTerm := rd.Entries[0].Term - 1
lastPurge := prevLastIndex // old end of the log, include in deletion
purgedSize, err := maybePurgeSideloaded(ctx, r.raftMu.sideloaded, firstPurge, lastPurge, purgeTerm)

Maybe it would be sufficient for this to be the term of Lo?

@tbg tbg closed this Dec 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants