Fix(cleanup): Avoid truncating in value.Open on error#1465
Merged
jarifibrahim merged 6 commits intomasterfrom Aug 27, 2020
Merged
Fix(cleanup): Avoid truncating in value.Open on error#1465jarifibrahim merged 6 commits intomasterfrom
jarifibrahim merged 6 commits intomasterfrom
Conversation
The vlog.Open() function can return an error denoting that the open was unsuccessful but we have `db.cleanup` which will be called to stop all the running go routines in case badger couldn't start. The db.cleanup function calls vlog.Close() which will truncate the maxFid vlog file based on the vlog.writableLogOffset. The vlog.writableLogOffset was not being updated in case open failed. As a result of this, we will end up truncating the vlog file to lenght 0 if open fails. This PR fixes this by setting writableLogOffset to endOffset of the replay so that we don't truncate the entire file.
jarifibrahim
commented
Aug 18, 2020
Contributor
Author
jarifibrahim
left a comment
There was a problem hiding this comment.
Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @ashish-goswami and @manishrjain)
value.go, line 1088 at r1 (raw file):
// TODO(ibrahim): Do we really want this? The DB might not open if we end // up in this situation. y.AssertTruef(endOffset < uint32(sz), "endoffset: %d should be more than sz:%d", endOffset, sz)
I am not sure if we should keep this. An error here would be unrecoverable (unless we manually truncate the file to the given offset which means we create a hole in the file).
ashish-goswami
approved these changes
Aug 27, 2020
Contributor
ashish-goswami
left a comment
There was a problem hiding this comment.
Reviewed 1 of 2 files at r2, 1 of 1 files at r3.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @manishrjain)
jarifibrahim
pushed a commit
that referenced
this pull request
Aug 27, 2020
* Fix(cleanup): Avoid truncating in value.Open on error The vlog.Open() function can return an error denoting that the open was unsuccessful but we have `db.cleanup` which will be called to stop all the running go routines in case badger couldn't start. The db.cleanup function calls vlog.Close() which will truncate the maxFid vlog file based on the vlog.writableLogOffset. The vlog.writableLogOffset was not being updated in case open failed. As a result of this, we will end up truncating the vlog file to lenght 0 if open fails. This PR fixes this by using vlog.stopDiscardStatFlush() instead of vlog.close. (cherry picked from commit ed3b219)
jarifibrahim
pushed a commit
that referenced
this pull request
Aug 27, 2020
The vlog.Open() function can return an error denoting that the open was unsuccessful but we have `db.cleanup` which will be called to stop all the running goroutines in case badger couldn't start. The db.cleanup function calls vlog.Close() which will truncate the maxFid vlog file based on the vlog.writableLogOffset. The vlog.writableLogOffset was not being updated in case open failed. As a result of this, we will end up truncating the vlog file to length 0 if open fails. This PR fixes this by using vlog.stopDiscardStatFlush() instead of vlog.close. (cherry picked from commit ed3b219)
jarifibrahim
pushed a commit
that referenced
this pull request
Oct 2, 2020
* Fix(cleanup): Avoid truncating in value.Open on error The vlog.Open() function can return an error denoting that the open was unsuccessful but we have `db.cleanup` which will be called to stop all the running go routines in case badger couldn't start. The db.cleanup function calls vlog.Close() which will truncate the maxFid vlog file based on the vlog.writableLogOffset. The vlog.writableLogOffset was not being updated in case open failed. As a result of this, we will end up truncating the vlog file to lenght 0 if open fails. This PR fixes this by using vlog.stopDiscardStatFlush() instead of vlog.close.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The vlog.Open() function can return an error denoting that the open was
unsuccessful but we have
db.cleanupwhich will be called to stop allthe running goroutines in case badger couldn't start. The db.cleanup
function calls vlog.Close() which will truncate the maxFid vlog file
based on the vlog.writableLogOffset. The vlog.writableLogOffset was not
being updated in case open failed. As a result of this, we will end up
truncating the vlog file to length 0 if open fails.
This PR fixes this by using
vlog.stopDiscardStatFlush()instead ofvlog.close.This change is