Skip to content

[BP] MB-66163: always protect the latest snapshot within bolt#2185

Merged
Thejas-bhat merged 1 commit into
7.6.x-couchbasefrom
backportBoltBug
Apr 24, 2025
Merged

[BP] MB-66163: always protect the latest snapshot within bolt#2185
Thejas-bhat merged 1 commit into
7.6.x-couchbasefrom
backportBoltBug

Conversation

@Thejas-bhat

@Thejas-bhat Thejas-bhat commented Apr 24, 2025

Copy link
Copy Markdown
Member

Backporting #2183 for couchbase 7.6.x release cycle builds.

- Currently because of the time-series based checkpointing mechanism,
there can be edge cases where we don't end up protecting the very latest
snapshot that's there in the bolt file.
- This can cause some serious issues along the persister path, where the
newly created file segments that's part of the new snapshot written to
bolt can potentially get cleaned up when the purge code `removeOldData`
is invoked. If there were a backup operation `indexReader.CopyTo()` to
be performed just after this purge operation, the latest root still has
a pointer to the file segment but the underlying file itself would have
been purged because its not part of `ineligibleForRemoval` structure and
also the corresponding bolt entry would have been deleted.
- The issue highlighted has also been unit tested and is consistently
reproducable.
  ```
rollback_test.go:559: error copying the index: error backing up index
snapshot: segment:
/var/folders/5n/g80bkslj5qn6xn8srh6pxv_r0000gp/T/bleve-scorch-test-TestLatestSnapshotCheckpointed/000000000004.zap
copy err: stat
/var/folders/5n/g80bkslj5qn6xn8srh6pxv_r0000gp/T/bleve-scorch-test-TestLatestSnapshotCheckpointed/000000000004.zap:
no such file or directory
  ```
- The PR addresses this issue by firstly preserving the latest snapshot
within bolt and n-1 time-series based checkpoints and also marking the
file as ineligible for removal as an layer of protection.
- There is also a bug fix along a specific merger path where the old
snapshot's ref count wasn't being decremented appropriately which can
potentially cause file leak and memory bloat due to invalid snapshots
still being present.
@Thejas-bhat Thejas-bhat changed the base branch from master to 7.6.x-couchbase April 24, 2025 05:37
@Thejas-bhat Thejas-bhat changed the title [BP] MB-66163: always protect the latest snapshot within bolt (#2183) [BP] MB-66163: always protect the latest snapshot within bolt Apr 24, 2025
@Thejas-bhat Thejas-bhat merged commit 273e2cb into 7.6.x-couchbase Apr 24, 2025
@abhinavdangeti abhinavdangeti deleted the backportBoltBug branch April 24, 2025 13:57
abhinavdangeti pushed a commit that referenced this pull request May 6, 2025
Backporting #2183 for couchbase 7.6.x release cycle builds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants