Skip to content

MB-60971: update the bolt with the correct file segments#2178

Merged
Thejas-bhat merged 2 commits into
masterfrom
boltBug
Apr 16, 2025
Merged

MB-60971: update the bolt with the correct file segments#2178
Thejas-bhat merged 2 commits into
masterfrom
boltBug

Conversation

@Thejas-bhat

@Thejas-bhat Thejas-bhat commented Apr 14, 2025

Copy link
Copy Markdown
Member
  • This is applicable to concurrent persister algorithm (introduced in v2.5.0) - which is not ON by default.
  • In cases where the persister creates new file segments, the bolt file is supposed to be updated with the complete set of file segments.
  • However currently, while loading the index from the bolt source, there can be data loss issues because a set of segments associated with the root snapshot weren't tracked in the bolt file and hence wouldn't be loaded from the disk.
  • Note that the live index doesn't have issues because the root would track the live persisted segments and point to the corresponding files.
  • Also added some better code comments

@abhinavdangeti abhinavdangeti added this to the v2.5.1 milestone Apr 14, 2025
@Thejas-bhat Thejas-bhat changed the title bug fix: update the bolt with the correct segments MB-60971: update the bolt with the correct file segments Apr 14, 2025
@abhinavdangeti

abhinavdangeti commented Apr 14, 2025

Copy link
Copy Markdown
Member

@Thejas-bhat For couchbase it seems if the node were to go down during this situation, KV would ask us to rollback yes?
And I suppose this is related to the concurrent persister model that we just introduced?

@Thejas-bhat

Copy link
Copy Markdown
Member Author

nope i don't think so - this is because bolt separately stores the high seqno. at vBucket level inside the index snapshot bucket of bolt which is tracked correctly. However, the segment snapshot bucket won't be filled with all the file segment's entries. So when the node goes down and comes back up, the seq no.s are correct (so no rollback) however the loadSegment() itself will be applied only to a subset of file segments that should've been there

@abhinavdangeti

abhinavdangeti commented Apr 14, 2025

Copy link
Copy Markdown
Member

I see, and does this come into play because of the concurrent persister model or has it been always around? Do we need to backport this asap?

@Thejas-bhat

Copy link
Copy Markdown
Member Author

i dont think we have to backport it because this is a bug that got introduced with the concurrent persister PR (finally a better name for it) that got merged recently in 2.5.0

abhinavdangeti
abhinavdangeti previously approved these changes Apr 14, 2025
@abhinavdangeti

Copy link
Copy Markdown
Member

Thanks, I've updated the commit description^.

@Thejas-bhat Thejas-bhat merged commit ba1d5a2 into master Apr 16, 2025
@Thejas-bhat Thejas-bhat deleted the boltBug branch April 21, 2025 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants