Skip to content

os/bluestore: Disable compaction then no-column-b is storing allocations to bluefs file#43645

Merged
liewegas merged 1 commit intoceph:masterfrom
aclamk:wip-compaction-and-allocation-fix-52399
Oct 28, 2021
Merged

os/bluestore: Disable compaction then no-column-b is storing allocations to bluefs file#43645
liewegas merged 1 commit intoceph:masterfrom
aclamk:wip-compaction-and-allocation-fix-52399

Conversation

@aclamk
Copy link
Contributor

@aclamk aclamk commented Oct 25, 2021

During BlueStore umount we store current allocation state to disk, in form of bluefs file.
If RocksDB was performing compaction during capture of allocator state, it could cause corruption.
Solution is to delete db (stop RocksDB) before state capture.

Fixes: https://tracker.ceph.com/issues/52399
Signed-off-by: Adam Kupczyk akupczyk@redhat.com

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

…ons to bluefs file

During BlueStore umount we store current allocation state to disk, in form of bluefs file.
If RocksDB was performing compaction during capture of allocator state, it could cause corruption.
Solution is to delete db (stop RocksDB) before state capture.

Fixes: https://tracker.ceph.com/issues/52399
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
@aclamk
Copy link
Contributor Author

aclamk commented Oct 25, 2021

https://gist.github.com/aclamk/cadc3561da8771901534aae4f6d20bde describes how compaction leads to HybridAllocator assert.

Copy link
Contributor

@ifed01 ifed01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

Copy link
Contributor

@benhanokh benhanokh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • [ ]

@liewegas
Copy link
Member

looks good! https://pulpito.ceph.com/sage-2021-10-27_19:00:50-rados:objectstore-wip-sage3-testing-2021-10-27-1300-distro-basic-smithi/

@neha-ojha
Copy link
Member

Let's run it through a broader rados run as well

@liewegas liewegas merged commit c9e647a into ceph:master Oct 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants