bluestore: Elastic Shared Blobs 3 - main part#51441
Merged
rzarzynski merged 34 commits intoceph:mainfrom Sep 13, 2023
Merged
Conversation
After introduction of lazy statfs updates and mechanism to store them at exit, some tests required tune-up. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Fix it, so it can be enabled and work. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Add more checks on consistency. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Upgrade local foreach_shared_blob into _fsck_foreach_shared_blob that can be used on entire BlueStore scope. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
When we do fsck with non-repair mode, we do not get any info about shared blobs that actually were corrupted. Now we print them. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Make faster exit when sharding not enabled. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
When reshard is applied for the first time, expand reshard range to encompas whole object. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
maybe_reshard is created to filter out unnecessary calls to request_reshard. The intended use is to let just request maybe_reshard, and delegate check if the action is really necessary to the implementation detail level. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
When encode_some fails twice ceph aborts. Now we print object details just before. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Modifed bluestore_blob_t to include current size of csum_data. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Add printing of len to operator<< for const bluestore_blob_t. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Modified TransContext. Changed std::set<SharedBlobRef> shared_blobs_written to std::set<BlobRef> blobs_written Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Move finish_write from SharedBlob to Blob. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
This is necessary to enable adding more Buffers to Blobs that are shared. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adapt split_cache to new situation. Now buffers are attached to Blob, and we need always move them, regardless that we already moved relevant SharedBlob. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Fix blobs having the same empty shared blob. Each blob on creation gets its own unique (empty) SharedBlob object. ExtentMap::dup() sometimes merges blobs together, so 2 different blobs get the same SharedBlob object. Function _do_remove() tries to convert shared blobs into regular ones. If it succeeds we could get 2 blobs having the same EMPTY SharedBlob object. The solution is to create detached SharedBlob if necessary. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
With BufferSpace now attached to Blob (was SharedBlob inside it), on-the-fly 'writing' buffers must be copied to clones. Otherwise those objects will read data from disk before it is written there. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
This was referenced May 11, 2023
ifed01
reviewed
May 22, 2023
Contributor
ifed01
left a comment
There was a problem hiding this comment.
I have two major questions regarding this part:
- make_range_shared_maybe_merge implementation looks suspicious to me - see comments inline
- do we really need this "ifdef WITH_ESB" stuff? Wouldn't be having runtime switch between the original and new dup functionality sufficient?
| if (copy_used_in_blob) { | ||
| used_in_blob = from.used_in_blob; | ||
| } else { | ||
| ceph_assert(from.blob.is_compressed()); |
Contributor
There was a problem hiding this comment.
nit: IMO this and the following assertions makes a little sense here. The function has got copy_used_in_blob parameter which defines the desired behavior. Other external factors (e.g. whether blob is compressed or not) which don't prevent the mutation shouldn't be considered at all - we can init used_in_blob in any case so let's do that.
Contributor
Author
There was a problem hiding this comment.
The assertion here is important. It checks against future attempts to reuse Blob::dup() in improper state.
It is true that for calling from dup_esb() the assert does not make sense.
But in future someone could attempt to call it of compressed blobs.
src/os/bluestore/BlueStore.cc
Outdated
| std::multimap<uint64_t /*blob_start*/, Blob*> candidates; | ||
| scan_shared_blobs(c, oldo, srcoff, length, candidates); | ||
|
|
||
| for (auto ep = oldo->extent_map.seek_lextent(srcoff); |
Contributor
There was a problem hiding this comment.
nit: if you get ep_start from scan_shared_blobs you might get rid off seek_lextent call here.
_dup_writing requires locking of BufferCacheShard Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
ifed01
reviewed
Jun 1, 2023
f01ab9e to
dab04d0
Compare
dab04d0 to
29d22e7
Compare
By moving BufferSpace from SharedBlob to Blob tracking of num_blobs get broken. Fixed that and reinforced by adding asserts to BufferCacheShard destructor. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
The reason was historical - it was to give one access method to bc regardles where it was defined SharedBlob or Blob. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Add functions that can copy parts of blobs. It is necessary for merging blobs together, which happens on cloning (ExtentMap::dup). Fixed: Blob::copy_extents_over_empty was faulty when insertion was targetting last extent and that extent was invalid(empty). Add dup() for bluestore_blob_t and bluestore_blob_use_tracker. Changed: Modified Blob::copy_from for better readability. Added bluestore_blob_t::adjust_to initization that conforms to other blob specifics. Move assert for is_mutable() out of bluestore_blob_t::add_tail, so it can be used in blobs that are shared. Modify bluestore_blob_use_tracker_t::get to automatically expand when accessing more AUs then originally declared. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Two new functions: 1) can_marge_blob checks if 2 blobs are compatible to be merged together 2) merge_blob merges 2 blobs into 1, emptying source and putting all to destination Modify merge_blob() to return logical length of produced blob. Clear "unused" bitmap When make blob shared or merge it with other blob, clear unused. It drops some potential optimizations for writing into large blobs, but it is unlikely to be useful. Such info can be useful only after it reverts from Shared to regular blob. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
When we punch_hole in blobs we leave Buffers unchanged. Normally it is not a problem, but when we merge blobs there is a collision. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Introduce function that will scan through relevant range that is to be cloned, and function that will find best matching blob to attach to. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Func reblob_extents is used to modify extents. It is final step of melding 2 blobs together. It removes reference to blob that is to be phased out, and replaces it with reference to the blob that is the sum of them. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adds function that converts a specified range in object to shared blobs, possibly merging them with other shared blobs. Modify make_range_shared_maybe_merge to allow for merging blobs. It now uses can_merge_blobs and merge_blobs. Add discard_unused_buffers() to make_range_shared. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Make ExtentMap::dup able to reuse some already existing shared blobs, when a regular blob has to be transformed to shared blob. Make ExtentMap::dup() use make_range_shared_maybe_merge a primary tool for cloning. Modify ExtentMap::dup to make a diligent merge of blobs. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Only copy Buffers that were not copied before. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
1. Rename ExtentMap::dup to ExtentMap::dup_esb 2. Ressurect ExtentMap::dup Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Having new ExtentMap::dup that introduces heavy changes to blob processing seems risky. We will enable it only on demand. In future, once the feature is tested in production, the choice should be removed (and feature always on). Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Modify _do_clone_range to select variant of ExtentMap::dup depending on elastic_shared_blob. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
29d22e7 to
d01c308
Compare
11 tasks
Contributor
Author
|
jenkins test api |
ifed01
reviewed
Jul 28, 2023
| if (copy_used_in_blob) { | ||
| used_in_blob = from.used_in_blob; | ||
| } else { | ||
| ceph_assert(from.blob.is_compressed()); |
Remove unnecessary arguments from functions. Reduce unneeded indirections for accessing extent_map. Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Contributor
|
jenkins test make check |
ifed01
approved these changes
Aug 10, 2023
14 tasks
Contributor
|
Merged as a part of #53178. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is part 3/4 of ESB work.
It modifies ExtentMap::dup() and adds all sub-functions required to make reuse of shared blobs.
This part can be enabled / disabled in runtime.
#51439
#51440
#51441
#51442
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard cephadmjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume toxjenkins test windows