Ceph » bluestore

Code:
https://github.com/ceph/ceph/compare/main...aclamk:ceph:wip-aclamk-more-deferred-overwrite-fix-redo-with-debug

Scheduled teuthology:
teuthology-suite -s crimson-rados -c wip-aclamk-more-deferred-overwrite-fix-redo-crimson-only -f crimson --filter thrashers/default ...

Result:
https://pulpito.ceph.com/akupczyk-2025-03-19_18:25:32-crimson-rados-wip-aclamk-more-deferred-overwrite-fix-redo-crimson-only-distro-crimson-smithi/

DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay replay 0x0000000000000029
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _eliminate_outdated_deferred input extents: [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _eliminate_outdated_deferred output extents: [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay 0x1000 writing to [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay csum=4d0eaae2
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay replay 0x000000000000002A
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _eliminate_outdated_deferred input extents: [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _eliminate_outdated_deferred output extents: [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay 0x1000 writing to [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay csum=1d9581b9
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay replay 0x000000000000002B
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _eliminate_outdated_deferred input extents: [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _eliminate_outdated_deferred output extents: [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay 0x1000 writing to [0x432000~1000]
DEBUG 2025-03-19 18:51:55,965 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay csum=d1474adf
...
DEBUG 2025-03-19 18:51:55,966 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replaysubmitting IO batch
DEBUG 2025-03-19 18:51:55,966 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replaywait done
DEBUG 2025-03-19 18:51:55,966 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _deferred_replay completed 3 events
...
DEBUG 2025-03-19 18:51:56,094 [shard 0:main] bluestore - bluestore(/var/lib/ceph/osd/ceph-2) _generate_read_result_bl  blob Blob(0x50c000276040 blob([0x432000~1000] llen=0x1000 csum crc32c/0x1000/4) use_tracker(0x1000 0x21b) (shared_blob=NULL)) need {<0x0, 0x1000> : [0x0:0~21b]}
ERROR 2025-03-19 18:51:56,094 [shard 0:main] none - bluestore(/var/lib/ceph/osd/ceph-2) _verify_csum bad crc32c/0x1000 checksum at blob offset 0x0, got 0x4d0eaae2, expected 0xd1474adf, device location [0x432000~1000], logical extent 0x0~1000, object #-1:7b3f43c4:::osd_superblock:0#

The problem here is that we put into one AIO same region multiple times.
Should the last update will stick, everything will be good, but it is the first one that gets chosen.
This case of replaying deferred writes will obviously be fixed.

But the main question is:
If we have multiple deferred updates to the same object, is it possible for them to be put to single AIO?