When testing the Quincy RC for 17.2.4, we discovered this failure:

Description: rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}

Dead job: http://pulpito.front.sepia.ceph.com/yuriw-2022-09-10_14:05:53-rados-quincy-release-distro-default-smithi/7024541/

The failure is intermittent, which explains why it did not show up in #47901 (comment), nor in the rerun of @yuriw's validation run.

We found that running the affected test 50 times is a reliable reproducer, as seen by:
http://pulpito.front.sepia.ceph.com/lflores-2022-09-14_15:11:39-rados-quincy-release-distro-default-smithi/

We suspected #47901 to be the culprit, so we reran the reproducer with those two commits reverted:
http://pulpito.front.sepia.ceph.com/lflores-2022-09-14_17:02:02-rados-wip-lflores-testing-quincy-release-distro-default-smithi/

All 50 tests passed with the commits from #47901 reverted.

So far, this bug has only been observed in Quincy (and once on Pacific, but the affected test branch included #47899, which hasn't been merged yet). It's possible that this bug exists in main since #44499 has merged.

Fixes: https://tracker.ceph.com/issues/57546
Signed-off-by: Laura Flores lflores@redhat.com

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests