mon: Add force-remove-snap mon command by Matan-B · Pull Request #53545 · ceph/ceph

Matan-B · 2023-09-20T10:51:14Z

    /*
     *  Forces removal of snapshots in the range of
     *  [lower_snapid_bound, upper_snapid_bound) on pool <pool>
     *  in order to cause OSDs to re-trim them.
     *  The command has two mutually exclusive variants:
     *  * Default: All the snapids in the given range which are not
     *    marked as purged in the Monitor will be removed. Mostly useful
     *    for cases in which the snapid is leaked in the client side.
     *    See: https://tracker.ceph.com/issues/64646
     *  * (Experimental) purged-snaps-only: Adding this flag will result
     *    in the reremoval of snapids in the given range.
     *    Only the snapids which are *already* marked as purged in the
     *    Monitor will be removed again. This may be useful for cases in
     *    which we would like to trigger OSD snap trimming again.
     */

Variant 1 (Default):

Remove all snapids in the range which are not marked as purged.

Useful for leaks such as: https://tracker.ceph.com/issues/64646

Example usage:

POOL_NAME         USED  OBJECTS  CLONES  COPIES
unique_pool_0    8 KiB        2       1       2

$ rados lssnap -p unique_pool_0
0 snaps

$ ceph osd pool force-remove-snap unique_pool_0 
force reremoving snap ids in the range of [1,2) from pool 2. reremoved snapids: 1

POOL_NAME         USED  OBJECTS  CLONES  COPIES
unique_pool_0    4 KiB        1       0       1

Case 2 - purged-snaps-only:

Reremove snap ids in the range which are already marked as purged.

Notes:

`scrub_purged_snaps` can also be used to cause the OSD to re-trim
purged snapshots. However, those will only get re-trimmed if they were
marked as purged (PSN_ keys) in the OSD store.
Using the command introduced here, the snapshots that were marked as
purged (purged_snaps_ keys) in the mon's store will be also marked,
correspondingly, in the OSD store.
That way `scrub_purged_snaps` will be able to re-trim the snapshots that weren't
marked as purged in the OSD side (for some reason).

The re-removed snapshots will be inserted to new_purged_snaps which
will be used when handling the new MOSDMap. These new_purged_snaps
are passed to SnapMapper::record_purged_snaps to be added to the OSD store.

Initial testing looks to be stable:

2023-09-19T14:33:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,16) from pool 3. reremoved snapids: 2,3,6,8,14
2023-09-19T14:36:28.131 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,206) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,189,192,193,195,196,197,199,200,201,205
2023-09-19T14:36:42.319 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,220) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,189,192,193,195,196,197,199,200,201,205,206,207,208,212,213,215,218,219
2023-09-19T14:37:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,234) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,157,189,190,218,219,220,221,222,225,226,229,232,233
2023-09-19T14:42:13.540 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,443) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,157,189,190,218,219,220,221,222,223,224,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,283,284,285,286,287,347,348,349,350,383,384,385,386,387,388,389,394,395,396,426,427,428,429,432,433,434,438,441,442
2023-09-19T14:44:49.541 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,563) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,157,189,190,218,219,220,221,222,223,224,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,283,284,285,286,287,347,348,349,350,383,384,385,386,387,388,389,394,395,396,426,427,428,429,492,493,494,495,496,497,498,499,526,527,528,529,530,536,537,538,539,543,552,554,561

https://pulpito.ceph.com/matan-2023-09-19_13:58:13-rados:thrash-wip-matanb-reremove-snap-only-distro-default-smithi/7398191/

Original PR : #53235

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

github-actions · 2023-10-11T17:36:16Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

Matan-B · 2023-10-31T10:45:41Z

https://shaman.ceph.com/builds/ceph/wip-matanb-reremove-snap-only/99d7d638529433a1fc99f9d1e6faf14203d6940b/

Matan-B · 2023-11-06T13:51:41Z

https://pulpito.ceph.com/matan-2023-11-06_13:51:11-rados:thrash-wip-matanb-reremove-snap-only-distro-default-smithi/

github-actions · 2023-12-18T15:06:34Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

github-actions · 2024-02-16T16:01:53Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

src/mon/OSDMonitor.cc

rzarzynski · 2024-03-05T12:25:00Z

Would be great to have the teuthology coverage for the new command.

Matan-B · 2024-03-05T14:54:59Z

Would be great to have the teuthology coverage for the new command.

I added teuthology coverage in the second commit: 57da5c5

athanatos

One substantive question about the --purged-snaps-only flag implementation along with some clarification requests.

src/mon/OSDMonitor.cc

athanatos · 2024-03-05T23:23:47Z

src/mon/OSDMonitor.cc

+      if (res == 0) {
+        ss << "snapids: " << i << "was already marked as purged";
+        // Reremove the already purged_snaps
+        if (purged_snaps_only) {


I don't understand this condition. In this branch, the snap in question has already been purged. Seems like that means we should re-remove whether or not --purged-snaps-only was passed?

When using the purged_snaps_only variant, only the snaps which were already marked as purged are removed again. I have added an explanatory comment to emphasize this.

The purged-snaps-only is considered "Experimental" (and not default) because of the Kludge mentioned with this variant later on. Adding the snap id both to new_purged_snaps and to new_removed_snaps is not conventional and may cause issues which are less obvious to identify easily.
Where the "Default" variant is imitating the normal behavior of removing snap ids and is less prone to unexpected results.

athanatos · 2024-03-05T23:25:04Z

src/mon/OSDMonitor.cc

+          force_removed_snapids.insert(i);
+        }
+      } else {
+        if (!purged_snaps_only) {


By contrast, this condition looks right. The snap wasn't purged so we only want to remove it if --purged-snaps-only wasn't passed.

In other words -- the set of snaps re-removed with --purged-snaps-only should be a subset of the set purged without it, right?

This is the kind of clarification the comment I requested at the top should address.

By contrast, this condition looks right. The snap wasn't purged so we only want to remove it if --purged-snaps-only wasn't passed.

Right. However, the default/non-default variant are mutually exclusive. (Mentioned in comment)

src/mon/OSDMonitor.cc

``` /* * Forces removal of snapshots in the range of * [lower_snapid_bound, upper_snapid_bound) on pool <pool> * in order to cause OSDs to re-trim them. * The command has two mutually exclusive variants: * * Default: All the snapids in the given range which are not * marked as purged in the Monitor will be removed. Mostly useful * for cases in which the snapid is leaked in the client side. * See: https://tracker.ceph.com/issues/64646 * * (Experimental) purged-snaps-only: Adding this flag will result * in the reremoval of snapids in the given range. * Only the snapids which are *already* marked as purged in the * Monitor will be removed again. This may be useful for cases in * which we would like to trigger OSD snap trimming again. */ ``` Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Matan-B · 2024-03-26T14:01:42Z

jenkins test make check

ljflores

Hey @Matan-B this failure in teuthology looks related to the snaps-few-objects-redelete change:
description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log
2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5}
backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure
d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only
objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow
thrashosds-health workloads/snaps-few-objects-redelete}
/a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652478

2024-04-12T01:06:16.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool force-remove-snap unique_pool_0 --purged-snaps-only
...
2024-04-12T01:09:09.030 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  finishing write tid 1 to smithi17628761-15
2024-04-12T01:09:09.030 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  finishing write tid 2 to smithi17628761-15
2024-04-12T01:09:09.030 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  finishing write tid 3 to smithi17628761-15
2024-04-12T01:09:09.030 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  finishing write tid 4 to smithi17628761-15
2024-04-12T01:09:09.030 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  oid 15 updating version 0 to 391
2024-04-12T01:09:09.030 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  oid 15 updating version 391 to 392
2024-04-12T01:09:09.031 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  oid 15 version 392 is already newer than 390
2024-04-12T01:09:09.031 INFO:tasks.rados.rados.0.smithi176.stdout:update_object_version oid 15 v 392 (ObjNum 642 snap 171 seq_num 642) dirty exists
2024-04-12T01:09:09.031 INFO:tasks.rados.rados.0.smithi176.stdout:1798:  left oid 15 (ObjNum 642 snap 171 seq_num 642)
2024-04-12T01:09:09.034 ERROR:teuthology.orchestra.daemon.state:Failed to send signal 1: None
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/daemon/state.py", line 108, in signal
    self.proc.stdin.write(struct.pack('!b', sig))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 385, in write
    raise IOError("File is closed")
OSError: File is closed

ljflores · 2024-04-17T21:34:58Z

qa/suites/rados/thrash/workloads/pool-snaps-few-objects-redelete.yaml

@@ -0,0 +1,23 @@
+overrides:


Also with this test:

description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log
2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5}
backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat
mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4
rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health
workloads/pool-snaps-few-objects-redelete}

/a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652491

2024-04-12T01:38:22.133 INFO:tasks.ceph.osd.7.smithi169.stderr:./src/osd/PG.cc: 1901: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps) 2024-04-12T01:38:22.133 INFO:tasks.ceph.osd.7.smithi169.stderr: 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: ceph version 19.0.0-2838-ga5074d45 (a5074d4516d566e9d8b6aec912f26afd099de101) squid (dev) 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x118) [0x55691a976362] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 2: ceph-osd(+0x3f6519) [0x55691a976519] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 3: ceph-osd(+0x38f81c) [0x55691a90f81c] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 4: (PeeringState::Active::react(PeeringState::AdvMap const&)+0x19e) [0x55691ad92e5e] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 5: ceph-osd(+0x840811) [0x55691adc0811] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 6: (PeeringState::advance_map(std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, std::vector<int, std::allocator<int> >&, int, std::vector<int, std::allocator<int> >&, int, PeeringCtx&)+0x266) [0x55691ad596e6] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 7: (PG::handle_advance_map(std::shared_ptr<OSDMap const>, std::shared_ptr<OSDMap const>, std::vector<int, std::allocator<int> >&, int, std::vector<int, std::allocator<int> >&, int, PeeringCtx&)+0xfb) [0x55691ab7a76b] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 8: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&, PeeringCtx&)+0x39c) [0x55691aaf318c] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 9: (OSD::dequeue_peering_evt(OSDShard*, PG*, std::shared_ptr<PGPeeringEvent>, ThreadPool::TPHandle&)+0x237) [0x55691ab04957] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 10: (ceph::osd::scheduler::PGPeeringItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x51) [0x55691ad374f1] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 11: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xab3) [0x55691ab0e6e3] 2024-04-12T01:38:22.134 INFO:tasks.ceph.osd.7.smithi169.stderr: 12: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x293) [0x55691b0024e3] 2024-04-12T01:38:22.135 INFO:tasks.ceph.osd.7.smithi169.stderr: 13: ceph-osd(+0xa82a44) [0x55691b002a44] 2024-04-12T01:38:22.135 INFO:tasks.ceph.osd.7.smithi169.stderr: 14: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f9933f96b43] 2024-04-12T01:38:22.135 INFO:tasks.ceph.osd.7.smithi169.stderr: 15: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f9934028a00]

The fix for https://tracker.ceph.com/issues/64347 is already included in the test branch, so this seems like a new crash related to this particular test.

ljflores · 2024-04-17T21:38:04Z

qa/suites/rados/thrash/workloads/snaps-few-objects-redelete.yaml

@@ -0,0 +1,19 @@
+overrides:


This failure also looks related to this test:

/a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652505

2024-04-12T01:51:20.020 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_a5074d4516d566e9d8b6aec912f26afd099de101/qa/tasks/ceph_manager.py", line 190, in wrapper return func(self) File "/home/teuthworker/src/github.com_ceph_ceph-c_a5074d4516d566e9d8b6aec912f26afd099de101/qa/tasks/ceph_manager.py", line 1483, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph-c_a5074d4516d566e9d8b6aec912f26afd099de101/qa/tasks/ceph_manager.py", line 1321, in <lambda> self.inject_pause(key, File "/home/teuthworker/src/github.com_ceph_ceph-c_a5074d4516d566e9d8b6aec912f26afd099de101/qa/tasks/ceph_manager.py", line 1065, in inject_pause self.ceph_manager.set_config(the_one, **{conf_key: duration}) File "/home/teuthworker/src/github.com_ceph_ceph-c_a5074d4516d566e9d8b6aec912f26afd099de101/qa/tasks/ceph_manager.py", line 2093, in set_config self.wait_run_admin_socket( File "/home/teuthworker/src/github.com_ceph_ceph-c_a5074d4516d566e9d8b6aec912f26afd099de101/qa/tasks/ceph_manager.py", line 2050, in wait_run_admin_socket raise Exception('timed out waiting for admin_socket ' Exception: timed out waiting for admin_socket to appear after osd.4 restart

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Matan-B · 2024-05-19T13:37:57Z

This PR introduces a new command with 2 variations.
The non default one is quite more complex and less urgent to be merged.
I will separate the default behavior into #57548 and the non default (purged-snaps-only) variant to #57549

github-actions bot added core mon tests labels Sep 20, 2023

github-actions bot added the needs-rebase label Oct 11, 2023

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from ae85111 to 70c85ab Compare October 31, 2023 10:07

github-actions bot removed the needs-rebase label Oct 31, 2023

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from 70c85ab to 99d7d63 Compare November 6, 2023 13:50

github-actions bot added the needs-rebase label Dec 18, 2023

github-actions bot added the stale label Feb 16, 2024

Matan-B changed the title ~~osd: Add force-reremove-snap mon command~~ osd: Add force-remove-snap mon command Mar 3, 2024

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from 99d7d63 to b76825b Compare March 3, 2024 14:44

github-actions bot removed the needs-rebase label Mar 3, 2024

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from b76825b to cd6fd40 Compare March 3, 2024 14:59

github-actions bot removed the stale label Mar 3, 2024

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from cd6fd40 to c979234 Compare March 3, 2024 15:08

Matan-B requested review from athanatos and rzarzynski March 3, 2024 15:09

Matan-B mentioned this pull request Mar 3, 2024

mon/OSDMonitor: fix rmsnap command #55841

Merged

14 tasks

Matan-B marked this pull request as ready for review March 4, 2024 09:09

Matan-B requested a review from a team as a code owner March 4, 2024 09:09

rzarzynski reviewed Mar 4, 2024

View reviewed changes

src/mon/OSDMonitor.cc Outdated Show resolved Hide resolved

src/mon/OSDMonitor.cc Outdated Show resolved Hide resolved

src/mon/OSDMonitor.cc Show resolved Hide resolved

src/mon/OSDMonitor.cc Outdated Show resolved Hide resolved

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from c979234 to 96cc714 Compare March 5, 2024 11:09

Matan-B requested a review from rzarzynski March 5, 2024 11:10

Matan-B changed the title ~~osd: Add force-remove-snap mon command~~ mon: Add force-remove-snap mon command Mar 5, 2024

rzarzynski approved these changes Mar 5, 2024

View reviewed changes

rzarzynski added the needs-qa label Mar 5, 2024

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from 96cc714 to 57da5c5 Compare March 5, 2024 15:00

athanatos requested changes Mar 5, 2024

View reviewed changes

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from 57da5c5 to b370cd5 Compare March 7, 2024 15:19

Matan-B requested a review from athanatos March 7, 2024 15:20

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from b370cd5 to 0ee333c Compare March 7, 2024 16:05

qa/tasks/ceph_manager: thrash - add force_reremove_snap

cafad77

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Matan-B force-pushed the wip-matanb-reremove-snap-only branch from 0ee333c to cafad77 Compare March 10, 2024 09:27

athanatos approved these changes Mar 27, 2024

View reviewed changes

ljflores added the wip-yuri6-testing label Apr 2, 2024

ljflores reviewed Apr 17, 2024

View reviewed changes

ljflores added TESTED and removed needs-qa wip-yuri6-testing labels Apr 17, 2024

ljflores reviewed Apr 17, 2024

View reviewed changes

qa/tasks/ceph_manager: force_reremove_snap only once

6539abb

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Matan-B mentioned this pull request May 19, 2024

mon/OSDMonitor: Add force-remove-snap mon command #57548

Merged

14 tasks

Matan-B added the DNM label May 19, 2024

Matan-B closed this May 19, 2024

Conversation

Matan-B commented Sep 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Variant 1 (Default):

Remove all snapids in the range which are not marked as purged.

Case 2 - purged-snaps-only:

Reremove snap ids in the range which are already marked as purged.

Contribution Guidelines

Checklist

Uh oh!

github-actions bot commented Oct 11, 2023

Uh oh!

Matan-B commented Oct 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Matan-B commented Nov 6, 2023

Uh oh!

github-actions bot commented Dec 18, 2023

Uh oh!

github-actions bot commented Feb 16, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rzarzynski commented Mar 5, 2024

Uh oh!

Matan-B commented Mar 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

athanatos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

athanatos Mar 5, 2024

Choose a reason for hiding this comment

Uh oh!

Matan-B Mar 7, 2024

Choose a reason for hiding this comment

Uh oh!

Matan-B Mar 7, 2024

Choose a reason for hiding this comment

Uh oh!

athanatos Mar 5, 2024

Choose a reason for hiding this comment

Uh oh!

Matan-B Mar 7, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Matan-B commented Mar 26, 2024

Uh oh!

ljflores left a comment

Choose a reason for hiding this comment

Uh oh!

ljflores Apr 17, 2024

Choose a reason for hiding this comment

Uh oh!

ljflores Apr 17, 2024

Choose a reason for hiding this comment

Uh oh!

Matan-B commented May 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Matan-B commented Sep 20, 2023 •

edited

Loading

Matan-B commented Oct 31, 2023 •

edited

Loading

Matan-B commented Mar 5, 2024 •

edited

Loading

Matan-B commented May 19, 2024 •

edited

Loading