rgw/cloud-restore: admin CLI for restore list and status by thotz · Pull Request #62830 · ceph/ceph

thotz · 2025-04-15T15:00:14Z

Adding radosgw-admin cli to list and check status of objects in the bucket

Sample O/P

sample output:
aws s3 ls --endpoint-url http://localhost:8002 s3://bkt
2025-08-20 23:12:57          0 blah
2025-08-20 17:52:41          0 file
2025-08-20 23:02:00          0 new


build$ aws s3api restore-object --endpoint-url http://localhost:8002 --bucket bkt --key new --restore-request Days=20
build$ aws s3api restore-object --endpoint-url http://localhost:8002 --bucket bkt --key file --restore-request Days=20

$ aws s3 ls --endpoint-url http://localhost:8002 s3://bkt
2025-08-20 23:12:57          0 blah
2025-08-20 17:52:41          6 file
2025-08-20 23:02:00          6 new


build$ ../src/mrun c2 radosgw-admin restore status --bucket bkt --object new
{
    "name": "new",
    "RestoreExpiryDate": "2025-08-21T14:28:13.033681Z",
    "RestoreStatus": "CloudRestored",
    "RestoreType": "Temporary",
    "RestoreTime": "2025-08-21T14:18:13.033678Z"
}
build$ ../src/mrun c2 radosgw-admin bucket stats --bucket bkt --show--restore-stats
{
    "bucket": "bkt",
    "tenant": "",
    "versioning": "off",
    "zonegroup": "bc56dd78-01cf-4122-9b0c-fdf492d7410f",
    "placement_rule": "default-placement",
    "explicit_placement": {
        "data_pool": "",
        "data_extra_pool": "",
        "index_pool": ""
    },
    "id": "8526fe7b-3682-41a5-b972-0245727f5409.14177.9",
    "marker": "8526fe7b-3682-41a5-b972-0245727f5409.14177.9",
    "index_type": "Normal",
    "index_generation": 0,
    "num_shards": 11,
    "reshard_status": "None",
    "judge_reshard_lock_time": "0.000000",
    "object_lock_enabled": false,
    "mfa_enabled": false,
    "owner": "testid",
    "ver": "0#1,1#1,2#1,3#1,4#36,5#1,6#1,7#1,8#50,9#1,10#1",
    "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0",
    "mtime": "2025-08-20T12:21:54.997530Z",
    "creation_time": "2025-08-20T11:19:49.007743Z",
    "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#",
    "usage": {
        "rgw.main": {
            "size": 0,
            "size_actual": 0,
            "size_utilized": 0,
            "size_kb": 0,
            "size_kb_actual": 0,
            "size_kb_utilized": 0,
            "num_objects": 0
        },
        "rgw.cloudtiered": {
            "size": 12,
            "size_actual": 8192,
            "size_utilized": 12,
            "size_kb": 1,
            "size_kb_actual": 8,
            "size_kb_utilized": 1,
            "num_objects": 3
        }
    },
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "read_tracker": 2
}
{
    "restore_stats": {
        "restore_completed_count": 2,
        "restore_in_progress_count": 0,
        "restore_failed_count": 0
    }
}

build$ ../src/mrun c2 radosgw-admin restore list --bucket bkt
{
    "file": "CloudRestored",
    "new": "CloudRestored"
}
build$ ../src/mrun c2 radosgw-admin restore list --bucket bkt --restore-status=CloudRestored
{
    "file": "CloudRestored",,
    "new": "CloudRestored"
}

Fixes: https://tracker.ceph.com/issues/70931

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins test classic perf Jenkins Job | Jenkins Job Definition
jenkins test crimson perf Jenkins Job | Jenkins Job Definition
jenkins test signed Jenkins Job | Jenkins Job Definition
jenkins test make check Jenkins Job | Jenkins Job Definition
jenkins test make check arm64 Jenkins Job | Jenkins Job Definition
jenkins test submodules Jenkins Job | Jenkins Job Definition
jenkins test dashboard Jenkins Job | Jenkins Job Definition
jenkins test dashboard cephadm Jenkins Job | Jenkins Job Definition
jenkins test api Jenkins Job | Jenkins Job Definition
jenkins test docs ReadTheDocs | Github Workflow Definition
jenkins test ceph-volume all Jenkins Jobs | Jenkins Jobs Definition
jenkins test windows Jenkins Job | Jenkins Job Definition
jenkins test rook e2e Jenkins Job | Jenkins Job Definition

github-actions · 2025-06-14T17:01:26Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

adamemerson · 2025-06-16T19:47:29Z

@soumyakoduri Could you take a look at this?

soumyakoduri

Could you please paste sample output of these cmds.

Also IMO it would benefit if we could list objects' names as part of restore_list output along with their restore status. These count (for each status value) can be added to bucket stats instead.

mattbenjamin

looks good to me

thotz · 2025-09-09T09:14:45Z

jenkins test api

thotz · 2025-09-09T09:14:50Z

jenkins test make check

anthonyeleven

Suggest uppercasing CLI in the PR title.

thotz · 2025-09-10T06:15:12Z

teutology cloud transition : https://pulpito.ceph.com/jthottan-2025-09-09_09:07:41-rgw:cloud-transition-wip-thotz-radosgw-admin-cli-restore-list-status-distro-default-smithi/

teutology full run : https://pulpito.ceph.com/jthottan-2025-09-09_09:09:19-rgw-wip-thotz-radosgw-admin-cli-restore-list-status-distro-default-smithi/

thotz · 2025-09-10T13:02:07Z

jenkins test make check

soumyakoduri

few comments wrt to error handling. Rest looks good to me.

soumyakoduri · 2025-09-10T13:09:17Z

+            ret = obj->get_obj_attrs(null_yield, dpp());
+            if (ret < 0) {
+              cerr << "ERROR: failed to stat object, returned error: " << cpp_strerror(-ret) << std::endl;
+              return 1;


What does the caller expects in case of error - negative or positive return codes ? Like was the case in bucket_restore_stats, I see mix of positive and negative values being returned in this routine too for error cases. Is that expected?

main() in radosgw-admin.cc expecting postive returns values, so bucket_restore_stats needs to return negative values I will change accordingly. Thanks for pointing it out

thotz · 2025-09-24T04:35:34Z

jenkins test api

thotz · 2025-09-24T04:35:50Z

jenkins test make check

Also added stats as part of radosgw-admin bucket stats command Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

soumyakoduri

LGTM

thotz · 2025-10-24T10:44:48Z

jenkins test make check

soumyakoduri · 2025-10-24T12:08:18Z

@thotz .. looks like make check is failing in radosgw-admin help. Could you please check

…cket stats Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

thotz · 2025-10-27T17:39:13Z

jenkins test make check arm64

thotz · 2025-10-27T20:38:41Z

jenkins test make check arm64

thotz · 2025-10-28T18:20:06Z

jenkins test make check arm64

thotz · 2025-10-28T22:16:59Z

jenkins test make check arm64

thotz · 2025-11-03T05:17:12Z

jenkins test make check arm64

thotz · 2025-11-03T12:26:34Z

jenkins test make check arm64

anrao19 · 2025-11-11T05:32:51Z

Pr testing completed , Approved by @ivancich details are in :https://tracker.ceph.com/issues/73724,
@thotz , Please let me know if no further testing needed if not we can merge the pr

thotz requested a review from a team as a code owner April 15, 2025 15:00

github-actions Bot added the rgw label Apr 15, 2025

thotz requested a review from soumyakoduri April 15, 2025 15:02

github-actions Bot added the stale label Jun 14, 2025

thotz removed the stale label Jun 16, 2025

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from 2afd3c6 to 1a5541a Compare July 21, 2025 11:41

soumyakoduri reviewed Aug 11, 2025

View reviewed changes

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from 1a5541a to 8d46060 Compare August 21, 2025 14:20

thotz requested a review from soumyakoduri August 21, 2025 14:21

mattbenjamin approved these changes Aug 25, 2025

View reviewed changes

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from 8d46060 to 1e217a8 Compare September 9, 2025 06:36

thotz requested a review from a team as a code owner September 9, 2025 06:36

github-actions Bot added the documentation label Sep 9, 2025

anthonyeleven reviewed Sep 9, 2025

View reviewed changes

Comment thread doc/man/8/radosgw-admin.rst Outdated

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from 1e217a8 to df9ae00 Compare September 9, 2025 17:16

thotz requested a review from anthonyeleven September 9, 2025 17:16

soumyakoduri reviewed Sep 9, 2025

View reviewed changes

anthonyeleven reviewed Sep 9, 2025

View reviewed changes

Comment thread doc/man/8/radosgw-admin.rst

Comment thread doc/man/8/radosgw-admin.rst Outdated

thotz changed the title ~~rgw/cloud-restore: admin cli for restore list and status~~ rgw/cloud-restore: admin CLI for restore list and status Sep 10, 2025

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from df9ae00 to 917c310 Compare September 10, 2025 06:13

thotz requested review from anthonyeleven, mattbenjamin and soumyakoduri September 10, 2025 07:35

soumyakoduri reviewed Sep 10, 2025

View reviewed changes

github-actions Bot removed the needs-rebase label Sep 22, 2025

rgw/cloud-restore: admin CLI for restore list and status

02c925c

Also added stats as part of radosgw-admin bucket stats command Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from f117ff4 to 02c925c Compare October 9, 2025 13:46

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from 26bea72 to d5f1eba Compare October 21, 2025 09:30

soumyakoduri reviewed Oct 21, 2025

View reviewed changes

Comment thread src/rgw/driver/rados/rgw_bucket.cc Outdated

Comment thread src/rgw/driver/rados/rgw_bucket.cc Outdated

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from d5f1eba to 0df2977 Compare October 23, 2025 09:32

thotz requested a review from soumyakoduri October 23, 2025 10:20

soumyakoduri approved these changes Oct 24, 2025

View reviewed changes

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from 0df2977 to baca588 Compare October 24, 2025 13:43

github-actions Bot added the tests label Oct 24, 2025

src/rgw_bucket.cc: include restore stats in the same json dict for bu…

1085dfc

…cket stats Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

thotz force-pushed the radosgw-admin-cli-restore-list-status branch from baca588 to 1085dfc Compare October 24, 2025 15:01

soumyakoduri approved these changes Oct 27, 2025

View reviewed changes

soumyakoduri added the needs-qa label Nov 5, 2025

anrao19 added wip-anrao1-testing TESTED ready-to-merge and removed wip-anrao1-testing labels Nov 5, 2025

soumyakoduri merged commit e745ac8 into ceph:main Nov 11, 2025
13 checks passed

Conversation

thotz commented Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution Guidelines

Checklist

Uh oh!

github-actions Bot commented Jun 14, 2025

Uh oh!

adamemerson commented Jun 16, 2025

Uh oh!

soumyakoduri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mattbenjamin left a comment

Choose a reason for hiding this comment

Uh oh!

thotz commented Sep 9, 2025

Uh oh!

thotz commented Sep 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anthonyeleven left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

thotz commented Sep 10, 2025

Uh oh!

thotz commented Sep 10, 2025

Uh oh!

soumyakoduri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

soumyakoduri Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

thotz Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

thotz commented Sep 24, 2025

Uh oh!

thotz commented Sep 24, 2025

Uh oh!

soumyakoduri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

thotz commented Oct 24, 2025

Uh oh!

soumyakoduri commented Oct 24, 2025

Uh oh!

thotz commented Oct 27, 2025

Uh oh!

thotz commented Oct 27, 2025

Uh oh!

thotz commented Oct 28, 2025

Uh oh!

thotz commented Oct 28, 2025

Uh oh!

thotz commented Nov 3, 2025

Uh oh!

thotz commented Nov 3, 2025

Uh oh!

thotz commented Apr 15, 2025 •

edited

Loading