rgw: user accounts implementation by cbodley · Pull Request #54333 · ceph/ceph

cbodley · 2023-11-03T16:36:41Z

user account design document in #54045

this revives previous work from #46373 (which iself was based on Abhishek's #35726). that became too painful to faithfully rebase, so i've reimplemented parts of it instead; i'll be sure to credit Abhishek for his work in the final commits

TODO

Account metadata

struct RGWAccountInfo
requirements for account ids and name (rgw::account::validate_id/name())
rgw::sal::Driver interfaces to read/write
admin ops to create/modify/list/remove account metadata
add email, and ensure it's globally unique (including user emails)

User metadata

add RGWUserInfo::account_id member
support --account-id argument for user modify to migrate existing legacy/non-account users into an account. account id can't change after
migrating a user should also migrate the user's buckets, changing their owner to the account
add other metadata fields needed for IAM User APIs (Path/Arn/CreateDate)
match user paths in iam policy principals

Role metadata

add RGWRoleInfo::account_id member
index account roles in the {account}.roles omap
within an account, role names don't have to be globally unique - don't store the 'path' object in rados, use omap for listing

Group metadata

struct RGWGroupInfo
rgw::sal::Driver interfaces to read/write/list
metadata handler for multisite replication

Topic metadata

change owner from rgw_user to rgw_owner
index account topics in the {account}.topics omap

RadosStore

implement account interfaces for RadosStore
use cls_user to track stats in {account}.buckets the same way users do in {user}.buckets
extend cls_user to track {account}.users, {account}.roles, {account}.groups (started in cls/user: add interfaces to index user account resources #54563)
RGWBucketMetadataHandler: on change to RGWBucketEntryPoint::owner, update linkage in {user}.buckets and/or {account}.buckets objects via cls_user
RGWUserMetadataHandler: on change to RGWUserInfo::account_id, update linkage in {account}.users via cls_user

ACLs

ACLOwner and CanonicalUser grants can either be a user id string or an account id string
AmazonCustomerByEmail supports account emails in addition to user emails
resources created by a user in an account use account id for default ACLOwner and FULL_CONTROL Grant
use account name for DisplayName

IAM Policy

Principal: disambiguate 'tenant' and 'account' principals using rgw::account::validate_id()
match 'account' principals against RGWUserInfo::account_id instead of RGWUserInfo::user_id.tenant
load group policies for use in verify_permission
support managed policies like AmazonS3FullAccess for AttachUserPolicy/AttachRolePolicy
implement cross-account policy evaluation
integrate 'account root' users into policy evaluation

Bucket Ownership

RGWBucketInfo::owner uses rgw_owner variant which can either be rgw_user or rgw_account_id
buckets created by a user/role in an account are owned by the account
rgw_quota.cc accumulates/flushes account stats similar to user stats based on RGWBucketInfo::owner
ListBuckets can list from the owning account

IAM APIs

CreateUser, GetUser, UpdateUser, DeleteUser, ListUsers apis (ex https://docs.aws.amazon.com/IAM/latest/APIReference/API_CreateUser.html)
AccessKey APIs
adapt existing Role APIs to account permission model
adapt existing UserPolicy APIs to account permission model
adapt existing Topic APIs to account permission model
adapt existing OpenIDConnectProvider APIs to account permission model
Group/GroupPolicy APIs

Testing

add account root user to s3tests config for iam api tests
s3tests cases for new apis in test_iam.py (User apis started in iam: add tests for account-based IAM apis s3-tests#537)
s3tests for account ids in bucket policy
in teuthology, run s3tests in configurations where the 's3 main' and 's3 tenant' users belong to an account
in teuthology, create account-root users for s3test config [iam root] and [iam alt root] to run iam_account and cross_account tests
test user migration into an account and verify access to existing buckets/objects
test account stats and quota enforcement
multisite test cases for sync of account metadata and related indices

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

github-actions · 2023-11-28T14:55:00Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

github-actions · 2023-12-01T17:57:06Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

github-actions · 2023-12-15T05:13:42Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

Signed-off-by: Casey Bodley <cbodley@redhat.com>

move the persistent queue removal into remove_topic() where we have access to the topic metadata. avoid trying to remove the queue if it isn't enabled Signed-off-by: Casey Bodley <cbodley@redhat.com>

Signed-off-by: Casey Bodley <cbodley@redhat.com>

a bucket's notification configuration may refer to topics from several different tenants or accounts. when publishing to a given topic, look in the correct namespace for each topic instead of defaulting to the requesting user's tenant namespace Signed-off-by: Casey Bodley <cbodley@redhat.com>

Signed-off-by: Casey Bodley <cbodley@redhat.com>

new sns test cases are using this for topic names, but the '.' is not allowed there: > api_params = {'Name': 'test-client.0-n3bdgre5el2jk8v-606'} > botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the CreateTopic operation: Name must be made up of only uppercase and lowercase ASCII letters, numbers, underscores, and hyphens Signed-off-by: Casey Bodley <cbodley@redhat.com>

BBoozmen · 2024-04-10T18:34:29Z

How is bucket quota managed for an account?

For a non-accounted rgw user, we can set user and bucket level quotas and both are visible in user info:

$ radosgw-admin user info --uid=ihummel | jq '.user_quota, .bucket_quota'
{
  "enabled": true,
  "check_on_raw": false,
  "max_size": 4028225472512,
  "max_size_kb": 3933813938,
  "max_objects": -1
}
{
  "enabled": true,
  "check_on_raw": false,
  "max_size": -1024,
  "max_size_kb": 0,
  "max_objects": 20000000
}

However, for an account, we can only set account level quota:

radosgw-admin account get --account-id=RGW12345678912345677 | jq '.quota'
{
        "enabled": false,
        "check_on_raw": false,
        "max_size": 2048,
        "max_size_kb": 2,
        "max_objects": -1
}

Shouldn't we also be able to set/view bucket quotas given an account?

cbodley · 2024-04-10T18:49:24Z

Shouldn't we also be able to set/view bucket quotas given an account?

@BBoozmen you can set bucket quotas directly on the buckets, so this didn't seem useful to me. if you disagree, would you open a tracker issue with a feature request? i'm trying to get this pr merged for squid asap

BBoozmen · 2024-04-10T18:52:23Z

Shouldn't we also be able to set/view bucket quotas given an account?

@BBoozmen you can set bucket quotas directly on the buckets, so this didn't seem useful to me. if you disagree, would you open a tracker issue with a feature request? i'm trying to get this pr merged for squid asap

Sounds good. We can discuss this as a separate item.

cbodley · 2024-04-11T13:21:03Z

jenkins retest this please

cbodley · 2024-04-11T16:50:52Z

The slave crashed or was restarted

cbodley · 2024-04-11T16:50:56Z

jenkins test make check

cbodley · 2024-04-11T18:56:43Z

the rgw/notifications tests are failing with An error occurred (AuthorizationError) when calling the CreateTopic operation after rebase over #55661, which changed verify_user_permissions() for subusers. i've opened #56838 to revert that change..

…create buckets" This reverts commit 3cc27f0. Signed-off-by: Casey Bodley <cbodley@redhat.com>

…pics Signed-off-by: Casey Bodley <cbodley@redhat.com>

cbodley · 2024-04-11T20:10:41Z

the rgw/notifications tests are failing with An error occurred (AuthorizationError) when calling the CreateTopic operation after rebase over #55661, which changed verify_user_permissions() for subusers. i've opened #56838 to revert that change..

@yuvalif the new test_ps_s3_persistent_notification_http_account test case was failing after that. i had to add another commit to teach wait_for_queue_to_drain() to add the --account-id argument to topic stats

cbodley · 2024-04-12T17:10:12Z

passed qa in https://pulpito.ceph.com/cbodley-2024-04-12_12:44:47-rgw-wip-rgw-account-v3-distro-default-smithi/ with rerun https://pulpito.ceph.com/cbodley-2024-04-12_16:16:29-rgw-wip-rgw-account-v3-distro-default-smithi/

github-actions bot added build/ops common rgw labels Nov 3, 2023

cbodley force-pushed the wip-rgw-account-v3 branch from 3db7cf1 to 964203e Compare November 3, 2023 20:56

cbodley mentioned this pull request Nov 8, 2023

rgw/sal: Bucket owner as rgw_user #54379

Merged

github-actions bot added the needs-rebase label Nov 28, 2023

cbodley force-pushed the wip-rgw-account-v3 branch from a6a85a8 to d8b7122 Compare November 30, 2023 22:49

github-actions bot added tests and removed needs-rebase labels Nov 30, 2023

cbodley force-pushed the wip-rgw-account-v3 branch from d8b7122 to c6c63fd Compare November 30, 2023 23:24

github-actions bot added the needs-rebase label Dec 1, 2023

This was referenced Dec 5, 2023

rgw: remove unused backend abstraction for metadata handlers #54777

Merged

RGW/STS: adding role ownership and verify permission policy given a role action #54826

Closed

rgw/auth: Identity generates ACLOwner for resources it creates #54830

Closed

cbodley force-pushed the wip-rgw-account-v3 branch from c6c63fd to 1b2359b Compare December 14, 2023 14:05

github-actions bot added documentation and removed needs-rebase labels Dec 14, 2023

cbodley force-pushed the wip-rgw-account-v3 branch from 1b2359b to a020826 Compare December 14, 2023 15:45

github-actions bot added the needs-rebase label Dec 15, 2023

cbodley force-pushed the wip-rgw-account-v3 branch from a020826 to 82a3e0c Compare December 17, 2023 15:42

github-actions bot removed the needs-rebase label Dec 17, 2023

cbodley force-pushed the wip-rgw-account-v3 branch from 42c6b16 to 635d1b5 Compare December 19, 2023 20:19

cbodley mentioned this pull request Dec 19, 2023

cls/user: add interfaces to index user account resources #54563

Closed

cbodley force-pushed the wip-rgw-account-v3 branch 3 times, most recently from 2481ee8 to 5c6bbe6 Compare December 20, 2023 22:04

github-actions bot added the needs-rebase label Dec 24, 2023

cbodley added 13 commits April 10, 2024 13:09

rgw/pubsub: CreateTopic consults existing topic policy for overwrite

9b6fe4a

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw/pubsub: topic policy doesn't deny access to owner

a75c3ac

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: add/remove_persistent_topic() takes topic queue, not name

ba2566a

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: rgw_pubsub_dest stores persistent queue oid

3ef1ab3

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: RGWPubSub::remove_topic() removes persistent queue

4c50ad6

move the persistent queue removal into remove_topic() where we have access to the topic metadata. avoid trying to remove the queue if it isn't enabled Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: use rgw_pubsub_dest::persistent_queue for queue oid

4bac81a

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/rgw: warn about topics under account migration

bfae83c

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/notify: publish functions use rgw_pubsub_dest::persistent_queue

64ac1de

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/pubsub: forward requests as s->owner instead of s->user

63e164f

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw/pubsub: test persistent notifications with account user

43ca67b

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/notify: populate event userIdentity with account ids

24841fb

Signed-off-by: Casey Bodley <cbodley@redhat.com>

This was referenced Apr 11, 2024

Revert "RGW: a subuser with no permission can still list buckets and create buckets" #56838

Closed

RGW: a subuser with no permission can still list buckets and create buckets #55661

Merged

cbodley added 2 commits April 11, 2024 15:19

Revert "RGW: a subuser with no permission can still list buckets and …

a794e12

…create buckets" This reverts commit 3cc27f0. Signed-off-by: Casey Bodley <cbodley@redhat.com>

test/rgw/pubsub: wait_for_queue_to_drain() supports tenant/account to…

4007284

…pics Signed-off-by: Casey Bodley <cbodley@redhat.com>

This was referenced Apr 12, 2024

rgw/iam: zero-pad milliseconds for role and oidc-provider CreateDate #55532

Closed

squid: rgw: user accounts implementation #56863

Merged

This was referenced Aug 9, 2024

rgw: removing account/tenant from the constructor #59113

Closed

rgw: code to return error if a role is deleted with permission #59208

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rgw: user accounts implementation#54333

rgw: user accounts implementation#54333
cbodley merged 171 commits intoceph:mainfrom
cbodley:wip-rgw-account-v3

cbodley commented Nov 3, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Nov 28, 2023

Uh oh!

github-actions bot commented Dec 1, 2023

Uh oh!

github-actions bot commented Dec 15, 2023

Uh oh!

BBoozmen commented Apr 10, 2024

Uh oh!

cbodley commented Apr 10, 2024

Uh oh!

BBoozmen commented Apr 10, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 12, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Conversation

cbodley commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO

Account metadata

User metadata

Role metadata

Group metadata

Topic metadata

RadosStore

ACLs

IAM Policy

Bucket Ownership

IAM APIs

Testing

Uh oh!

github-actions bot commented Nov 28, 2023

Uh oh!

github-actions bot commented Dec 1, 2023

Uh oh!

github-actions bot commented Dec 15, 2023

Uh oh!

BBoozmen commented Apr 10, 2024

Uh oh!

cbodley commented Apr 10, 2024

Uh oh!

BBoozmen commented Apr 10, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 11, 2024

Uh oh!

cbodley commented Apr 12, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

cbodley commented Nov 3, 2023 •

edited

Loading