Skip to content

prevent anonymous topic operations#49335

Merged
yuvalif merged 2 commits intoceph:mainfrom
yuvalif:wip-yuval-fix-58167
Feb 20, 2023
Merged

prevent anonymous topic operations#49335
yuvalif merged 2 commits intoceph:mainfrom
yuvalif:wip-yuval-fix-58167

Conversation

@yuvalif
Copy link
Contributor

@yuvalif yuvalif commented Dec 8, 2022

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@github-actions
Copy link

github-actions bot commented Dec 9, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

using namespace std;

void RGWHandler_REST_IAM::rgw_iam_parse_input()
using op_generator= std::function<RGWOp*(const bufferlist&)>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using op_generator = RGWOp* (*)(const bufferlist&);

works for gcc and clang in godbolt: https://godbolt.org/z/YGhT3z4xK

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it failed with "typedef", let me try "using".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have to be explicit with the return types of the lambdas

@yuvalif yuvalif force-pushed the wip-yuval-fix-58167 branch from 88b14e4 to a006c53 Compare December 13, 2022 17:50
@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 13, 2022

jenkins test docs

@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 18, 2022

when running the S3 test suite, the test_user_policy test is failing as RGWSI_MetaBackend_SObj::get_entry() get ENOENT when calling rgw_get_system_obj().
this is happening when the PutUserPolicy operation is trying to fetch the user with RGWSI_User_RADOS::read_user_info()

@yuvalif yuvalif force-pushed the wip-yuval-fix-58167 branch from a006c53 to f7d1326 Compare December 19, 2022 15:37
@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 27, 2022

when running the S3 test suite, the test_user_policy test is failing as RGWSI_MetaBackend_SObj::get_entry() get ENOENT when calling rgw_get_system_obj(). this is happening when the PutUserPolicy operation is trying to fetch the user with RGWSI_User_RADOS::read_user_info()

this failure is happening also on master (when running locally with vstart): https://tracker.ceph.com/issues/58365

@yuvalif yuvalif added needs-qa and removed DNM labels Dec 27, 2022
@yuvalif yuvalif requested a review from mattbenjamin December 27, 2022 16:32
@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 27, 2022

jenkins test make check

@yuvalif yuvalif requested a review from cbodley December 27, 2022 16:35
@yuvalif yuvalif changed the title [DNM] prevent anonymous topic operations prevent anonymous topic operations Dec 27, 2022
@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 27, 2022

jenkins test make check

return true;
}

if (op == rgw::IAM::s3CreateBucket || op == rgw::IAM::s3ListAllMyBuckets) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yuvalif : can you help me understand why the check for CreateBucket/ListAllMyBuckets has been removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pritha-srivastava after my changes, we reach this line when we do bucket list, and return false.
why did we add this "if" statement i nthe first place?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yuvalif : iirc, I added that because if a user tries to create a bucket/ list all buckets and there wasn't any policy (user policy, or role policy or session policy) for the same, or the policy didn't have any action to allow these s3 ops, then the code would go to the last statement and return false and these ops would be denied, which shouldn't be. I added the check as part of this commit: acee911

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before my change, the code was doing the check only for bucket creation and listing all buckets.
so, for these operations nothing will change.
the change is that the chek will be done for other operation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But for other iam ops (like create oidc provider etc) the ops must be allowed only based on the evaluation of the policies and in the absence of these policies, the op must be denied.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. so in the "common" function I should just return "false" if there is no policy?
and add the policy-less check to the verify_permission() of s3CreateBucket and s3ListAllMyBuckets ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually if verify_user_permission() returns false we wouldn't know whether its an actual 'deny' or simply because the policies don't exist or the policy doesn't have an action for these ops. So a better option would be to change the return type of the current verify_user_permission() to reflect the return type of policy evaluation, and then have a wrapper verify_user_permission with return type bool, that can invoke this new verify_user_permission. In this case we can move the extra 'if' check part to CreateBucket and ListAllMyBuckets verify_permission().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Policy is not mandatory in this case, a user with admin caps for oidc-provider can create an openidconnect provider in rgw.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, for the OIDC ops with no policy, we should run verify_user_permission_no_policy() and not fail immediatly?
this is not what the code initally did.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, for OIDC ops we do not need to go to verify_user_permission_no_policy(). Is it not possible to invoke verify_user_permission_no_policy() in CreateBucket/ListAllMyBuckets(), after policy evaluation is complete and the result is an Effect::Pass?

@yuvalif yuvalif force-pushed the wip-yuval-fix-58167 branch from f7d1326 to c552ce7 Compare January 16, 2023 17:22
@cbodley
Copy link
Contributor

cbodley commented Jan 19, 2023

@yuvalif do you think it would be worth splitting out the refactoring so we can merge/backport the trivial fix?

@yuvalif
Copy link
Contributor Author

yuvalif commented Jan 19, 2023

@yuvalif do you think it would be worth splitting out the refactoring so we can merge/backport the trivial fix?

it is split to 2 commits. one for the refactoring and one for the topic authentication fix.
do you wan tto split further?

@yuvalif
Copy link
Contributor Author

yuvalif commented Jan 22, 2023

baseline results
branch results
most failures are same with baseline (multisite, tempest, ragweed). However following s3test STS tests are failing only in branch and seems to be related to the change.
following run-datacache.sh test fails only on branch, but does not seem related:
http://qa-proxy.ceph.com/teuthology/yuvalif-2023-01-20_20:26:47-rgw-wip-yuval-fix-58167-distro-default-smithi/7131990/teuthology.log

seperate between the different non-bucket handler operations:
iam, sts, sns (topic) and non bucket s3 ops

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
fixes: https://tracker.ceph.com/issues/58167

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
@yuvalif yuvalif force-pushed the wip-yuval-fix-58167 branch from c552ce7 to 5bb78ab Compare January 25, 2023 06:20
@yuvalif
Copy link
Contributor Author

yuvalif commented Jan 25, 2023

jenkins test api

@yuvalif
Copy link
Contributor Author

yuvalif commented Jan 25, 2023

@pritha-srivastava could you please review the latest?

@pritha-srivastava
Copy link
Contributor

@pritha-srivastava could you please review the latest?

looks good to me.

@yuvalif
Copy link
Contributor Author

yuvalif commented Feb 12, 2023

@pritha-srivastava could you please review the latest?

looks good to me.

thanks. can you please approve?

@yuvalif
Copy link
Contributor Author

yuvalif commented Feb 16, 2023

teuthology results.
7 failures. similar to: #48709

main difference is the crash backtrace in multisite (showing an issue in tcmalloc):

        "/lib64/libpthread.so.0(+0x12cf0) [0x7f3c0ed6dcf0]",
        "(tcmalloc::CentralFreeList::FetchFromOneSpans(int, void**, void**)+0x43) [0x7f3c0fb23603]",
        "(tcmalloc::CentralFreeList::RemoveRange(void**, void**, int)+0xe2) [0x7f3c0fb239d2]",
        "(tcmalloc::ThreadCache::FetchFromCentralCache(unsigned int, int, void* (*)(unsigned long))+0x73) [0x7f3c0fb273e3]",
        "/lib64/librados.so.2(+0x101db6) [0x7f3c13660db6]",
        "/lib64/librados.so.2(+0x13a689) [0x7f3c13699689]",
        "/lib64/libstdc++.so.6(+0xc2ba3) [0x7f3c0e4e6ba3]",
        "/lib64/libpthread.so.0(+0x81ca) [0x7f3c0ed631ca]",
        "clone()"

@yuvalif
Copy link
Contributor Author

yuvalif commented Feb 20, 2023

all teuthology failures are known issues. see: #48709

@yuvalif yuvalif merged commit b619fda into ceph:main Feb 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants