Ceph » rgw

recent rgw/sts jobs are showing lots of failures around AssumeRoleWithWebIdentity

ex https://qa-proxy.ceph.com/teuthology/cbodley-2025-12-04_17:21:30-rgw:sts-main-distro-default-smithi/8641535/teuthology.log

2025-12-04T18:15:56.891 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity - b...
2025-12-04T18:15:56.891 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_check_on_different_buckets
2025-12-04T18:15:56.891 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_check_on_same_bucket
2025-12-04T18:15:56.891 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_check_put_obj_denial
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_swapping_role_policy_and_session_policy
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_check_different_op_permissions
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_check_with_deny_effect
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_check_with_deny_on_same_op
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_bucket_policy_role_arn
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_bucket_policy_session_arn
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_copy_object - boto...
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_no_bucket_role_policy
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_session_policy_bucket_policy_deny
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_with_sub
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_with_azp
2025-12-04T18:15:56.892 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_with_request_tag
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_with_principal_tag
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_for_all_values
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_for_all_values_deny
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_tag_keys_trust_policy
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_tag_keys_role_policy
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_resource_tag
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_resource_tag_deny
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_wrong_resource_tag_deny
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_resource_tag_princ_tag
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_resource_tag_copy_obj
2025-12-04T18:15:56.893 INFO:teuthology.orchestra.run.smithi183.stdout:FAILED s3tests/functional/test_sts.py::test_assume_role_with_web_identity_role_resource_tag

from rgw logs, it looks like these requests are failing due to expired tokens
ex https://qa-proxy.ceph.com/teuthology/cbodley-2025-12-04_17:21:30-rgw:sts-main-distro-default-smithi/8641535/remote/smithi183/log/rgw.ceph.client.0.log.gz

2025-12-04T18:15:36.901+0000 7f010540a640 10 req 10512356278025954736 0.035998888s sts:assume_role_web_identity Signature validation using x5c failedtoken verification failed: token expired
2025-12-04T18:15:36.901+0000 7f010540a640  0 req 10512356278025954736 0.035998888s sts:assume_role_web_identity Signature can not be validated with the JWKS present.
2025-12-04T18:15:36.901+0000 7f010540a640  5 req 10512356278025954736 0.035998888s sts:assume_role_web_identity Invalid JWT token
2025-12-04T18:15:36.901+0000 7f010540a640 20 req 10512356278025954736 0.035998888s sts:assume_role_web_identity rgw::auth::sts::WebTokenEngine denied with reason=-13
2025-12-04T18:15:36.901+0000 7f010540a640  5 req 10512356278025954736 0.035998888s sts:assume_role_web_identity Failed the auth strategy, reason=-13
2025-12-04T18:15:36.901+0000 7f010540a640 10 failed to authorize request
2025-12-04T18:15:36.901+0000 7f010540a640 20 req 10512356278025954736 0.035998888s op->ERRORHANDLER: err_no=-13 new_err_no=-13