Skip to content

quincy: ceph_fs.h: add separate owner_{u,g}id fields#53139

Merged
yuriw merged 1 commit intoceph:quincyfrom
mihalicyn:wip-62571-quincy
Oct 4, 2023
Merged

quincy: ceph_fs.h: add separate owner_{u,g}id fields#53139
yuriw merged 1 commit intoceph:quincyfrom
mihalicyn:wip-62571-quincy

Conversation

@mihalicyn
Copy link
Contributor

@mihalicyn mihalicyn commented Aug 24, 2023

This patch adds separate fields to pass inode owner's UID/GID for operations which create new inodes:
CEPH_MDS_OP_CREATE, CEPH_MDS_OP_MKNOD, CEPH_MDS_OP_RENAME, CEPH_MDS_OP_MKDIR, CEPH_MDS_OP_SYMLINK.

Before we have used caller_{u,g}id fields for two purposes:

  • for MDS permission checks
  • to set UID/GID for new inodes

but during a long discussion around adding idmapped mounts support to cephfs kernel client we decided to extend wire protocol [1].

Originally, Christian Brauner pointed to this problem [2], but a few initial versions of "ceph: support idmapped mounts" patchset were based on a compromise between bringing idmapped mounts support and changing on-wire data structures. The only problem with this approach is that if we have UID/GID-based permission checks on the MDS side they won't work as intended. Xiubo Li pointed to this issue and said that it's critical enough to not being ignored.

This patch does not change behavior for client or MDS, because in the current client implementation we do:
req->set_inode_owner_uid_gid(perms.uid(), perms.gid()); so, these new fields contain the same value as caller_{u,g}id.

This new extension will be used in a non-trivial way in the Linux kernel client.

[1] https://lore.kernel.org/lkml/CAEivzxcipdTQ9-b2zk-7-AMDfwPk2Brtp=X6H8xXsHMEtQJKFQ@mail.gmail.com/
[2] https://lore.kernel.org/all/20220104140414.155198-3-brauner@kernel.org/

Fixes: https://tracker.ceph.com/issues/62217

Signed-off-by: Alexander Mikhalitsyn aleksandr.mikhalitsyn@canonical.com
(cherry picked from commit 46cb244)

backport tracker https://tracker.ceph.com/issues/62571
backport of #52575
parent tracker https://tracker.ceph.com/issues/62217

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

This patch adds separate fields to pass inode owner's UID/GID
for operations which create new inodes:
CEPH_MDS_OP_CREATE, CEPH_MDS_OP_MKNOD, CEPH_MDS_OP_RENAME,
CEPH_MDS_OP_MKDIR, CEPH_MDS_OP_SYMLINK.

Before we have used caller_{u,g}id fields for two purposes:
- for MDS permission checks
- to set UID/GID for new inodes

but during a long discussion around adding idmapped mounts
support to cephfs kernel client we decided to extend wire protocol [1].

Originally, Christian Brauner pointed to this problem [2], but
a few initial versions of "ceph: support idmapped mounts" patchset
were based on a compromise between bringing idmapped mounts support
and changing on-wire data structures. The only problem with this approach
is that if we have UID/GID-based permission checks on the MDS side they
won't work as intended. Xiubo Li pointed to this issue and said that
it's critical enough to not being ignored.

This patch does not change behavior for client or MDS, because
in the current client implementation we do:
req->set_inode_owner_uid_gid(perms.uid(), perms.gid());
so, these new fields contain the same value as caller_{u,g}id.

This new extension will be used in a non-trivial way in the Linux kernel client.

[1] https://lore.kernel.org/lkml/CAEivzxcipdTQ9-b2zk-7-AMDfwPk2Brtp=X6H8xXsHMEtQJKFQ@mail.gmail.com/
[2] https://lore.kernel.org/all/20220104140414.155198-3-brauner@kernel.org/

Fixes: https://tracker.ceph.com/issues/62217

Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>
(cherry picked from commit 46cb244)
@github-actions github-actions bot added the cephfs Ceph File System label Aug 24, 2023
@github-actions github-actions bot added this to the quincy milestone Aug 24, 2023
@mihalicyn
Copy link
Contributor Author

jenkins retest this please

1 similar comment
@mihalicyn
Copy link
Contributor Author

jenkins retest this please

@mihalicyn
Copy link
Contributor Author

Failure looks unrelated:

275/275 Test #144: readable.sh ...............................   Passed  1456.61 sec

99% tests passed, 1 tests failed out of 275

Total Test time (real) = 1523.61 sec

The following tests FAILED:
	 32 - run-rbd-unit-tests-61.sh (Failed)
Errors while running CTest

@mihalicyn
Copy link
Contributor Author

jenkins retest this please

@mihalicyn
Copy link
Contributor Author

mihalicyn commented Aug 25, 2023

cc @vshankar @lxbsz @batrick

Copy link
Contributor

@mchangir mchangir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mchangir
Copy link
Contributor

mchangir commented Oct 4, 2023

Reviewed-by: Milind Changire mchangir@redhat.com

@yuriw yuriw merged commit d873f1e into ceph:quincy Oct 4, 2023
@vshankar
Copy link
Contributor

@yuriw @mihalicyn @lxbsz @mchangir - fails smoke test - https://pulpito.ceph.com/yuriw-2023-10-19_16:09:31-smoke-quincy-release-distro-default-smithi/7432399/

2023-10-19T16:44:02.283 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0
2023-10-19T16:44:02.297 INFO:tasks.ceph.mds.a.smithi084.stderr:/build/ceph-17.2.6-1415-g9d6d32bb/src/mds/Server.cc: In function 'CInode* Server::prepare_new_inode(MDRequestRef&, CDir*, inodeno_t, unsigned int, const file_layout_t*)' thread 7f9910df5700 time 2023-10-19T16:44:02.291873+0000
2023-10-19T16:44:02.298 INFO:tasks.ceph.mds.a.smithi084.stderr:/build/ceph-17.2.6-1415-g9d6d32bb/src/mds/Server.cc: 3449: FAILED ceph_assert(_inode->gid != (unsigned)-1)
2023-10-19T16:44:02.302 INFO:tasks.ceph.mds.a.smithi084.stderr: ceph version 17.2.6-1415-g9d6d32bb (9d6d32bb3452d5179bde2ee1cfa05df7a65f4586) quincy (stable)
2023-10-19T16:44:02.303 INFO:tasks.ceph.mds.a.smithi084.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14f) [0x7f991631c89e]
2023-10-19T16:44:02.303 INFO:tasks.ceph.mds.a.smithi084.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x27eab0) [0x7f991631cab0]
2023-10-19T16:44:02.303 INFO:tasks.ceph.mds.a.smithi084.stderr: 3: (Server::prepare_new_inode(boost::intrusive_ptr<MDRequestImpl>&, CDir*, inodeno_t, unsigned int, file_layout_t const*)+0x2791) [0x55563bec4831]
2023-10-19T16:44:02.304 INFO:tasks.ceph.mds.a.smithi084.stderr: 4: (Server::handle_client_mkdir(boost::intrusive_ptr<MDRequestImpl>&)+0x1ec) [0x55563bede88c]
2023-10-19T16:44:02.304 INFO:tasks.ceph.mds.a.smithi084.stderr: 5: (Server::dispatch_client_request(boost::intrusive_ptr<MDRequestImpl>&)+0xa1c) [0x55563bef469c]
2023-10-19T16:44:02.304 INFO:tasks.ceph.mds.a.smithi084.stderr: 6: (Server::handle_client_request(boost::intrusive_ptr<MClientRequest const> const&)+0x725) [0x55563bef5585]
2023-10-19T16:44:02.304 INFO:tasks.ceph.mds.a.smithi084.stderr: 7: (Server::dispatch(boost::intrusive_ptr<Message const> const&)+0x12c) [0x55563befa9bc]
2023-10-19T16:44:02.304 INFO:tasks.ceph.mds.a.smithi084.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr<Message const> const&)+0x3eb) [0x55563be33a4b]
2023-10-19T16:44:02.304 INFO:tasks.ceph.mds.a.smithi084.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr<Message const> const&, bool)+0x643) [0x55563be36b83]
2023-10-19T16:44:02.305 INFO:tasks.ceph.mds.a.smithi084.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr<Message const> const&)+0x5c) [0x55563be371fc]
2023-10-19T16:44:02.305 INFO:tasks.ceph.mds.a.smithi084.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x1d8) [0x55563be24b48]
2023-10-19T16:44:02.305 INFO:tasks.ceph.mds.a.smithi084.stderr: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr<Message> const&)+0x460) [0x7f99165badd0]
2023-10-19T16:44:02.305 INFO:tasks.ceph.mds.a.smithi084.stderr: 13: (DispatchQueue::entry()+0x58f) [0x7f99165b866f]
2023-10-19T16:44:02.305 INFO:tasks.ceph.mds.a.smithi084.stderr: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f991668b2f1]
2023-10-19T16:44:02.306 INFO:tasks.ceph.mds.a.smithi084.stderr: 15: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9916048609]
2023-10-19T16:44:02.306 INFO:tasks.ceph.mds.a.smithi084.stderr: 16: clone()
2023-10-19T16:44:02.306 INFO:tasks.ceph.mds.a.smithi084.stderr:*** Caught signal (Aborted) **
2023-10-19T16:44:02.306 INFO:tasks.ceph.mds.a.smithi084.stderr: in thread 7f9910df5700 thread_name:ms_dispatch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants