cephadm: Give iscsci a RO /lib/modules bind mounted#35141
cephadm: Give iscsci a RO /lib/modules bind mounted#35141sebastian-philipp merged 1 commit intoceph:masterfrom
Conversation
|
with this patch applied, when I deploy ceph-iscsi I can: |
The ceph iscsi container needs to be able to insert the iscsi_target_mod
but it doesn't exist in the container. for security reasons bind
mounting /lib/modules seems a little dangerous unless we can mount it
RO.
Unfortuntly the docker volume mount (-v) doesn't allow you mount
readonly, adding a `--read-only` actaully does the opposite, makes the
root on the container RO and expects you to write to the mounted volumes
(-v).
However, we get more grainular control over bind mount options if we use
`--mount`[0]. Here we can still bind mound the volume into the container,
but can also add additional options, like bind mounting RO.
This patch adds at addiontal `bind_mounts` option to the CephContainer
along side `volume_mounts`. The `bind_mounts` take a List[List[str]]:
binds = []
lib_modules = ['type=bind',
'source=/lib/modules',
'destination=/lib/modules',
'ro=true']
binds.append(lib_modules)
And this is plumbed through into cephadm. Bind_mounts only needs to be
used if you need a little more control over the mounting, otherwise the
volume_mounts are easier to use.
[0] - https://docs.docker.com/engine/reference/commandline/service_create/#add-bind-mounts-volumes-or-memory-filesystems
Fixes: https://tracker.ceph.com/issues/45252
Signed-off-by: Matthew Oliver <moliver@suse.com>
5cccfaa to
d9b5371
Compare
mgfritch
left a comment
There was a problem hiding this comment.
any ideas on what the implications might be for SELinux?
Per the doc:
The --mount flag does not allow you to relabel a volume with Z or z flags
|
Thanks. Tested and targets are created successfully. |
Good question. To be honest, I don't know. But we don't need write access only readonly, so hopefully selinux let's us do that? I guess I'll have to test it with selinux and find out |
Sweet, thanks Kiefer! |
|
@b-ranto as Teuthology doesn't test this yet, I'll need your input here. Do you think we'll run into SELinux issues? |
|
@sebastian-philipp There is a lot of going on when you are doing bind mounts inside containers so it is really hard to tell if this will produce any SELinux denials. The best way to make sure we don't get any denials would be test it, even if we only test it manually and check the |
|
yeah, there is just one minor thing:
that's why I pinged you. |
|
In that case, it would definitely be better to get this tested, SELinux can deny even |
|
OK, I can try and test it. I guess all I can do it attempt to test it on the "default" policy, because who knows what sets for selinux rules others are using. Although I guess if worse somes to worse we might be able to add something to the ceph/selinux/ |
|
I wonder if https://github.com/ceph/ceph/blob/master/selinux/ceph.te#L74-L76 already has things covered. |
|
it should have it covered. However, there are few intricacies when doing SELinux inside containers so it is safer to actually test it out. |
|
(still on my todo list) |
|
blocked by #35543 |
|
I've built up an environment and installed the reference selinux policy. Currently have it in permissive mode and watching the audit log. So far I've only "added" the iscsi daemon, I see some docker warnings pre adding (but that's probably because docker isn't covered in the reference policy). Nothing major I see initially, will look a bit closer. I'll try and use it, in case the module insert hasn't happened yet (which is where I expect the RO bindmount to fail in selinux). I thought it loaded at daemon start, but will confirm. |
|
OK, connecting the dashboard to the ceph-iscsi server and then creating a target didn't throw anything to the audit log. And I can see that the iscsi_target_mod kernel module has been loaded. So it seems loading from a RO bindmount seems fine, at least with just the selinux reference policy enabled. I'll try again with the targetted policy, now that it seems I've found: https://build.opensuse.org/package/show/security:SELinux/selinux-policy But suspect it'll be the same. |
|
OK reran with the targeted policy. Run Connect the dashboard to the running instance, go create a target. Works fine. |
mgfritch
left a comment
There was a problem hiding this comment.
lgtm! thanks for the test!
|
jenkins test dashboard backend |
|
jenkins test dashboard backend |
blocked by
The ceph iscsi container needs to be able to insert the iscsi_target_mod
but it doesn't exist in the container. for security reasons bind
mounting /lib/modules seems a little dangerous unless we can mount it
RO.
Unfortuntly the docker volume mount (-v) doesn't allow you mount
readonly, adding a
--read-onlyactaully does the opposite, makes theroot on the container RO and expects you to write to the mounted volumes
(-v).
However, we get more grainular control over bind mount options if we use
--mount[0]. Here we can still bind mound the volume into the container,but can also add additional options, like bind mounting RO.
This patch adds at addiontal
bind_mountsoption to the CephContaineralong side
volume_mounts. Thebind_mountstake a List[List[str]]:binds = []
lib_modules = ['type=bind',
'source=/lib/modules',
'destination=/lib/modules',
'ro=true']
binds.append(lib_modules)
And this is plumbed through into cephadm. Bind_mounts only needs to be
used if you need a little more control over the mounting, otherwise the
volume_mounts are easier to use.
[0] - https://docs.docker.com/engine/reference/commandline/service_create/#add-bind-mounts-volumes-or-memory-filesystems
Fixes: https://tracker.ceph.com/issues/45252
Signed-off-by: Matthew Oliver moliver@suse.com
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard backendjenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume tox