cgroup: make paths available, simplify getting manager#3216
cgroup: make paths available, simplify getting manager#3216mrunalp merged 9 commits intoopencontainers:masterfrom
Conversation
e0d48ea to
da9b846
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
How many PRs until the cgroup handling is completely rewritten, it feels like this code is runc's Ship of Theseus. 😉 I'll review this once #3215 is done. |
da9b846 to
48ad4ed
Compare
48ad4ed to
5352638
Compare
| // Some v1 controllers (cpu, cpuset, and devices) expect | ||
| // cgroups.Resources to not be nil in Apply. |
There was a problem hiding this comment.
I'm not 100% sure this is the right thing to do. Maybe we should not require it.
It is required because
- cpu needs to set RT params, see
runc/libcontainer/cgroups/fs/cpu.go
Lines 26 to 29 in 79185bc
- cpuset needs to set cpus and mems in the parent cgroup (yes this is super ugly), see
runc/libcontainer/cgroups/fs/cpuset.go
Lines 150 to 166 in 79185bc
- devices need to check SkipDevices flag, see
runc/libcontainer/cgroups/fs/devices.go
Line 24 in 79185bc
Now, we can modify all the three places to allow nil (and assume RT, cpus/mems, and SkipDevices are not set), and drop this requirement. After all, it is only for some specific cases, and not always required.
Honestly I don't know which way is better.
There was a problem hiding this comment.
I think that creating a cgroup should be separated from the applying values, so my preference is to allow nil at those 3 places and remove this check.
There was a problem hiding this comment.
creating a cgroup should be separated from the applying values
Alas, with cgroup v1 it is not possible in some cases (outlined above), so as much as we want to separate create and set, we can not.
In this case, though, we can allow nil (the price to pay is to fail later on Apply in case either of CPU RT priority, cpus/mems, SkipDevices is set).
There was a problem hiding this comment.
OTOH it's easier to require non-nil Resources here, otherwise it's hard to explain later why cpu/cpuset/devices configuration has unexpectedly failed.
There was a problem hiding this comment.
Yes, let's leave it as is, unless there are other opinions.
There was a problem hiding this comment.
I think it's better to leave it as-is for the moment. The fact that we need to configure other cgroups for cpu makes me think that allowing nil Resources will make this even more ugly than it is today.
b3d89b9 to
8fb2e2c
Compare
8fb2e2c to
08b0751
Compare
| // For cgroup v1, the keys are controller/subsystem name, and the values | ||
| // are absolute filesystem paths to the appropriate cgroups. | ||
| // | ||
| // For cgroup v2, the only key allowed is "" (empty string), and the value |
There was a problem hiding this comment.
nit: Instead of "", could we use a predefined string like "unified"?
There was a problem hiding this comment.
I seriously considered doing it (about a year ago) and ultimately decided against it, mostly because changing it requires adding some backward-compatible hacks (this key and value is saved into state.json). I am not even taking external users such as kubernetes and cadvisor into account -- with those it will be a bloody mess.
Note that this is the convention that is universally used in many places in runc, I am not introducing it here, merely documenting the fact (which is also documented in some other places, for example
runc/libcontainer/cgroups/cgroups.go
Lines 38 to 46 in 147ad56
| if cg.Resources != nil && cg.Resources.Unified != nil { | ||
| return nil, cgroups.ErrV1NoUnified | ||
| } | ||
| if paths == nil { |
There was a problem hiding this comment.
Looking for potential corner-cases: is an empty map[string]string to be considered different for this code path, or should it be treated the same? (if "the same", we'd have to check for len(paths))
There was a problem hiding this comment.
Forget to reply -- I thought about that and I guess the non-nil map (even when empty) should be handled as it is right now, IOW it means "paths are set, no need to guess any". Not sure if/when this can be useful but theoretically it kind of makes sense.
1. Make Rootless and Systemd flags part of config.Cgroups. 2. Make all cgroup managers (not just fs2) return error (so it can do more initialization -- added by the following commits). 3. Replace complicated cgroup manager instantiation in factory_linux by a single (and simple) libcontainer/cgroups/manager.New() function. 4. getUnifiedPath is simplified to check that only a single path is supplied (rather than checking that other paths, if supplied, are the same). [v2: can't -> cannot] Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Separate path initialization logic from Apply to initPaths,
and call initPaths from NewManager, so:
- we can error out early (in NewManager rather than Apply);
- always have m.paths available (e.g. in Destroy or Exists).
- do not unnecessarily call subsysPath from Apply in case
the paths were already provided.
2. Add a check for non-nil cgroups.Resources to NewManager,
since initPaths, as well as some controller's Apply methods,
need it.
3. Move cgroups.Resources.Unified check from Apply to NewManager,
so we can error out early (same check exists in Set).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This is already documented but I guess more explanations (in particular, why the path is being removed from paths) won't hurt. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This way we - won't re-initialize the paths if they were provided; - will always have paths ready for every method. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
cgName and cgParent are only used when cgPath is empty, so move their cleaning to the body of the appropriate "if" statement. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This fixes the same issue as e.g. commit 4f8ccc5 but in a more universal way. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Many operations require fsMgr, so let's create it right in NewUnifiedManager and reuse. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Cgroup controllers should never panic, and yet sometimes they do. Add a unit test to check that controllers never panic when called with nil arguments and/or resources, and fix a few found cases. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It is assumed that m.config is not nil, so these checks are redundant (in case it is nil, NewManager panics and this code is unreachable). Note that cgroups/manager.New checks that config is not nil. Remove them. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
08b0751 to
99ddc1b
Compare
|
Implemented some very minor changes ( |
|
This is a big improvement in cleaning up the code. Thanks! |
|
@AkihiroSuda PTAL (this is merged but if there's anything you feel is wrong here, we can address in a followup) |
|
FWIW, LGTM. |
This
is based on and currently includes #3215.used to be part of #3131.High level overview:
NewManager) to return errors;(and now we can use a new instance for e.g.
DestroyorExists,which was not possible before);
logic of choosing a cgroup manager with a simple
manager.New()call.Resources==nil).This helps further commits, and makes cgroup manager easier to use
from e.g. Kubernetes.
Fixes: #3177
Fixes: #3178
Proposed changelog entry