Network: add support for 'dangling' filter#31551
Network: add support for 'dangling' filter#31551vdemeester merged 1 commit intomoby:masterfrom KarthikNayak:dry_run
Conversation
There was a problem hiding this comment.
Not sure whether this part is needed.
danglingOnly is bool, which means it can be true or false.
There was a problem hiding this comment.
Makes sense, will remove that 👍
There was a problem hiding this comment.
NOTE: This is a suggestion.
I would prefer get rid of the switch block and do the following:
for _, nw := range nws {
if !runconfig.IsPreDefinedNetwork(nw.Name) && len(nw.Containers) == 0 && danglingOnly {
retNws = append(retNws, nw)
}
}I think it'll work.
There was a problem hiding this comment.
But how will this work when dangling=false. In that case we'd have to append non-dangling networks to retNws. Which is done only when, either, the network has been attached to more than a container or is a pre-defined network.
There was a problem hiding this comment.
Yeah, I knew I missed something.
What about setting a filter function instead of using a condition.
filterFunc := func(nw types.NetworkResource) bool {
if danglingOnly {
return !runconfig.IsPreDefinedNetwork(nw.Name) && len(nw.Containers) == 0
}
return runconfig.IsPreDefinedNetwork(nw.Name) || len(nw.Containers) > 0
}
...
for _, nw := range nws {
if filterFunc(nw) {
retNws = append(retNws, nw)
}
}Even though I believe it's possible to put those conditions in one line, I am afraid it won't be readable.
There was a problem hiding this comment.
That works too, but I don't see why we need to do this?
I mean the code currently tries to be consistent with the existing function filterNetworksByType, and staying consistent would mean better readability. Not that I'm against your suggestion, I just don't see any advantage in doing so.
There was a problem hiding this comment.
@vdemeester I'm ok with changing to that, but that wouldn't be consistent with the existing code, that's all.
There was a problem hiding this comment.
This does not work for swarm networks, because nw.Containers does not contain containers running on other nodes
There was a problem hiding this comment.
I see, any suggestions how to go about this?
There was a problem hiding this comment.
My opinion is to add DryRun to RemoveNetworkRequest.
https://github.com/docker/swarmkit/blob/450df1ca02727e79b4c58e8448173150f3c56c25/api/control.proto#L289-L292
There was a problem hiding this comment.
That would mean dropping this commit and re-working, i.e. adding the DryRun option to RemoveNetworkRequest and perhaps then have a this filter just call that?
There was a problem hiding this comment.
IIUC, ListTasksRequest does not work for docker run containers.
i.e.
$ docker network create --driver overlay --attachable foo
$ docker run --network foo ...
There was a problem hiding this comment.
I would expect it to work for docker run containers attached to networks. Those are represented by tasks, and I don't know of anything that filters them out in swarmkit (maybe there is something filtering them in the engine?).
There was a problem hiding this comment.
So basically this PR would be dependent on #24186, right?
There was a problem hiding this comment.
Probably yes. But it would be OK if ListTasksRequest work.
There was a problem hiding this comment.
Now that #31710 is accepted, I'll finish this soon 👍
|
I think this PR changes filters for |
|
@AkihiroSuda @aaronlehmann Do have a second look. |
There was a problem hiding this comment.
I believe these fields are provided by the gossip layer and are eventually consistent. Is it a problem if the dangling filter may provide results that are slightly out of date?
There was a problem hiding this comment.
I don't think I know enough to answer that. I'll look around.
There was a problem hiding this comment.
I don't think it should be a problem as long as the consumer knows this is potentially racey.
Trying to rm a network from this list wouldn't really pose a problem except for potentially racey behavior from the state propagation, but this problem would exist in any case.
|
How does it work with |
|
@AkihiroSuda It won't, since this is based off that patch |
|
Then please add a doc so as to clarify that |
|
I could do that, would that be ok though? @AkihiroSuda |
|
I think it is ok |
|
Yes, the situation with (e.g.) volumes is no different; a volume can be "dangling" when filtering, but there can be a race where it's being used by a container directly after that. Moving this to code review |
|
looks like it needs a rebase @KarthikNayak 😢 |
|
Already on it :) |
There was a problem hiding this comment.
add "0" and "1" ? Print->Prints>
WDYT
There was a problem hiding this comment.
will do s/Print/Prints.
About '0' and '1', I'm a little confused.
~ docker image ls --filter dangling=0
Error response from daemon: Invalid filter 'dangling=[0]'
~ docker volume ls --filter dangling=0
DRIVER VOLUME NAME
So I'm on the fence with that
There was a problem hiding this comment.
Yeah, I am just referenced to https://github.com/docker/docker/blob/master/api/swagger.yaml#L5089-L5091 for image prune dangling:
- `dangling=<boolean>` When set to `true` (or `1`), prune only
unused *and* untagged images. When set to `false`
(or `0`), all unused images are pruned.
|
ping @mavenugo @fcrisciani @thaJeztah @vieux what is the status with this one ? 👼 |
|
@vdemeester I'm still 👍 for this feature |
|
I'm still 👍 as well, but it needs a rebase and the docs, completion-scripts need to be moved to https://github.com/docker/cli @KarthikNayak |
|
I went ahead and rebased this, updated based on code comments. |
There was a problem hiding this comment.
Wonder if we should be explicit here, and add danglingOnly = false. Perhaps because I'm used to "fall-through" being the default in other languages, but I had to look twice to interpret this correctly 😊
There was a problem hiding this comment.
Yeah, I know, it's probably just me, reading;
switch {
case foo:
dofoo()
case bar:
dobar()
case baz:
default:
dosomethingelse()
}
I always tend to first read that as baz: falls through to default: (so error in this case)
There was a problem hiding this comment.
Actually; should this use filter.ExactMatch() here?
Lines 185 to 191 in 2a7388a
There was a problem hiding this comment.
I'm not sure why one would be better than the other. This filters package is pretty confusing.
There was a problem hiding this comment.
Include() is deprecated; can you change to Contains()?
There was a problem hiding this comment.
Is it a concern that when filtering for "dangling", the returned results will be different than without? (i.e. it having "detailed" information)?
Looking at buildDetailedNetworkResources(), and how we use that information during filtering (we're only interested in knowing if containers or services are attached);
moby/api/server/router/network/network_routes.go
Lines 359 to 403 in be14665
Wondering if we should have a simplified version, or at least reset the Containers and Services information afterwards, so that the information that's returned for the network is consistent.
|
This network code is awful :( We should not be dealing with libnetwork types in the API layer. |
|
@KarthikNayak looks like flaky test. |
Codecov Report
@@ Coverage Diff @@
## master #31551 +/- ##
=========================================
Coverage ? 36.45%
=========================================
Files ? 613
Lines ? 45873
Branches ? 0
=========================================
Hits ? 16722
Misses ? 26856
Partials ? 2295 |
|
I found couple of other PRs too where TestStartReturnCorrectExitCode has failed and also did try to run it 100 times on loop which this PR passed just fine. @thaJeztah @cpuguy83 can we get this one merged before its second anniversary? 😮 |
There was a problem hiding this comment.
Sorry to do this, but do you think we can alter the test to actually test for which networks should be returned? Can you also add a network that is not a built-in but is not dangling?
|
whoops; looks like this needs @KarthikNayak can you run |
Like its counterpart in images and volumes, introduce the dangling filter while listing networks. When the filter value is set to true, only networks which aren't attached to containers and aren't builtin networks are shown. When set to false, all builtin networks and networks which are attached to containers are shown. Signed-off-by: Karthik Nayak <Karthik.188@gmail.com>
Definitely! Fixed and pushed :) |
|
Thanks for implementing this feature @KarthikNayak. |
|
@albers thanks for finishing it up with the bash completion. |
Closes #30825
Like its counterpart in images and volumes, introduce the dangling
filter while listing networks. When the filter value is set to true,
only networks which aren't attached to containers and aren't builtin
networks are shown. When set to false, all builtin networks and
networks which are attached to containers are shown.
Signed-off-by: Karthik Nayak Karthik.188@gmail.com
- How I did it
Mostly by mimicking the existing code, and writing new code for supporting this feature.
- How to verify it
Either by running the unit test included, or by simply building and running
docker network ls --filter dangling=trueanddocker network ls --filter dangling=false- Description for the changelog
Network: add support for 'dangling' filter
- A picture of a cute animal (not mandatory but encouraged)
Credits: Flickr/NEKOFighter