Test & Fix build with rm/force-rm matrix#35139
Conversation
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
nit: maybe reuse the context for all the calls ? 🤔
thaJeztah
left a comment
There was a problem hiding this comment.
LGTM, but I'd like @tonistiigi to have a look as well 👍
dc4cffc to
9361b6c
Compare
tonistiigi
left a comment
There was a problem hiding this comment.
@andrewhsu We should take this for next v17.10 RC. lmk if you need cherry-pick prs.
cc @dnephin to check the test
builder/dockerfile/builder.go
Outdated
There was a problem hiding this comment.
There is another ForceRemove defer inside the dispatch function
moby/builder/dockerfile/evaluator.go
Line 56 in d9cd40d
There was a problem hiding this comment.
Good catch @tonistiigi. I'll put remove handling in the global dispatch function.
|
@tonistiigi I centralized intermediate container removal logic in a single defer block in the dispatch() function with proper comments. |
87680a5 to
f420a43
Compare
builder/dockerfile/evaluator.go
Outdated
There was a problem hiding this comment.
Perhaps making these two separate conditions (small bit of duplication, but may be better to understand without having to add comments to explain)
if d.builder.options.ForceRemove {
d.builder.containerManager.RemoveAll(d.builder.Stdout)
return
}
if d.builder.options.Remove && err == nil {
d.builder.containerManager.RemoveAll(d.builder.Stdout)
return
}One thing I was wondering; it --rm=false, should --force-remove still remove containers? Can you check on an older version?
If that's the case, then it should likely look like:
if !d.builder.options.Remove {
return
}
if err == nil || d.builder.options.ForceRemove {
d.builder.containerManager.RemoveAll(d.builder.Stdout)
}There was a problem hiding this comment.
@thaJeztah on options deserialization, there is a reconcilation that forces rm to true if force-rm is true:
if httputils.BoolValue(r, "forcerm") && versions.GreaterThanOrEqualTo(version, "1.12") {
options.Remove = true
} else if r.FormValue("rm") == "" && versions.GreaterThanOrEqualTo(version, "1.12") {
options.Remove = true
} else {
options.Remove = httputils.BoolValue(r, "rm")
}
|
@thaJeztah condition expression split :) |
dnephin
left a comment
There was a problem hiding this comment.
thanks, fix looks good, left some suggestions for the tests
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
You can use https://golang.org/pkg/testing/#T.Run so that all test cases run even if one of the require fail.
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
Maybe TestBuildWithRemoveAndForceRemove or something like that? Intermediate containers seem like more of the test result than the test condition. And this tests both failures and non-failures, so failures doesn't seem relevant to the name.
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
If you use t.Run() you can use this extra string as the test name so that any assert/require failure has this extra information associated with it, instead of just the last assert.Equal()
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
This will make it impossible to run the test in parallel with any other test that creates containers.
Instead of counting the total number of containers I think it would be possible to parse the build output and grab the container ids. Then attempt to list or inspect the containers with those ids to see if they exist.
|
LGTM |
thaJeztah
left a comment
There was a problem hiding this comment.
one nit in the code; also can you check if some of the commits should be squashed?
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
Can you remove the blank lines between the github.com... dependencies, and make sure they're sorted correctly?
import (
"archive/tar"
"bytes"
"context"
"encoding/json"
"io"
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/integration/util/request"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/stretchr/testify/require"
)
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
These names could be a lot more descriptive. The dashes can also be spaces, testing.Run() will take care of adding the dashes.
- "successful build with no removal"
- "successful build with remove"
- "successful build with remove and force remove"
- "failed build with no removal"
...
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
this cleanup will be missed if require.NoError(t, err) fails the test, could be deferred.
integration/build/build_test.go
Outdated
There was a problem hiding this comment.
parsing of ids from the response would be great in a separate function to keep the test more readable.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
4473173 to
172e73a
Compare
|
@dnephin, @thaJeztah, latest nits are fixed, everything squashed and rebased |
|
powerpc failures mostly look like flakes, but the busybox missing flag is weird. I'll try restarting it. |
fix #35116
- What I did
Added test cases for build related to rm/force-rm flags, and fixed a few related issues
- How I did it
Added a test with a case matrix in integration/build reproducing potential issues and fixed them in build code base
- How to verify it
run the tests :)
Note:
docker build
rmandforce-rmflags might be better documented:It is unclear that on build failures, rm still remove successfull individual intermediate containers (only the failing one is kept).