Skip to content

[WIP] Use whitelist for xattrs#40087

Closed
thaJeztah wants to merge 1 commit intomoby:masterfrom
thaJeztah:support_xattrs
Closed

[WIP] Use whitelist for xattrs#40087
thaJeztah wants to merge 1 commit intomoby:masterfrom
thaJeztah:support_xattrs

Conversation

@thaJeztah
Copy link
Copy Markdown
Member

addresses #35699

Before this patch:

DOCKER_BUILDKIT=0 docker build --no-cache -<<'EOF'
FROM alpine
RUN apk add --no-cache attr
RUN touch /testfile \
 && setfattr -n user.pax.flags -v ok /testfile

RUN test "$(getfattr -n user.pax.flags --only-values testfile)" = "ok"
EOF

getfattr: testdir/testfile: No such file or directory
The command '/bin/sh -c test "$(getfattr -n user.pax.flags --only-values testdir/testfile)" = "ok"' returned a non-zero code: 1

With this patch:

DOCKER_BUILDKIT=0 docker build --no-cache -<<'EOF'
FROM alpine
RUN apk add --no-cache attr
RUN touch /testfile \
 && setfattr -n user.pax.flags -v ok /testfile

RUN test "$(getfattr -n user.pax.flags --only-values testfile)" = "ok"
EOF

Successfully built 5ecde85b467d

Signed-off-by: Sebastiaan van Stijn github@gone.nl

- What I did

- How I did it

- How to verify it

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

@thaJeztah
Copy link
Copy Markdown
Member Author

Ok, so the new test passes:

[2019-10-14T01:29:43.102Z] === RUN   TestBuildPreserveXattrs
[2019-10-14T01:29:43.102Z] === RUN   TestBuildPreserveXattrs/testdata
[2019-10-14T01:29:46.380Z] === RUN   TestBuildPreserveXattrs/original
[2019-10-14T01:29:47.751Z] === RUN   TestBuildPreserveXattrs/copy_dir
[2019-10-14T01:29:49.648Z] === RUN   TestBuildPreserveXattrs/copy_file
[2019-10-14T01:29:51.020Z] --- PASS: TestBuildPreserveXattrs (9.35s)
[2019-10-14T01:29:51.020Z]     --- PASS: TestBuildPreserveXattrs/testdata (4.57s)
[2019-10-14T01:29:51.020Z]     --- PASS: TestBuildPreserveXattrs/original (1.38s)
[2019-10-14T01:29:51.020Z]     --- PASS: TestBuildPreserveXattrs/copy_dir (1.75s)
[2019-10-14T01:29:51.020Z]     --- PASS: TestBuildPreserveXattrs/copy_file (1.64s)

But it looks like it broke other tests, all related to symlinks; e.g.;

[2019-10-14T02:00:06.632Z]     --- FAIL: TestDockerSuite/TestBuildAddToSymlinkDest (4.44s)
[2019-10-14T02:00:06.632Z]         docker_cli_build_test.go:2252: assertion failed: 
[2019-10-14T02:00:06.632Z]             Command:  /usr/local/cli/docker build -t testbuildaddtosymlinkdest .
[2019-10-14T02:00:06.632Z]             ExitCode: 1
[2019-10-14T02:00:06.632Z]             Error:    exit status 1
[2019-10-14T02:00:06.632Z]             Stdout:   Sending build context to Docker daemon  3.072kB


[2019-10-14T02:00:06.632Z]             Step 1/6 : FROM busybox
[2019-10-14T02:00:06.632Z]              ---> 6ad733544a63
[2019-10-14T02:00:06.632Z]             Step 2/6 : RUN sh -c "mkdir /foo"
[2019-10-14T02:00:06.632Z]              ---> Running in f3d102c6e830
[2019-10-14T02:00:06.632Z]             Removing intermediate container f3d102c6e830
[2019-10-14T02:00:06.632Z]              ---> 1ff27df67640
[2019-10-14T02:00:06.632Z]             Step 3/6 : RUN ln -s /foo /bar
[2019-10-14T02:00:06.632Z]              ---> Running in 924347718bb1
[2019-10-14T02:00:06.632Z]             Removing intermediate container 924347718bb1
[2019-10-14T02:00:06.632Z]              ---> 122c203210a5
[2019-10-14T02:00:06.632Z]             Step 4/6 : ADD foo /bar/
[2019-10-14T02:00:06.632Z]              ---> 6b36a814d34b
[2019-10-14T02:00:06.632Z]             Step 5/6 : RUN sh -c "[ -f /bar/foo ]"
[2019-10-14T02:00:06.632Z]              ---> Running in 146a5946ac48
[2019-10-14T02:00:06.632Z]             Removing intermediate container 146a5946ac48
[2019-10-14T02:00:06.632Z]              ---> c0cd04d98034
[2019-10-14T02:00:06.632Z]             Step 6/6 : RUN sh -c "[ -f /foo/foo ]"
[2019-10-14T02:00:06.632Z]              ---> Running in 2507fb55d3d8
[2019-10-14T02:00:06.632Z]             
[2019-10-14T02:00:06.632Z]             Stderr:   The command '/bin/sh -c sh -c "[ -f /foo/foo ]"' returned a non-zero code: 1
[2019-10-14T02:00:06.632Z]             
[2019-10-14T02:00:06.632Z]             
[2019-10-14T02:00:06.632Z]             Failures:
[2019-10-14T02:00:06.632Z]             ExitCode was 1 expected 0
[2019-10-14T02:00:06.632Z]             Expected no error
[2019-10-14T01:36:12.515Z] === Failed
[2019-10-14T01:36:12.515Z] === FAIL: amd64.integration.container TestCopyFromContainer// (0.10s)
[2019-10-14T01:36:12.515Z]     --- FAIL: TestCopyFromContainer// (0.10s)
[2019-10-14T01:36:12.515Z]         copy_test.go:159: assertion failed: expression is false: found[f]: /bar/filesymlink not found in archive
[2019-10-14T01:36:12.515Z]         copy_test.go:159: assertion failed: expression is false: found[f]: /bar/dirsymlink not found in archive
[2019-10-14T01:36:12.515Z]         copy_test.go:159: assertion failed: expression is false: found[f]: /bar/notarget not found in archive
[2019-10-14T01:36:12.515Z] 
[2019-10-14T01:36:12.515Z] === FAIL: amd64.integration.container TestCopyFromContainer//bar/root/ (0.27s)
[2019-10-14T01:36:12.515Z]     --- FAIL: TestCopyFromContainer//bar/root/ (0.27s)
[2019-10-14T01:36:12.515Z]         copy_test.go:159: assertion failed: expression is false: found[f]: root/bar/filesymlink not found in archive
[2019-10-14T01:36:12.515Z]         copy_test.go:159: assertion failed: expression is false: found[f]: root/bar/dirsymlink not found in archive
[2019-10-14T01:36:12.515Z]         copy_test.go:159: assertion failed: expression is false: found[f]: root/bar/notarget not found in archive
[2019-10-14T01:36:12.515Z] 
[2019-10-14T01:36:12.515Z] === FAIL: amd64.integration.container TestCopyFromContainer/bar/filesymlink (0.02s)
[2019-10-14T01:36:12.515Z]     --- FAIL: TestCopyFromContainer/bar/filesymlink (0.02s)
[2019-10-14T01:36:12.515Z]         copy_test.go:129: assertion failed: error is not nil: Error: No such container:path: c3027b5505f88363023b573a6b0a2dc53bec04cd8e59a46c4bb5a71420a6321b:bar/filesymlink
[2019-10-14T01:36:12.515Z] 
[2019-10-14T01:36:12.515Z] === FAIL: amd64.integration.container TestCopyFromContainer/bar/dirsymlink (0.02s)
[2019-10-14T01:36:12.515Z]     --- FAIL: TestCopyFromContainer/bar/dirsymlink (0.02s)
[2019-10-14T01:36:12.515Z]         copy_test.go:129: assertion failed: error is not nil: Error: No such container:path: c3027b5505f88363023b573a6b0a2dc53bec04cd8e59a46c4bb5a71420a6321b:bar/dirsymlink
[2019-10-14T01:36:12.515Z] 
[2019-10-14T01:36:12.515Z] === FAIL: amd64.integration.container TestCopyFromContainer/bar/dirsymlink/ (0.03s)
[2019-10-14T01:36:12.515Z]     --- FAIL: TestCopyFromContainer/bar/dirsymlink/ (0.03s)
[2019-10-14T01:36:12.515Z]         copy_test.go:129: assertion failed: error is not nil: Error: No such container:path: c3027b5505f88363023b573a6b0a2dc53bec04cd8e59a46c4bb5a71420a6321b:bar/dirsymlink/
[2019-10-14T01:36:12.515Z] 
[2019-10-14T01:36:12.515Z] === FAIL: amd64.integration.container TestCopyFromContainer/bar/notarget (0.16s)
[2019-10-14T01:36:12.515Z]     --- FAIL: TestCopyFromContainer/bar/notarget (0.16s)
[2019-10-14T01:36:12.515Z]         copy_test.go:129: assertion failed: error is not nil: Error: No such container:path: c3027b5505f88363023b573a6b0a2dc53bec04cd8e59a46c4bb5a71420a6321b:bar/notarget
[2019-10-14T01:36:12.515Z] 
[2019-10-14T01:36:12.515Z] === FAIL: amd64.integration.container TestCopyFromContainer (3.09s)

@mrueg
Copy link
Copy Markdown
Contributor

mrueg commented Jan 14, 2020

@thaJeztah do you still plan to get this in? Would be great to have this in a release. :-)

@thaJeztah
Copy link
Copy Markdown
Member Author

I can do a quick rebase of this PR, but haven't had time to look at the failing tests (to verify if the tests make the wrong assumption, or if the failures are legit)

Help on verifying that would be appreciated 🤗 (as I'm not sure when I come round to doing that) 😅

addresses moby#35699

Before this patch:

    DOCKER_BUILDKIT=0 docker build --no-cache -<<'EOF'
    FROM alpine
    RUN apk add --no-cache attr
    RUN touch /testfile \
     && setfattr -n user.pax.flags -v ok /testfile

    RUN test "$(getfattr -n user.pax.flags --only-values testfile)" = "ok"
    EOF

    getfattr: testdir/testfile: No such file or directory
    The command '/bin/sh -c test "$(getfattr -n user.pax.flags --only-values testdir/testfile)" = "ok"' returned a non-zero code: 1

With this patch:

    DOCKER_BUILDKIT=0 docker build --no-cache -<<'EOF'
    FROM alpine
    RUN apk add --no-cache attr
    RUN touch /testfile \
     && setfattr -n user.pax.flags -v ok /testfile

    RUN test "$(getfattr -n user.pax.flags --only-values testfile)" = "ok"
    EOF

    Successfully built 5ecde85b467d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@kstruczy
Copy link
Copy Markdown

I had a look at the failing tests. The problem is caused by the unix.Listxattr() function that follows symlinks. Symlink's target isn't found (different layer) and getXattrByPrefix() returns an error. I changed it to Llistxattr() - version that lists xattrs for the source file.

I also added ReadSecurityXattrToTarHeader() to CopyFileWithTar() in containerfs/archiver - that fixes the legacy builder in terms of preserving xattrs.

Besides that, I thought it may be useful to use PAXRecords from the tar header, given that Xattrs field is deprecated.

I pushed all changes here:
https://github.com/kstruczy/moby/commits/kstruczy-support-xattrs

Hope that it helps.

@thaJeztah It would be great if this pull request can make it upstream.

@thaJeztah
Copy link
Copy Markdown
Member Author

@kstruczy Thanks for looking! I don't think I'll find time soon to work on this; could you open your changes as a new PR perhaps? Feel free to "squash" my commit into your commits if needed

@AkihiroSuda
Copy link
Copy Markdown
Member

closing as this was carried as #40855

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants