Description

Hello,

I ran into an issue where file extended attributes inside docker images are not persisted after docker push.

Steps to reproduce the issue:

1. Here's a simple Dockerfile I'm using to test this:

   FROM ubuntu:latest
   WORKDIR /test
   RUN apt-get update && apt-get install -yqq --no-install-recommends attr ca-certificates curl && apt-get clean  && rm -rf /var/lib/apt/lists/*
   RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.17.0/bin/linux/amd64/kubectl && chmod +x kubectl
   RUN setfattr -n user.pax.flags -v "testing" kubectl

2. build it using docker kit DOCKER_BUILDKIT=1 docker build --pull -t docker.io/serg1i89/test-xattr:test .:

      ○ → DOCKER_BUILDKIT=1 docker build --pull -t docker.io/serg1i89/test-xattr:test .
   [+] Building 22.0s (9/9) FINISHED                                                                                                                        
    => [internal] load build definition from Dockerfile                                                                                                0.0s
    => => transferring dockerfile: 397B                                                                                                                0.0s
    => [internal] load .dockerignore                                                                                                                   0.0s
    => => transferring context: 2B                                                                                                                     0.0s
    => [internal] load metadata for docker.io/library/ubuntu:latest                                                                                    1.2s
    => [1/5] FROM docker.io/library/ubuntu:latest@sha256:250cc6f3f3ffc5cdaa9d8f4946ac79821aafb4d3afc93928f0de9336eba21aa4                              4.0s
    => => resolve docker.io/library/ubuntu:latest@sha256:250cc6f3f3ffc5cdaa9d8f4946ac79821aafb4d3afc93928f0de9336eba21aa4                              0.0s
    => => sha256:4c1d20cdee96111c8acf1858b62655a37ce81ae48648993542b7ac363ac5c0e5 35.36kB / 35.36kB                                                    0.4s
    => => sha256:0d3160e1d0de4061b5b32ee09af687b898921d36ed9556df5910ddc3104449cd 854B / 854B                                                          0.3s
    => => sha256:250cc6f3f3ffc5cdaa9d8f4946ac79821aafb4d3afc93928f0de9336eba21aa4 1.42kB / 1.42kB                                                      0.0s
    => => sha256:2695d3e10e69cc500a16eae6d6629c803c43ab075fa5ce60813a0fc49c47e859 1.15kB / 1.15kB                                                      0.0s
    => => sha256:549b9b86cb8d75a2b668c21c50ee092716d070f129fd1493f95ab7e43767eab8 3.41kB / 3.41kB                                                      0.0s
    => => sha256:2746a4a261c9e18bfd7ff0429c18fd7522acc14fa4c7ec8ab37ba5ebaadbc584 26.69MB / 26.69MB                                                    1.6s
    => => sha256:c8e37668deea784f47c8726d934adc12b8d20a2b1c50b0b0c18cb62771cd3684 163B / 163B                                                          0.5s
    => => extracting sha256:2746a4a261c9e18bfd7ff0429c18fd7522acc14fa4c7ec8ab37ba5ebaadbc584                                                           1.3s
    => => extracting sha256:4c1d20cdee96111c8acf1858b62655a37ce81ae48648993542b7ac363ac5c0e5                                                           0.0s
    => => extracting sha256:0d3160e1d0de4061b5b32ee09af687b898921d36ed9556df5910ddc3104449cd                                                           0.0s
    => => extracting sha256:c8e37668deea784f47c8726d934adc12b8d20a2b1c50b0b0c18cb62771cd3684                                                           0.0s
    => [2/5] WORKDIR /test                                                                                                                             0.1s
    => [3/5] RUN apt-get update && apt-get install -yqq --no-install-recommends attr ca-certificates curl && apt-get clean  && rm -rf /var/lib/apt/l  12.1s
    => [4/5] RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.17.0/bin/linux/amd64/kubectl && chmod +x kubectl                2.9s 
    => [5/5] RUN setfattr -n user.pax.flags -v "testing" kubectl                                                                                       1.1s 
    => exporting to image                                                                                                                              0.4s 
    => => exporting layers                                                                                                                             0.4s 
    => => writing image sha256:0b70d2aae02091ece2520a65217a431f6dc7609d1d1399da6934232ca54ffa23                                                        0.0s 
    => => naming to docker.io/serg1i89/test-xattr:test                                                                                                 0.0s 
   
   

3. Checking getfattr right after build confirms it to works as expected:

○ → docker run --rm docker.io/serg1i89/test-xattr:test getfattr -n user.pax.flags /test/kubectl

# file: test/kubectl
user.pax.flags="testing"

getfattr: Removing leading '/' from absolute path names

4. However, when I push image to docker registry, remove it locally and pull it back on this or any other machine attributes are not there anymore:

   ○ → docker run --rm docker.io/serg1i89/test-xattr:test getfattr -n user.pax.flags /test/kubectl
   Unable to find image 'serg1i89/test-xattr:test' locally
   test: Pulling from serg1i89/test-xattr
   2746a4a261c9: Pull complete 
   4c1d20cdee96: Pull complete 
   0d3160e1d0de: Pull complete 
   c8e37668deea: Pull complete 
   d172907fd527: Pull complete 
   84e9f3fefd1c: Pull complete 
   f52cce342e49: Pull complete 
   Digest: sha256:759d068d68f8ce01493c86d8b3bd432925b50d5d8319daf5ca4723aa664775c3
   Status: Downloaded newer image for serg1i89/test-xattr:test
   /test/kubectl: user.pax.flags: No such attribute
   

Describe the results you received:
After docker push/pull previously set extended attributes disappear from files.

Describe the results you expected:
Attributes should be available.

Additional information you deem important (e.g. issue happens only occasionally):
I have tested this on both MacOs Mojave 10.14.6 with overlay2 and on Ubuntu 19.10 with btrfs and overlay2.
I also have tried building images with DOCKER_BUILDKIT=1 and pushing/pulling with and without this variable.
as for the docker-registry versions, this is reproducible on both hub.docker.com and on local docker-registry.

Output of docker version:

○ → docker version
Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:22:34 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea
  Built:            Wed Nov 13 07:29:19 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

○ → docker system info
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 19.03.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.9.184-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 5
 Total Memory: 8.759GiB
 Name: docker-desktop
 ID: 7GPO:HSIH:QMXP:QSUW:X4EF:WAUV:HNWA:RCK5:V3ML:KDRI:TPDD:6QSM
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 30
  Goroutines: 46
  System Time: 2020-01-14T22:37:09.3648853Z
  EventsListeners: 2
 HTTP Proxy: gateway.docker.internal:3128
 HTTPS Proxy: gateway.docker.internal:3129
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical devices running MacOs and Ubuntu linux