Skip to content

Unable to enable/disable authZ plugins via the plugin APIs #31836

New issue
New issue
Closed
Closed
Unable to enable/disable authZ plugins via the plugin APIs#31836
Assignees
anusha-ragunathan
Labels
area/pluginsversion/17.03
Milestone
17.04.0
@rogaha

Description

@rogaha
rogaha
opened on Mar 15, 2017

Description

I'm trying to disable an authZ plugin using the docker CLI, but the daemon is still routing the events to the stopped plugin. The docker-engine doesn't work after disabling the plugin -- it requires a manual removal.

Steps to reproduce the issue:

  1. docker plugin install index-stage.docker.io/rogaha/rogaha-auth-plugin:latest
  2. add --authorization-plugin=index-stage.docker.io/rogaha/rogaha-auth-plugin:latest to the dockerd parameters (I modified the /etc/init.d/docker in Moby)
  3. restart the docker service
  4. docker plugin disable index-stage.docker.io/rogaha/rogaha-auth-plugin:latest
  5. Try any docker command and it will hang trying to reach the disabled authz plugin.

Describe the results you received:

Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.254597975Z" level=debug msg="Calling POST /v1.26/plugins/index-stage.docker.io/rogaha/rogaha-auth-plugin:latest/disable"
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.254673208Z" level=debug msg="AuthZ request using plugin index-stage.docker.io/rogaha/rogaha-auth-plugin:latest"
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27Z" level=info msg="time=\"2017-03-15T01:15:27Z\" level=info msg=\"{User: UserAuthNMethod: RequestMethod:POST RequestURI:/v1.26/plugins/index-stage.docker.io/rogaha/rogaha-auth-plugin:latest/disable RequestBody:[] RequestHeaders:map[Content-Length:0 Content-Type:text/plain User-Agent:Docker-Client/17.03.0-ce (darwin)] RequestPeerCertificates:[] ResponseStatusCode:0 ResponseBody:[] ResponseHeaders:map[]}\" " plugin=90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.377523224Z" level=debug msg="containerd: process exited" id=90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548 pid=init status=2 systemPid=6723
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.382569366Z" level=debug msg="libcontainerd: received containerd event: &types.Event{Type:\"exit\", Id:\"90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548\", Status:0x2, Pid:\"init\", Timestamp:(*timestamp.Timestamp)(0xc421032900)}"
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.382763034Z" level=debug msg="plugin state changed 90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548 libcontainerd.StateInfo{CommonStateInfo:libcontainerd.CommonStateInfo{State:\"exit\", Pid:0x0, ExitCode:0x2, ProcessID:\"\"}, OOMKilled:false}"
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.382832644Z" level=debug msg="Clean shutdown of plugin"
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.397478862Z" level=debug msg="AuthZ response using plugin index-stage.docker.io/rogaha/rogaha-auth-plugin:latest"
Mar 15 01:15:27 moby root: time="2017-03-15T01:15:27.397734817Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock/AuthZPlugin.AuthZRes: Post http://%2Frun%2Fdocker%2Fplugins%2F90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548%2Frogaha.sock/AuthZPlugin.AuthZRes: dial unix /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock: connect: no such file or directory, retrying in 1s"
Mar 15 01:15:28 moby root: time="2017-03-15T01:15:28.399704517Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock/AuthZPlugin.AuthZRes: Post http://%2Frun%2Fdocker%2Fplugins%2F90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548%2Frogaha.sock/AuthZPlugin.AuthZRes: dial unix /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock: connect: no such file or directory, retrying in 2s"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.526485973Z" level=info msg="agent: found node update" module="node/agent"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.526683662Z" level=debug msg="agent: rebuild session" module="node/agent"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.526842431Z" level=debug msg="(*session).start" module="node/agent"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.533631110Z" level=debug msg="node status updated" method="(*Dispatcher).processUpdates" module=dispatcher node.id=3ae4dpdznkbm5x3n1e2qtomai
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.539761085Z" level=debug method="(*session).logSubscriptions" module="node/agent" session.id=q77l0f8yew6xp7jvedb5epbhs
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.539831626Z" level=debug msg="(*session).heartbeat" module="node/agent" session.id=q77l0f8yew6xp7jvedb5epbhs
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.539889048Z" level=debug method="(*session).watch" module="node/agent" session.id=q77l0f8yew6xp7jvedb5epbhs
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.539920683Z" level=debug msg="(*session).listen" module="node/agent" session.id=q77l0f8yew6xp7jvedb5epbhs
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.539959765Z" level=debug msg="agent: registered" module="node/agent"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.540092862Z" level=debug method="(*Dispatcher).Assignments" node.id=3ae4dpdznkbm5x3n1e2qtomai node.session=q77l0f8yew6xp7jvedb5epbhs
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.540429606Z" level=debug msg="node registered" method="(*LogBroker).ListenSubscriptions" node=3ae4dpdznkbm5x3n1e2qtomai
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541141613Z" level=debug msg="(*worker).Assign" len(assignments)=5 module="node/agent"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541170272Z" level=debug msg="(*worker).reconcileSecrets" len(removedSecrets)=0 len(updatedSecrets)=0 module="node/agent"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541215015Z" level=debug msg="(*worker).reconcileTaskState" len(removedTasks)=0 len(updatedTasks)=5 module="node/agent"
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541302834Z" level=debug msg=assigned module="node/agent" task.desiredstate=SHUTDOWN task.id=48kg3574enk1atvdc5wr5qgie
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541389724Z" level=debug msg=assigned module="node/agent" task.desiredstate=SHUTDOWN task.id=q2xu69kuakvrl64tq8zpsxafg
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541587333Z" level=debug msg=assigned module="node/agent" task.desiredstate=RUNNING task.id=q9i05l9vp1yy0cmg8xius5iyj
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541646601Z" level=debug msg=assigned module="node/agent" task.desiredstate=SHUTDOWN task.id=qsbcp55lhnp3mr8r32bfuz2xi
Mar 15 01:15:29 moby root: time="2017-03-15T01:15:29.541669671Z" level=debug msg=assigned module="node/agent" task.desiredstate=SHUTDOWN task.id=zxne6z25vc15wnvhxe02j20lw
Mar 15 01:15:30 moby root: time="2017-03-15T01:15:30.400743335Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock/AuthZPlugin.AuthZRes: Post http://%2Frun%2Fdocker%2Fplugins%2F90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548%2Frogaha.sock/AuthZPlugin.AuthZRes: dial unix /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock: connect: no such file or directory, retrying in 4s"
Mar 15 01:15:34 moby root: time="2017-03-15T01:15:34.405922304Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock/AuthZPlugin.AuthZRes: Post http://%2Frun%2Fdocker%2Fplugins%2F90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548%2Frogaha.sock/AuthZPlugin.AuthZRes: dial unix /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock: connect: no such file or directory, retrying in 8s"
Mar 15 01:15:42 moby root: time="2017-03-15T01:15:42.410673503Z" level=error msg="AuthZResponse for POST /v1.26/plugins/index-stage.docker.io/rogaha/rogaha-auth-plugin:latest/disable returned error: plugin index-stage.docker.io/rogaha/rogaha-auth-plugin:latest failed with error: Post http://%2Frun%2Fdocker%2Fplugins%2F90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548%2Frogaha.sock/AuthZPlugin.AuthZRes: dial unix /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock: connect: no such file or directory"
Mar 15 01:15:42 moby root: time="2017-03-15T01:15:42.410850409Z" level=error msg="Handler for POST /v1.26/plugins/index-stage.docker.io/rogaha/rogaha-auth-plugin:latest/disable returned error: plugin index-stage.docker.io/rogaha/rogaha-auth-plugin:latest failed with error: Post http://%2Frun%2Fdocker%2Fplugins%2F90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548%2Frogaha.sock/AuthZPlugin.AuthZRes: dial unix /run/docker/plugins/90f69d63bc4a0f2f7317ef4a5ccec3741c9bc68266cdd87fe63781d6c733d548/rogaha.sock: connect: no such file or directory"

Describe the results you expected:

  • Enable/disable the plugin without having to restart the daemon using the plugin's API (e.g. docker plugin disable ...)

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client:
 Version:      17.03.0-ce
 API version:  1.26
 Go version:   go1.7.5
 Git commit:   60ccb22
 Built:        Thu Feb 23 10:40:59 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.03.0-ce
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   3a232c8
 Built:        Tue Feb 28 07:52:04 2017
 OS/Arch:      linux/amd64
 Experimental: true

Output of docker info:

Containers: 30
 Running: 1
 Paused: 0
 Stopped: 29
Images: 136
Server Version: 17.03.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Authorization: index-stage.docker.io/rogaha/rogaha-auth-plugin:latest
Swarm: active
 NodeID: 3ae4dpdznkbm5x3n1e2qtomai
 Is Manager: true
 ClusterID: o0sxc2wbd0pcpg7zoezvy341w
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: 192.168.65.2
 Manager Addresses:
  192.168.65.2:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 977c511eda0925a723debdc94d09459af49d082a
runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.12-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952 GiB
Name: moby
ID: KICG:OKKZ:44AF:Q665:VP5O:6Q4L:R5JP:7Z2C:R64P:SY5U:67HK:6KII
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 45
 Goroutines: 148
 System Time: 2017-03-15T13:20:50.315011712Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Username: rogaha
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

Metadata

Metadata

Assignees

Labels

area/pluginsversion/17.03

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions