1. Introduction

This proposal is for adding hooks support to docker daemon, currently OCI spec defined 3 kinds of hooks: prestart, poststart and poststop (see https://github.com/opencontainers/runtime-spec/blob/master/config.md#hooks) and runc has implemented this, so everything is ready, what we really need to do is expose an appropriate API from docker.

2. Scenarios

From our side we have two special requirement that better to be done with a hook support from docker:

1. We want to integrate our own network solution into docker, our own network project can support cross host communication and it manages its network interfaces and rules by itself. What we need to do is insert the existing network card created by it into the container. Also, a special need is we need to insert the nic before container CMD executed so that the user application can detect the network once it started. So prestart hook is exactly what we need.

2. We want to run systemd as pid 1 in our OS container, then we find two components from redhat: oci-systemd-hook and oci-register-machine, they can satisfy our need, with new feature of hook support. Also I read about their docker patch for this: Add dockerhooks program to run hooks under runc #17021 , the function is good, though I also agree with the idea that this is dangerous, BUT, we can find a safer way with a better UX and API.

3. Proposed Design and API

To make this happen, I have two design idea currently in my mind:

(a) new --hook-spec STRING flag for docker create and docker run

The idea is simple, allow user to specify a json file containing hooks definition, docker read the file and register the hooks in the stage of container creating, and run the hook program when start. We have already done this in our private repo in this way, it works perfectly.

For example, to support redhat oci-systemd-hook and oci-register-machine, we only need a custom hook spec file in this format:

{
        "prestart": [
            {   
                "path": "/usr/libexec/oci/hooks.d/oci-systemd-hook",
                "args": ["oci-systemd-hook", "prestart"],
                "env": ["container=runc"]
            },  
            {   
                "path": "/usr/libexec/oci/hooks.d/oci-register-machine",
                "args": ["oci-register-machine", "prestart"]
            }   
        ],  
        "poststop":[
            {   
                "path": "/usr/libexec/oci/hooks.d/oci-systemd-hook",
                "args": ["oci-systemd-hook", "poststop"],
                "env": ["container=runc"]
            },  
            {   
                "path": "/usr/libexec/oci/hooks.d/oci-register-machine",
                "args": ["oci-register-machine", "poststop"]
            }   
        ]   
}

The flag is simple and powerful, but maybe someone will be worried that it exposed too much low level stuff to user, that's why I bring a second design.

(b) new docker hook command with some subcommands

I'll list all the commands which I think necessary one by one:

All subcommands:

Usage: docker hook [COMMANDS]

Commands:
create: create a hook
inspect:  return low level information on a hook
ls:  list hooks
rm:  remove a hook

docker hook create accepts two args, HOOK-NAME and PATH, path is the binary/script path you'll execute in this hook, PATH is mandatory(see definition https://github.com/opencontainers/runtime-spec/blob/master/specs-go/config.go#L109-L115) so it should be a REQUIRED field.

Usage: docker hook create [OPTIONS] HOOK-NAME PATH

Options:
--type STRING:  the hook type(prestart, poststart, poststop), default is "prestart"
--env []STRING:  environments passed to hooks
--timeout INT:  timeout before aborting hook execution
--args []STRING:  args of the hook, default to []

docker hook inspect would inspect hook definition, docker hook list will list all the hooks, and docker hook rm will remove one/more hooks.

And for docker create, docker run, we add a new --add-hook HOOK-NAME:[extra-args] flag, to allow register a new hook into container configuration, and of course it will keep hooks created by docker(currently only libnetwork prestart hook), it will simply append new hooks after libnetwork prestart hook.

Example workflow:

# docker hook create --type prestart --env ["platform=linux"] --timeout 5 --args ["ls", "/tmp/a"]  myFirstHook /bin/ls
# docker run --add-hook myFirstHook:["/tmp/b", "/tmp/c"] busybox top
/

So before top command executed in busybox container, myFirstHook will be executed first, the hook command is ls /tmp/a /tmp/b /tmp/c. (I know this is a useless hook, just for demonstration)

# docker hook inspect myFirstHook
{
  "Name": "myFirstHook",
  "Type": "prestart",
  "Path": "/bin/ls",
  "Args":  ["ls", "/tmp/a"],
  "Timeout": 5,
  "Env": ["platform=linux"] 
}

# docker hook ls
Name          Path        Args
MyFirstHook  "/bin/ls"   ["ls", "/tmp/a"]

# docker hook rm myFirstHook
myFirstHook

# docker hook ls
Name          Path        Args

I think I should already make it clear. This will need some efforts to implement, but it hides low level structure and expose a better UX to user.

4. Reference

Other related topics: #14542