This PR completely reworks the way the localstack process is started in the container.

The original motivation was to implement a way for localstack to actually exit with a proper exit code from inside the container. It turned out to be non-trivial, given several challenges:

• previously this was not possible. exiting localstack in the container did not work because of how supervisord was used. it just continued to restart localstack
• there is the exitcodesand autorestart=unexpected configuration in supervisor, but then actually terminating supervisor correctly turned out to also be non-trivial and involve hacking
• removing supervisor would have had significant implications on the "restart backdoor" introduced in 2a0a9a0
• clean shutdown of localstack subprocesses: Zombie reaping with PID 1

Solution

The localstack supervisor is a python program that behaves as follows (from the doc)

Supervisor script for managing localstack processes, acting like a mini init system tailored to
localstack. This can be used on the host or in the docker-entrypoint.sh.

The supervisor behaves as follows:
* SIGHUP to supervisor will terminate the running localstack instance and then start a new localstack instance
* SIGTERM or SIGINT to supervisor will terminate the running localstack instance and then return
* if the localstack instance exits, then the supervisor exits with the same exit code.

bin/localstack-supervisor.py is now used in docker-entrypoint.sh together with exec, which can forward the exit code of localstack without complicated traps and can properly terminate the container.

The PR also adds:

• LocalstackExit exception that indiciates during the startup procedure that localstack should
• a small main script to run localstack without the localstack CLI: localstack.runtime.main.

Effects / Implications

• Removes supervisord (introduced in bb73d56 to start localstack and the webapp in the container)
• localstack can exit with an exit code when started from the container
• The CLI is no longer used to start localstack, progressing our strategy to separate the CLI and runtime
• Significantly simplify the docker-entrypoint.sh and signal handling
• localstack will no longer be automatically restarted on failure. I argue below why I think this is fine, but can be easily introduced again
• The restart backdoor now works without brute-force killing all processes in the container. Instead, the new supervisor can be signaled using SIGHUP to cleanly restart localstack. This also works outside docker now.
• Removed /var/lib/logs/localstack_infra.log (and the stderr equivalent). The log was introduced in Allow a init.d like dir to be used to execute scripts upon startup #1018 with the docker-entrypointy.sh, but seemed unecessary with the new implementation. If people want this behavior back (i.e., the extra log file), we can introduce it in localstack-supervisor.py to write to stdout and a logfile. When starting from the CLI, we still have the main container log.
• It removes the signal handler behavior that was changed in Enable signal handlers per default for LocalStack inside a docker container #6000 - basically signals to the container are now always propagated to the localstack process. There was no obvious reason why we would want any other behavior.

On automatically restarting localstack

The original behavior I had implemented was:

* if the localstack instance exits with any of ``NO_RESTART_EXIT_CODES``, then the supervisor exits
* if the localstack instance exits with any other exit code (1, 128, ...) then we restart localstack

but i decided to remove it because it seems to me that, in most cases, when localstack terminates, there's no obvious motivation to try to restart it. in fact, i've seen instance be in an endless loop of trying to start up localstack when re-trying makes no sense (e.g., import errors).

if people need this behavior, then we can easily re-introduce it. it's a few lines of code in the localstack-supervisor.py script.

How to test

From the host:

start the supervisor:

.venv/bin/python bin/localstack-supervisor.py

to restart localstack you can:

• pkill -HUP -f "bin/localstack-supervisor.py"
• curl -X POST -H 'Content-Type: application/json' http://localhost:4566/_localstack/health -d '{"action":"restart"}'

to terminate localstack you can:

• press Ctrl+C in the terminal running the localstack-supervisor
• pkill -f "bin/localstack-supervisor.py"
• curl -X POST -H 'Content-Type: application/json' http://localhost:4566/_localstack/health -d '{"action":"kill"}'
• from anywhere within localstack, call infra.exit_infra(0)
• from within the startup procedure, raise a LocalstackExit exception. (in the future, we can write handlers that can exit localstack from an HTTP request - but there seems to be no use case for this yet)

Using docker

You can check that the subprocesses of localstack terminate correctly:

run

bin/localstack-start-docker-dev.sh

in a new terminal window, you can observe the process tree:

docker exec -it localstack_main bash -c 'watch -n1 ps aux'

You can start up some processes by running:

awslocal dynamodb list-tables
awslocal opensearch create-domain --domain-name "foobar"

Then, you can HUP the docker process, or follow any of the steps from earlier:

docker kill --signal=HUP localstack_main

TODO

• Remove docs introduced in Document DISABLE_TERM_HANDLER, update docker run command in getting-started docs#150
• Update v2 changelog to describe the change in behavior
• Document the new behavior
• adapt for localstack-pro

In the future, I would like to further separate bootstrap.py and infra.py. Where infra.py is tasked with starting the runtime itself, and bootstrap.py is associated with starting localstack from the CLI.