takelship is a podman container runtime environment for docker with batteries included.

Project	Artifacts

Project	Pipelines

The takelship project is a general purpose framework to ship complex software setups. It does not ship the software itself but the configuration (docker compose files) and the runtime environment (a debian docker container with podman installed). The software runs in the takelship or on your host.

The configuration and data is stored in the takelship directory. It is created when running ship start [<project>]. If a project name is omitted, the default project of the takelship is started which is a forgejo server with three runners and an apt proxy.

The ship command line tool is „directory-aware“ in the sense that it starts one takelship per directory. It depends on the directory which takelship ship is talking to, use --workdir or -w to specify a different directory. Run ship ls to see all takelships and their directories.

Every directory can host a different project. You can start the same project multiple times by starting them from different directories. ship can be configured via environment variables or a takelage.yml next to the takelship directory. See tau config for available configuration keys.

These basic conditions yield some amazing advantages:

• Moderate prerequisites to run a takelship: basically docker and ruby.
• Very moderate changes to the system: one directory per project and one takelship container on the docker host.
• Complete isolation between takelship projects in different directories which do do not pollute your docker host environment.
• User friendly command line interface (ship) with sane defaults (following the convention over configuration paradigm).
• Still takelage is highly customisable, transparent, free and open source software. Build your own takelship and ship any software setup with one command!
• Well-known tools for easy backup (cp), versioning (git), update (docker pull), migration (rsync) and uninstallation (rm) of the configuration and project data are readily available.
• Fallback mode if the takelship sinks: run the docker-compose-up bash script to start the project on your host computer. Continue to use your configured takelship project data with no takelship involved.
• If we forget about the takelship and make the fallback mode the new default then ship becomes a docker-compose generator with preconfigured data.

You need Docker (Desktop or Engine) on the host to run a takelship. You have to run docker as user, not as root. It is not possible to run the takelship container with podman although (or because?) it uses itself podman to run containers.

At the moment, these setups have been tested:

• Docker Engine for Linux (Debian) (amd64 and arm64)
• Docker Desktop for Linux (Debian) (amd64)
• Docker Desktop for macOS (arm64)

(See FAQs for Docker Desktop for Linux for limitations regarding file sharing.)

If you want to use the ship command line tool (which is a wrapper for tau ship) then you need Ruby and the takeltau gem which can be installed like this:

gem install takeltau

We recommend to install ruby as a user by using the Ruby Version Manager (rvm) if you want to keep your host system clean.

To update the ship command line tool run

gem update

To update the takelwerk/takelship docker container run

ship update

To update the docker compose configuration of a takelship project just ship start it normally or run

ship project update

Add this to your bash startup files:

source <(ship completion bash)

List takelships:

ship list

List available projects:

ship project list

Start a project:

ship start [<project>]

Stop a project:

ship stop

Afterwards, you can omit the project name:

ship start

A takelship project consists of a directory containing a takelage.yml configuration file a takelship data directory next to it. So it's one directory including everything: configuration, data and the images in the cache directory.

In the takelship directory you will find a cache directory. This is where the internal takelship registry stores the downloaded docker images. The internal podman registry and the internal takelship registry are synced regularly by cron (unless you set ship_images_sync: false)

The ship start command stores the ship_default_project as well as the current port configuration in the takelage.yml.

If you like nautical language you can also use ship sail (for ship start), ship board (for ship login) and ship wreck (for ship stop).

(There is a short presentation of the TeamCity takelship-project in German language. The TeamCity takelship project includes the services of the Forgejo takelship project.)

We can start the takelship project forgejo which includes a Forgejo build server with three runners and an Apt Proxy cache server with one command:

~/takelshiptest$ ship start forgejo
The takelship takelship_xitet-folal
departed from ~/takelshiptest
ships project forgejo

localhost:50600 [docker-host docker]      (DOCKER_HOST=tcp://localhost:50600)
localhost:60261 [takelship-registry http]
localhost:56628 [forgejo-server http]     (administrator/administrator)
localhost:47015 [forgejo-server ssh]
localhost:57509 [aptproxy-server http]

When you run ship start forgejo in a directory for the first time, the ship cli queries the takelwerk/takelship container about the project. (It exposes the result via ship info takelship and notes it down in the file takelship/takelship.yml.)

Now that ship knows about the internal port configuration of the project it can dynamically choose free local ports on your host system. It notes the configuration down in the takelship.yml and will use it the next time you start the project:

---
ship_default_project: forgejo
ship_ports_aptproxy_server_http_33142: 57509
ship_ports_docker_host_docker_32375: 50600
ship_ports_forgejo_server_http_33000: 56628
ship_ports_forgejo_server_ssh_30022: 47015
ship_ports_takelship_registry_http_5555: 60261

If you don't like the port configuration then change it by editing the takelage.yml configuration file. Afterwards restart the takelship to activate the new configuration:

ship restart

Maybe you'll end up with something like this:

---
ship_default_project: forgejo
ship_ports_aptproxy_server_http_33142: 3142
ship_ports_docker_host_docker_32375: 3375
ship_ports_forgejo_server_http_33000: 3000
ship_ports_forgejo_server_ssh_30022: 3022
ship_ports_takelship_registry_http_5555: 3555

You can disable a port by setting it to 0 or nothing:

ship_ports_aptproxy_server_http_33142: 0
ship_ports_docker_host_docker_32375:

Be careful with comments:

# I will be gone after the next ship restart
ship_ports_takelship_registry_http_5555: 3555
ship_ports_takelship_registry_http_5555_my_comment: I will survive restarts

If you want ship to choose a new port configuration then delete one ore more config lines. When you change a port, all configuration scripts are updated the next time a takelship is started.

Just be sure to leave the ship_default_project in the takelage.yml or you'll end up with forgejo and its runners (which is the takelship default project) and a preconfigured docker compose project ready to run on your host. But maybe that's exactly what you want...

When you start a takelship forgejo project some pretty complex things happen. The server preinstallation, the server postinstallation and the runner preinstallation is done in the takelship but the configuration and the data end up in the takelship directory on your host. From here, you can run it on your host with docker compose.

The configuration for podman-compose which is used in the takelship differs slightly from the docker-compose one on your host. For example, the port configuration is probably different from the default configuration in the takelship.

The solution to this problem are docker-env files for each service:

takelship/compose/services/<service-name>/docker-env

There is a docker compose run script docker-compose-up for each project which includes those env-files:

takelship/compose/projects/forgejo/docker-compose-up

It will also start the takelship registry service so that you have access to your already downloaded docker images of the project.

Run the docker-compose-down script to stop the forgejo project and the takelship registry:

takelship/compose/projects/forgejo/docker-compose-down

You have used the takelship as a one-time command to create and preconfigure a docker compose project. Used this way, ship becomes a docker compose project generator.

You can find a preconfigured tea configuration file in the forgejo-server service directory:

takelship/compose/services/forgejo-server/config.yml

This file can be symlinked as your host tea configuration file:

• Linux: ~/.config/tea/config.yml
• macOS: ~/Library/Application\ Support/tea/config.yml

The podman account on the takelship has its own tea config file targeting the internal port of the forgejo server which is static.

You can create a forgejo repo (testorg/testrepo) from the command line like this:

ship command tea org create testorg
ship command tea repos create --name testrepo --owner testorg

You don't need a key to push a repo. Nonetheless, a key pair is generated for you and has already been added to the admin account:

takelship/compose/services/forgejo-server/id_ed25519.administrator
takelship/compose/services/forgejo-server/id_ed25519.administrator.pub

You can add your own key like this:

ship command fortea-add-ssh-key ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMC8y1nRdLdyFPl9h7JajWa/AA1vYLV+4j0cvTgA8h0v

The fortea-add-ssh-key script serves as an example on how to query the fabulous forgejo API. Have a look at the forgejo swagger page for all options:

http://localhost:<forgejo_port>/api/swagger

The takelship forgejo server environment variables controlling the default push behaviour are permissive:

FORGEJO__REPOSITORY__ENABLE_PUSH_CREATE_USER: 'true'
FORGEJO__REPOSITORY__ENABLE_PUSH_CREATE_ORG: 'true'
FORGEJO__REPOSITORY__DEFAULT_PUSH_CREATE_PRIVATE: 'false'

You can simply push a repo and a personal repo will be created for you. The problem is: who is you? And you probably don't want personal repos anyway.

Suppose you have a local git repo that you want to deploy as testorg/testrepo which should belong to the prefconfigured admin user of a takelship in ~/forgejo. You might do this:

# start takelship
TAKELSHIP=~/forgejo
mkdir -p $TAKELSHIP
ship -w $TAKELSHIP start

Wait until you can access the forgejo localhost website.

# create org/repo as admin user
ship -w $TAKELSHIP command tea org create testorg
ship -w $TAKELSHIP command tea repos create --name testrepo --owner testorg

# add git takelship remote
SSH_PORT=$(cat $TAKELSHIP/takelage.yml | yq .ship_ports_forgejo_server_ssh_30022)
git remote add takelship ssh://git@localhost:$SSH_PORT/testorg/testrepo.git

This way, the organization and the repository belong to the forgejo admin user as its preconfigured token is used by tea.

Now you could simply push your repo. But how does forgejo know who pushes? If there is a second forgejo user and you happen to have the public ssh key of that user in your ssh agent then forgejo might use that key. Unexpected authors in your git history will be at least confusing if your goal is a reproducible scripted environment.

You can control which ssh key is used by git by disabling the socket connection to your ssh agent, disabling the global ssh configuration file and telling git explicitly which private key file to use:

# disable ssh agent keys and use the takelship ssh key to push
SSH_AUTH_SOCK= GIT_SSH_COMMAND="ssh -F /dev/null -i $TAKELSHIP/takelship/compose/services/forgejo-server/id_ed25519.administrator" git push --set-upstream takelship main

Alternatively, you could configure different remotes with different keys in your .ssh/config.

You can try out the takelship forgejo runners by adding this to .forgejo/workflows/demo.yml in your git repo:

---
on:
  - push
  - workflow_dispatch
jobs:
  test:
    runs-on: debian
    steps:
      - run: echo All Good

When pushed, this will trigger a workflow run on a node:20-bookworm image – the act container. You don't need to use the preconfigured images as you can specify any image with the container: key word.

When you run a pipeline in forgejo in a takelship, the forgejo server, the forgejo runners and the act container all runs in the takelship. They all share the same network so the act container can checkout the code from the forgejo server.

The runner containers are configured to detect a docker host (and it detects a podman socket) but they won't mount the socket to job containers. It simply doesn't work.

The takelship is itself some kind of docker in docker but it uses podman, not docker. Although podman aims for compatibility there are limits.

Fortunately, there is a solution for this problem: when you start a takelship, the docker socket of your host (in case you use docker engine) or the socket of the docker vm on your host (in case you use docker desktop) will be mounted to /var/run/docker.sock. When you start an act container you can mount the socket to the act container and set the DOCKER_HOST environment variable.

This has two effects: if your act container starts another docker container, this container will be started on your host. And if your act container produces a local docker image it will end up on your host and not in the takelship.

An example is the forgejo workflow of the takelship. All containers run in the takelship except the last one which is started by packer on the host. When the takelship builds itself then the image ends up in you local docker registry.

The takelship comes preshipped with a single internal docker image: Distribution Registry. This is the official docker registry which has a size of less than 26 Megabytes.

When you start a takelship, a registry container is started: the takelship-registry. Its data directory is takelship/cache.

When you start a takelship project on your host with a docker-compose-up script then docker will pull a copy of the docker registry and start a takelship-registry on your host using the same data directory.

The docker-compose-up will use the env-docker environment files so that the docker-compose.yml files will point to the correct takelship-registry host.

When you manually update the docker-compose.yml files on your host then you have to set ship_update: false or your changes will be overwritten.

If you want to debug ship or takelship then invoke the ship command with --debug or -d. If you run ship start -d then first you'll see a lot of debug output by the ship command.

But ship also sets the environment variable TAKELSHIP_DEBUG=true when starting the takelship container. Now ship logs will see set -x invoked in all bash scripts which run on the takelship.

Get info how to run a takelship project:

$ docker run -it --rm takelwerk/takelship
[takelship] This is a takelwerk takelship container
[takelship] Image: takelwerk/takelship:0.1.169
[takelship] Info: https://github.com/takelwerk/takelship
[takelship] CLI: https://github.com/takelwerk/takelage-cli
[takelship] No project selected. Available projects:

== Project: all
= All services of all projects.
docker run --rm --interactive --tty --name takelship --hostname takelship --privileged --publish "127.0.0.1:32375:32375" --publish "127.0.0.1:38111:38111" --publish "127.0.0.1:33000:33000" --publish "127.0.0.1:30022:30022" --publish "127.0.0.1:33142:33142" --publish "127.0.0.1:39000:39000" --publish "127.0.0.1:35000:35000" --publish "127.0.0.1:35080:35080" --volume ./takelship:/home/podman/takelship takelwerk/takelship all

== Project: aptproxy
= APT Proxy (github.com/soulteary/apt-proxy) Provides package caching.
docker run --rm --interactive --tty --name takelship --hostname takelship --privileged --publish "127.0.0.1:32375:32375" --publish "127.0.0.1:33142:33142" --volume ./takelship:/home/podman/takelship takelwerk/takelship aptproxy

== Project: registry
= Docker registry (distribution.github.io/distribution). Registry UI (github.com/quiq/registry-ui). Provides image hosting. Provides Docker registry web interface.
docker run --rm --interactive --tty --name takelship --hostname takelship --privileged --publish "127.0.0.1:32375:32375" --publish "127.0.0.1:35000:35000" --publish "127.0.0.1:35080:35080" --volume ./takelship:/home/podman/takelship takelwerk/takelship registry

== Project: forgejo
= Forgejo Gitea fork (forgejo.org). Provides git hosting. Provides CI/CD pipelines (GitHub style). Provides image hosting. Runs with Forgejo Runners. Runs with Docker in Docker. Runs with Apt Proxy. Runs with Docker in Docker.
docker run --rm --interactive --tty --name takelship --hostname takelship --privileged --publish "127.0.0.1:32375:32375" --publish "127.0.0.1:33000:33000" --publish "127.0.0.1:30022:30022" --publish "127.0.0.1:33142:33142" --volume ./takelship:/home/podman/takelship takelwerk/takelship forgejo

== Project: teamcity
= TeamCity build server (jetbrains.com/teamcity). Provides CI/CD pipelines (JetBrains style). Runs with TeamCity Runners. Runs with Forgejo and its runners. (Mimics GitHub) Runs with Registry and its UI. (Mimics DockerHub) Runs with Apt Proxy. Runs with Portainer. Runs with Docker in Docker.
docker run --rm --interactive --tty --name takelship --hostname takelship --privileged --publish "127.0.0.1:32375:32375" --publish "127.0.0.1:38111:38111" --publish "127.0.0.1:33000:33000" --publish "127.0.0.1:30022:30022" --publish "127.0.0.1:35000:35000" --publish "127.0.0.1:35080:35080" --publish "127.0.0.1:33142:33142" --publish "127.0.0.1:39000:39000" --volume ./takelship:/home/podman/takelship takelwerk/takelship teamcity

List available container commands of a running takelship:

$ docker exec -it takelship cli
takelship command line interface:

== cli
= List available commands.

== pod
= Run as podman user.

== compose-nonroot
= Run nonroot podman-compose.

== forgejo
= Forgejo admin command line interface.

== fortea-add-ssh-key
= Add ssh key to forgejo server admin account.

Run a command as podman user:

docker exec -it takelship pod podman ps -a

Alternatively, get the same result using docker on your host:

DOCKER_HOST=tcp://localhost:32375 docker ps

Run a command as podman user in a different directory (-w or --workdir) than the default directory (/home/podman):

docker exec -it takelship pod -w /tmp "podman info > ./podman_info"
docker exec -it takelship pod cat /tmp/podman_info

Enter a login shell as podman user:

docker exec -it takelship pod shell

Become root:

docker exec -it takelship bash

The takelship project is written in five different languages:

• Ansible (provisioning, compile time)
• Jinja2 (templating, compile time)
• Python (testing, compile time)
• Bash (scripting, runtime)
• Ruby (cli, runtime)

Ansible is well suited for building complex machines like a takelship. We use molecule to create and test the ansible code. Then packer builds the docker image which is then tested in Python by using testinfra and takeltest.

We use Jinja2 to create configuration files and bash scripts in the takelship. They will be run each time the takelship starts. We favour many simple bash scripts over a few complex ones. But you can't always get what you want.

The command line tool is written in Ruby because Ruby makes programmers happy.

Don't run it in production. It's meant to be hacked.