Skip to content

Clean up and rationalize use of Docker images in build scripts #7297

@creachadair

Description

@creachadair

This is a follow-up to #7272, which addressed the proto builder specifically.

The Makefiles and CI workflows for several tendermint repositories rely on pre-built Docker images stored on Dockerhub under the tendermintdev repository. The credentials for that repository have been lost, which complicates updates to those images. More generally, we have cases where use of pre-baked images complicates fixing bugs and rolling out new features. See tendermint/tm-db#203 for a recent case in point.

Proposed solution

I recommend a two-pronged approach:

  1. Where possible, get rid of pre-baked images from our build and CI steps, and instead build the images we need as part of the CI workflow. Judicious use of buildx caching can help reduce the expense of complex images that don't change often.

  2. Where (1) is not possible, move images to ghcr.io (a.k.a. GitHub packages) and tie them to our existing GitHub permissions rather than separate credentials.

For (2), we should probably also have a separate manual workflow for updating the images. Right now, there are some places where updates to the image are gated by the same CI that requires the image to pass, and that makes updating both complicated.

To the extent possible, we should try to have a build and testing setup that is easy to replicate locally, without having to emulate GitHub Actions. It is possible to run Actions locally, but it's heavyweight and tedious to set up, and our CI is already a meaningful impediment to velocity.

Background: Images mentioned in Actions workflow configs

This is probably incomplete.

Images from DockerHub

Most of these are built from Dockerfiles in https://github.com/tendermint/images.

tendermintdev repository

Many of these do not seem to be regularly updated, and could maybe be ignored.

  • tendermintdev/docker-build-proto
  • tendermintdev/docker-docs-deployment
  • tendermintdev/docker-hashicorp-go
  • tendermintdev/docker-protoc
  • tendermintdev/docker-tendermint-build
  • tendermintdev/docker-tm-db-testing
  • tendermintdev/docker-website-deployment
  • tendermintdev/docker-website-deployment-yarn
  • tendermintdev/docs_deployment
  • tendermintdev/ethermint
  • tendermintdev/gaia_sim
  • tendermintdev/iam_management
  • tendermintdev/jq_curl
  • tendermintdev/rbuilder
  • tendermintdev/rosetta-cli
  • tendermintdev/sdk-proto-gen
  • tendermintdev/tendermint
  • tendermintdev/terragrunt
  • tendermintdev/voyager_node
  • tendermintdev/voyager_node_browser
  • tendermintdev/website_deployment
  • tendermintdevelopment/starport
  • tendermintdevelopment/testchain

tendermint repository

  • tendermint/abacas
  • tendermint/devdoc
  • tendermint/ethermint
  • tendermint/gaia
  • tendermint/gaiadnode
  • tendermint/kms
  • tendermint/localnode
  • tendermint/tendermint
  • tendermint/testing
  • tendermint/tm-signer-harness

Metadata

Metadata

Assignees

No one assigned

    Labels

    T:jankType Jank! Non-urgent but still high-impact fixes.stalefor use by stalebot

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions