Skip to content

Get rid of historic bloat on the build pipeline #5457

@antondlr

Description

@antondlr

Description

We currently do a lot of different builds (esp. on the docker front) which slows down CI pipelines without any clear benefit.
Additionally, we publish a lot of different docker image tags as part of the build pipeline.

Tangentially related is this issue about the docker pipeline being started more often than necessary, resulting in even more CI overhead.

Present Behaviour

Release binaries are typically built with portable and modern blst config, which has been redundant since august 2023, when we introduced runtime cpu feature detection to the portable builds.

On the docker front, another flavour in built for every spec (-dev), which only includes spec-minimal

This results on these builds being released:

Binaries:
- lighthouse-v5.1.1-x86_64-apple-darwin.tar.gz
- lighthouse-v5.1.1-x86_64-apple-darwin-portable.tar.gz
- lighthouse-v5.1.1-x86_64-unknown-linux-gnu.tar.gz
- lighthouse-v5.1.1-x86_64-unknown-linux-gnu-portable.tar.gz
- lighthouse-v5.1.1-aarch64-unknown-linux-gnu.tar.gz
- lighthouse-v5.1.1-aarch64-unknown-linux-gnu-portable.tar.gz
- lighthouse-v5.1.1-x86_64-windows.tar.gz
- lighthouse-v5.1.1-x86_64-windows-portable.tar.gz

Docker: 
- v5.1.1
- v5.1.1-modern
- v5.1.1-amd64
- v5.1.1-amd64-dev
- v5.1.1-amd64-modern
- v5.1.1-amd64-modern-dev
- v5.1.1-arm64
- v5.1.1-arm64-dev
- v5.1.1-arm64-modern
- v5.1.1-arm64-modern-dev

Expected Behaviour

  • make portable the default and get rid of portable and modern tags, as this should always give you (very close to?) optimal performance, with fallback in place for older systems.
  • add the spec-minimal profile to regular docker builds and get rid of -dev
  • stop publishing architecture-specific docker images, only publish proper multiarch images
  • (build and add an aarch64 lcli image to the lcli image manifest)

The releases would thus be cut down to:

Binaries:
- lighthouse-v5.1.1-x86_64-apple-darwin.tar.gz
- lighthouse-v5.1.1-x86_64-unknown-linux-gnu.tar.gz
- lighthouse-v5.1.1-aarch64-unknown-linux-gnu.tar.gz

Docker: 
- v5.1.1

Why now?

As we have just had a hard fork, this feels like the safest timing to push out changes to releases and docker images: missing out on an update or two because of hard-coded specificity (eg. latest-amd64-unstable-dev) won't be as impactful as it would be right before a fork.
However, we should still tread carefully, eg. by keeping an eye on actual downloads (releases) and pulls (docker images).

Metadata

Metadata

Assignees

Labels

infra-cioptimizationSomething to make Lighthouse run more efficiently.v5.2.0Q2 2024

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions