[rfe] use docker buildx for remote layer caching

Bug #2087865 reported by Michael Sherman
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
New
Undecided
Unassigned

Bug Description

The current way that kolla does builds, either with the docker or podman backends, only makes use of cache if it's already present on the build machine. It's fairly cumbersome to ensure this build cache is populated, especially with (non-buildx) docker.

I wanted to propose using the `python_on_whales` library which wraps the docker buildx CLI to add buildx support, and then use the `cache_from` and `cache_to` arguments to handle remote layer caching.

In particular, this enables a workflow where building a branch can fetch multiple cache sources, such as

1. upstream/$IMAGE:2024.1-ubuntu-jammy
2. fork/$IMAGE:2024.1-ubuntu-jammy
3. fork/$IMAGE:$git_sha_of_change

This helps ensure that even if rebuilding many images, they'll share a majority of layers with ones that are already pushed. (modulo other cache-breaking issues like https://bugs.launchpad.net/kolla/+bug/1839319)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to kolla (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/kolla/+/949230

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/kolla/+/949231

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.