Throughout building steps, "pulling base image" is one of the time-consuming steps.
Recently, containerd community started "Stargz Snapshotter"(containerd/stargz-snapshotter) non-core subproject for speeding up image pulling. With this snapshotter, we can directly mount stargz-formatted (but still docker-compatible) image to nodes, which takes much shorter than downloading the image to nodes.

The latest benchmarking results show performance improvement on the "pull" operation(measured on Github Actions. the log is here).

We experimentally integrated the stargz snapshotter with Buildkit for speeding up preparing base images. Our patched version of buildkit is here. We are seeing performance improvement on building a sample image. Please see the detailed result on containerd/stargz-snapshotter#53 (comment) (log is here).

When we use the legacy base image with normal OCI Worker, time for building the sample image is the following,

real	0m16.144s
user	0m0.022s
sys	0m0.018s

When we use stargz-formatted base image with our stargz-snapshotter-integrated OCI Worker, building the sample image is much faster,

real	0m3.124s
user	0m0.012s
sys	0m0.008s

The status of the integration is WIP and we are working on the following issues for exporting image output.

• Archiving layers takes a long time and it makes the building performance lower than legacy images. It is because of the low READ performance for some files which cause fetching contents from the registry (Please also see the becnhmarking result on [WIP] Buildkit integration containerd/stargz-snapshotter#53 (comment)).
• Our integrated version of bulidkit only supports exporting type tar. It is because when we use stargz snapshotter, containerd doesn't store the image contents in the content store but buildkit needs to use these contents for exporting images.

Maybe we should selectively apply stargz snapshotter (e.g. only for non-exported stages).

Open questions

• Is this kind of concept acceptable for buildkit?
• If not, are there any considerations/feedbacks/alternatives?
• Very welcome for any ideas/suggestions to solve the issue related to exporting output images (mentioned in the above)