Change LGTM, I can't see why the new images won't build. Oh, looks like I need to give you access to this repo. Once sec...

...access granted. @mtrmac you'll need to re-push unfortunately. There's a validation check that will fail in Cirrus if I simply tell it to "go" as-is.

Thanks, re-pushed.

rawhide: No match for argument: mlocate

was retired this week (🍷 for a very old project of mine). Do we actually need that?

Nuke it and we'll find out. Do you know if there's a successor? I'm going to miss it, I use it daily at home.

Nuke it and we'll find out.

Done.

Do you know if there's a successor? I'm going to miss it, I use it daily at home.

https://fedoraproject.org/wiki/Changes/Plocate_as_the_default_locate_implementation , supposedly since F36.

Do we actually need that? (mlocate)

Git blame says it came in by 38fa0c6 which is incredibly ancient. However, the origin of that change was exclusive to podman CI. So if removing mlocate still passes podman CI I think we're good.

Note: I re-ran a few container build tasks. Looks like quay or networking flakes.

Test get_ci_vm_entrypoint seems to be failing hard:

rm 'get_ci_vm/good_repo_test/uninit_gcloud.output'
Testing: Verify mock 'gcevm' flavor main() workflow produces expected output
fail - Expected exit-code 0 but received 128 while executing mock_gcevm_workflow (output follows)
Winning lottery-number checksum: 0
gcloud --configuration=automation_images --project=automation_images compute instances create --zone=us-central1-a --image-project=automation_images --image=test-image-name --custom-cpu=0 --custom-memory=0Gb --boot-disk-size=0 --labels=in-use-by=foobar foobar-test-image-name
gcloud --configuration=automation_images --project=automation_images compute ssh --ssh-flag=-o=AddKeysToAgent=yes --force-key-file-overwrite --strict-host-key-checking=no --zone=us-central1-a root@foobar-test-image-name -- true
Cloning into '/tmp/get_ci_vm_ifMxex.tmp/var/tmp/automation_images'...
fatal: detected dubious ownership in repository at '/tmp/cirrus-ci-build/get_ci_vm/good_repo_test/.git'
To add an exception for this directory, call:

	git config --global --add safe.directory /tmp/cirrus-ci-build/get_ci_vm/good_repo_test/.git
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

This is not code touched by this PR.

@cevich any chance you could take a quick look? I'm having trouble understanding. https://github.com//containers/automation_images/blob/b7395d11fee6e977d2256b536a485fdb9811839b/get_ci_vm/test.sh#L276

This is not code touched by this PR.

Agreed, this is unrelated. Must be the result of a git fix/update. I'll take a look and work on this today, hopefully it's an easy fix.

Well...was an easy fix but hard to find the right place to implement. Local testing shows I found it, so #356 should fix this.

Done. Okay, rebase this and it should work 🤞

Rebased. Thanks @cevich .

Cirrus CI build successful. Found built image names and IDs:

• 20240523t185657z-f40f39d13
• 20240513t140131z-f40f39d13 ⇑ (built in debian: use tmpfs on /tmp + bump /tmp size on fedora #351)

Downstream tests:

• Test https://github.com/containers/automation_images/pull/355 skopeo#2340 to see whether this fixes the panic
• Test https://github.com/containers/automation_images/pull/355 podman#22820 to verify removing mlocate does not break tests

@mtrmac see my comments in your podman PR. Debian is broken, not booting.

CAUSE: new systemd (see above table)

SOLUTION:

1. wait for Debian: remove force-cgroups-v1 code #338 to merge (will take a long time because it will need podman-buildah-everything testing and I will be OOTO much of today); or
2. block systemd upgrade on debian. Example on how to do that: https://github.com//containers/automation_images/blob/afe1ced362caf3ad65fc502da6af3567de0266f3/base_images/debian_base-setup.sh#L46-L57

@mtrmac Ed's second suggestion (unfortunately) isn't abnormal for this repo. If you're at all uncomfortable making that change, poke me and I'll take care of it for you.

Edit: It looks like your workaorund is at least functional as the containers/skopeo#2340 CI passed.

Cirrus CI build successful. Found built image names and IDs:

NO GO. Repeat, NO GO. Something went wrong. systemd is still a bad version:

• 20240528t183210z-f40f39d13
• 20240513t140131z-f40f39d13 ⇑ (built in debian: use tmpfs on /tmp + bump /tmp size on fedora #351) [Ed note: same baseline as above, i.e. what's in podman now]

I can't figure out what went wrong. The log shows your pin file, then:

    debian: Unpacking libnss-resolve:amd64 (256~rc3-4) over (252.22-1~deb12u1) ...

Pinning allows globs, so maybe retry the 256 block with just 256* ? (At least I think it's globs. If it's regexps, that won't work at all). Sorry, I'm stuck.

I'd like to suggest something helpful, but I'm out of my depth here. Debugging problems in the "base" stage build can be challenging. It's not possible to use hack/get_ci_vm.sh to get anything earlier than a "base" stage VM (which is what is coming in here). The only way I know of is using the GCE Web UI or CLI to create a custom one. In this case Packer is grabbing the latest image by "family", so debian-12 according to the Makefile. In case that helps.

EXECUTIVE DECISION: I am merging this. DO NOT USE THESE VMS!!!!!!!!!!!!!!!!

I am merging because the Skopeo fixes look good, and the mlocate/plocate too, and we REALLY NEED #338 SO PLEASE NO MORE MERGES INTO THIS REPO

Ah phooey. Too late. Never mind, I'll just bring in the skopeo commit

Since you're going to be building images in the other PR right away anyway (with a newer IMG_SFX) maybe it's okay to just force-merge this?

Doesn't seem to work. Wants to rerun CI, which is going to fail.

Dang 😢

Oh! You could stick '[skip-ci]' in the title, and re-push. That will 100% bypass all of Cirrus-CI.

#338 now contains the wanted fixes from this PR, so I think this can just be closed after #338 merges.

I’m leaving it open for now just to reduce the number of changes in flight.

#338 was merged, contains the commits we want, and makes the attempts to pin systemd unnecessary.

@edsantiago @cevich thanks!

Thank you for finding a solution to the Go issue and for catching (and fixing!) the mlocate issue.