jenkins retest windows

jenkins test windows

jenkins test make check arm64

I don't know what's going on with the windows tests or make check arm64 tests, but I can guarantee it's not related to this PR.

I sympathize with the "no sudo", but the failures were so weird on the build slaves that I couldn't decode them. Of course I don't know now what they were exactly, but I was out of patience that day and just did what everything else does. It was never necessary on my test hosts. If you're adamant I can try to debug again.

I don't get the 'podman image pull'; is the implicit pull suboptimal somehow?

I'll fix spaces on redirects.

If you find you can't remove the sudo invocations because of the CI environment, could you replace them with a function call something like:

_sudo() {
   if [ "$UID" = "0" -o "$USE_SUDO" = "no" ]; then
        "$@"
   else
        sudo "$@"
   fi
}

so that someone invoking the script manually can choose disable sudo?

I can help with issues around rootless podman.

I'd like to see sane defaults set for the inputs - the script refuses to run without them all set.
BRANCH should maybe default to the current branch if there is one
CEPH_SHA should default to the commit that's checked out
ARCH should default to the machine's actual arch
CONTAINER_REPO_HOSTNAME and CONTAINER_REPO_ORGANIZATION could default to 'localhost' and null, respectively. If the hostname is 'localhost', we can skip pushing and therefore caring about credentials

I would actually argue that the script should be unaware of authentication, or at most re-export REGISTRY_AUTH_FILE if required. podman doesn't require the password to be submitted cleartext as a flag, and it ending up in logfiles or history files doesn't feel good to me.

Re: pulling before building, you can do it with podman build --pull=newer

@Phlogiston: sure, if non-sudo doesn't work for the builders, we can make it optional for non-builders. I hadn't really spent that much time optimizing the script for non-CI use, as you might expect, so that informs a number of the choices I made/flexibility I didn't add, but it won't hurt to have some more.

@zmc: so can you elaborate on why you want an image pull? Does FROM not check the image for "latest available"? @ktdreyer thanks for the tip if we need it

@zmc: I will set up a test without sudo and see what happened (happens).

@zmc: defaults: I had them initially, but again, focusing on CI, I wanted primarily to make sure the job failed if it didn't have enough information, and make sure I was consciously setting everything to exactly what I wanted. It's less convenient for outside-CI use, true. I'm torn on this; it's easy to add a wrapper to call build.sh with whatever defaults someone wishes, but harder to catch "oh shit I didn't specify a branch and this is going to build and push main". Undecided, more input welcome.

@zmc: auth is an interesting question. Of course when run from Jenkins the passwords will be elided, as usual, and it's sort of our pattern for all these tools/authentication mechs. I'm not opposed to the idea of external-to-build.sh auth, but jenkins needs to do it somehow, so would take another shell snippet somewhere. Maybe we could decompose it into pieces; Ken originally pushed toward "as simple as possible to invoke" for flexibility for other build systems, but maybe having a bit specific to Jenkins would allow the script to assume things like auth are taken care of somewhere else (and they would be different for, say, Tekton, one assumes).

Enough open questions here that maybe we should meet? Maybe invite Ken to our Wednesday noon meeting this week if he can make it?

A small change landed in ceph-container that we should copy here. We bumped nfs-ganesha from 5 to 6 for main and squid, and we added a new gmonitoring sub-package to the list of nfs-ganesha packages. Details in ceph/ceph-container#2235

With some offline discussion, the plan is:

1. remove sudo (retesting builds now)
2. presume that repo auth is taken care of outside build.sh; add an early test to build.sh (and perhaps an even earlier one to the jenkins job changes that enable this) to failfast if auth isn't current/valid
3. set appropriate defaults in build.sh to reduce load for direct invocation (testing and non-Jenkins use)
4. nfs-ganesha is moving to build 6; straighten that out and update the dependency before merge
5. whatever I forgot that @zmc and @ktdreyer can remind me of

Other refinements can wait for incremental changes here once we get this in-tree

@andrewschoen thanks for the look. I think --platform and --arch are of limited utility; I have to RUN a bunch of stuff inside the container and I think that needs to run on the right host to work. The manifest list image is easy enough to do; it's mostly about coordinating between the two separate builds (one for each arch) so that they're both done by the time the manifest-list build runs.

@zmc @ktdreyer @andrewschoen @phlogistonjohn I think this is ready for re-review. I'll squash all the 'review' commits once everyone's happy. I managed to get a build done and test the resulting container with the original two tests (cephadm iscsi and nfs) so I'm confident the container is OK. There are a few changes to the ceph-build PR as well; that'll happen tomorrow.

jenkins test api

There is a follow up fix in #60255.