While working on #951, I started facing:
> gpg: keyserver receive failed: No data

Indeed, https://pgp.surfnet.nl shows:
```
503 Service Unavailable
No server is available to handle this request.
```

It turns out `pgp.surfnet.nl` had posed problems in the past:
- 2023: crosstool-ng/crosstool-ng#1945
- 2021: https://unix.stackexchange.com/q/656205
- 2021: https://dev.gnupg.org/T5751

The present change proposes to switch to `keyserver.ubuntu.com` (based
on [Hockeypuck](https://hockeypuck.io)) because it's:
- well maintained and synchronized,
- already used here in the repo:
  https://github.com/DataDog/datadog-agent-buildimages/blob/018496a52eca3a52fba20a2dbdb00b3a8dad6af2/agent-deploy/Dockerfile#L118

Note: `keys.openpgp.org` would also be a fine alternative (based on
[hagrid](https://gitlab.com/keys.openpgp.org/hagrid)).

While working on #951, I started facing:
> gpg: keyserver receive failed: No data

At the time of writing, https://pgp.surfnet.nl shows:
```
503 Service Unavailable
No server is available to handle this request.
```

It turns out `pgp.surfnet.nl` had already posed problems in the past:
- 2023: crosstool-ng/crosstool-ng#1945
- 2021: https://unix.stackexchange.com/q/656205
- 2021: https://dev.gnupg.org/T5751

The present change proposes to switch to `keyserver.ubuntu.com` (based
on [Hockeypuck](https://hockeypuck.io)) because it's:
- well maintained and synchronized,
- already used here in the repo:
  https://github.com/DataDog/datadog-agent-buildimages/blob/018496a52eca3a52fba20a2dbdb00b3a8dad6af2/agent-deploy/Dockerfile#L118

Note: `keys.openpgp.org` would also be a fine alternative (based on
[hagrid](https://gitlab.com/keys.openpgp.org/hagrid)).

While working on #951, I started facing:
> gpg: keyserver receive failed: No data

At the time of writing, https://pgp.surfnet.nl shows:
```
503 Service Unavailable
No server is available to handle this request.
```

It turns out `pgp.surfnet.nl` had already posed problems in the past:
- 2023: crosstool-ng/crosstool-ng#1945
- 2021: https://unix.stackexchange.com/q/656205
- 2021: https://dev.gnupg.org/T5751

The present change proposes to switch to `keyserver.ubuntu.com` (based
on [Hockeypuck](https://hockeypuck.io)) because it's:
- well maintained and synchronized,
- already used in the repo:
  https://github.com/DataDog/datadog-agent-buildimages/blob/018496a52eca3a52fba20a2dbdb00b3a8dad6af2/agent-deploy/Dockerfile#L118

Note: `keys.openpgp.org` would also be a fine alternative (based on
[hagrid](https://gitlab.com/keys.openpgp.org/hagrid)).

While working on #951, I started facing:
> gpg: keyserver receive failed: No data

At the time of writing, https://pgp.surfnet.nl shows:
```
503 Service Unavailable
No server is available to handle this request.
```

It turns out `pgp.surfnet.nl` had already posed problems in the past:
- 2023: crosstool-ng/crosstool-ng#1945
- 2021: https://unix.stackexchange.com/q/656205
- 2021: https://dev.gnupg.org/T5751

The present change proposes to switch to `keyserver.ubuntu.com` (based
on [Hockeypuck](https://hockeypuck.io)) because it's:
- well maintained and synchronized,
- already used in the repo:
  https://github.com/DataDog/datadog-agent-buildimages/blob/018496a52eca3a52fba20a2dbdb00b3a8dad6af2/agent-deploy/Dockerfile#L118

Note: `keys.openpgp.org` would also be a fine alternative (based on
[hagrid](https://gitlab.com/keys.openpgp.org/hagrid)).

While working on #951, I started facing:
> gpg: keyserver receive failed: No data

At the time of writing, https://pgp.surfnet.nl shows:
```
503 Service Unavailable
No server is available to handle this request.
```

It turns out `pgp.surfnet.nl` had already posed problems in the past:
- 2023: crosstool-ng/crosstool-ng#1945
- 2021: https://unix.stackexchange.com/q/656205
- 2021: https://dev.gnupg.org/T5751

The present change proposes to switch to `keyserver.ubuntu.com` (based
on [Hockeypuck](https://hockeypuck.io)) because it's:
- well maintained and synchronized,
- already used in the repo:
  https://github.com/DataDog/datadog-agent-buildimages/blob/018496a52eca3a52fba20a2dbdb00b3a8dad6af2/agent-deploy/Dockerfile#L118

Note: `keys.openpgp.org` would also be a fine alternative (based on
[hagrid](https://gitlab.com/keys.openpgp.org/hagrid)).

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,source=../scripts,target=/scripts /scripts/script.sh
```
... over:
```
COPY ../scripts/script.sh /scripts
RUN /scripts/script.sh
```
... are that:
- image layers will no longer be bloated by install-only files,
- scriptlets may then be reused.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- containers/buildah#3217
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,source=../scripts,target=/scripts /scripts/script.sh
```
... over:
```
COPY ../scripts/script.sh /scripts
RUN /scripts/script.sh
```
... are that:
- image layers will no longer be bloated by install-only files,
- scriptlets may then be reused.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- containers/buildah#3217
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,src=/scripts,dst=/scripts /scripts/script.sh
```
... over:
```
COPY /scripts/script.sh /scripts
RUN /scripts/script.sh
```
... are that:
- image layers will no longer be bloated by install-only files,
- scripts may then be reused.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,src=/scripts,dst=/scripts /scripts/script.sh
```
... over:
```
COPY /scripts/script.sh /scripts
RUN /scripts/script.sh
```
... are that:
- image layers will no longer be bloated by install-only files,
- scripts may then be reused.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,src=/scripts,dst=/scripts /scripts/script.sh
```
... over:
```
COPY /scripts/script.sh /scripts
RUN /scripts/script.sh
```
... are that:
- image layers will no longer be bloated by install-only files,
- scripts may then be reused.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,dst=/mnt /mnt/setup/script.sh
```
... over:
```
COPY /setup/script.sh /
RUN /script.sh
```
... are that:
- fewer layers are built, (~COPY~)
- no layer is bloated by build-only files,
- the filesystem root is kept in a cleaner state at every stage,
- repeated invocations over `Dockerfile`s are limited to single RUN
  lines.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

While working on #951, I started facing:
> gpg: keyserver receive failed: No data

At the time of writing, https://pgp.surfnet.nl shows:
```
503 Service Unavailable
No server is available to handle this request.
```

It turns out `pgp.surfnet.nl` had already posed problems in the past:
- 2023: crosstool-ng/crosstool-ng#1945
- 2021: https://unix.stackexchange.com/q/656205
- 2021: https://dev.gnupg.org/T5751

The present change proposes to switch to `keyserver.ubuntu.com` (based
on [Hockeypuck](https://hockeypuck.io)) because it's:
- well maintained and synchronized,
- already used in the repo:
  https://github.com/DataDog/datadog-agent-buildimages/blob/018496a52eca3a52fba20a2dbdb00b3a8dad6af2/agent-deploy/Dockerfile#L118

Note: `keys.openpgp.org` would also be a fine alternative (based on
[hagrid](https://gitlab.com/keys.openpgp.org/hagrid)).

Co-authored-by: Régis Desgroppes <rdesgroppes@gmail.com>

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,dst=/mnt /mnt/setup/script.sh
```
... over:
```
COPY /setup/script.sh /
RUN /script.sh
```
... are that:
- fewer layers are built, (~`COPY`~)
- no layer is bloated by build-only files,
- the filesystem's root is kept in a cleaner state at every stage,
- repeated invocations over `Dockerfile`s are limited to single RUN
  lines.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

* Use `bind` mounts in Docker builds

This is to address an earlier review comment on #951:
#951 (comment)

From https://docs.docker.com/engine/storage/bind-mounts:
> Bind mounts are also available for builds: you can bind mount source
> code from the host into the build container to test, lint, or compile
> a project.

The advantages of:
```
RUN --mount=type=bind,dst=/mnt /mnt/setup/script.sh
```
... over:
```
COPY /setup/script.sh /
RUN /script.sh
```
... are that:
- fewer layers are built, (~`COPY`~)
- no layer is bloated by build-only files,
- the filesystem's root is kept in a cleaner state at every stage,
- repeated invocations over `Dockerfile`s are limited to single RUN
  lines.
  ([DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself))

Some more refs:
- moby/buildkit#2893
- https://stackoverflow.com/questions/72882140/docker-buildkit-new-mount-command-during-build-and-large-files-in-the-context
- https://www.bestonlinetutorial.com/docker/how-can-you-mount-host-volumes-into-docker-containers-in-a-dockerfile-during-build.html

* Set up Zsh before setting up ~/.scripts

This is to address 3 related review comments at once:
- #958 (comment)
- #958 (comment)
- #958 (comment)

It consists in moving the Zsh setup close to its Nushell peer, that is
prior to installing helper scripts.

* Honor local consistency: ~ -> "$HOME"

This is to comply with
#958 (comment),
as it turns out repo code owners prefer using "$HOME" over ~, despite
the guaranteed tilde expansion in POSIX Shells:
- https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01
- https://www.gnu.org/software/bash/manual/html_node/Tilde-Expansion.html#Tilde-Expansion-1

Now `bazelisk` install snippets benefit from a single point of
maintenance, `setup/bazel.sh`, it's time to address a duly noted review
comment on #951:
> Looks good to me, however to cut down on build times I would much
> prefer that we use the standalone binaries they release.

This is now way closer to what was done for macOS runners:
https://github.com/DataDog/ci-platform-machine-images/pull/395/files

We don't expect to bump the `bazelisk` version very often. The main
scenario where this might be required is if `bazel` deliverables stop
being published at their usual URLs. Even in that case, `bazelisk`
supports overriding the download location via the `BAZELISK_BASE_URL`
environment variable.
(https://github.com/bazelbuild/bazelisk?tab=readme-ov-file#where-does-bazelisk-get-bazel-from)

Now `bazelisk` install snippets benefit from a single point of
maintenance, `setup/bazel.sh`, it's time to address a duly noted review
comment on #951:
> Looks good to me, however to cut down on build times I would much
> prefer that we use the standalone binaries they release.

This is now way closer to what was done for macOS runners:
https://github.com/DataDog/ci-platform-machine-images/pull/395/files

We don't expect to bump the `bazelisk` version very often. The main
scenario where this might be required is if `bazel` deliverables stop
being published at their usual URLs. Even in that case, `bazelisk`
supports overriding the download location via the `BAZELISK_BASE_URL`
environment variable.
(https://github.com/bazelbuild/bazelisk?tab=readme-ov-file#where-does-bazelisk-get-bazel-from)

Now `bazelisk` install snippets benefit from a single point of
maintenance, `setup/bazel.sh`, it's time to address a duly noted review
comment on #951:
> Looks good to me, however to cut down on build times I would much
> prefer that we use the standalone binaries they release.

This is now way closer to what was done for macOS runners:
https://github.com/DataDog/ci-platform-machine-images/pull/395/files

We don't expect to bump the `bazelisk` version very often. The main
scenario where this might be required is if `bazel` deliverables stop
being published at their usual URLs. Even in that case, `bazelisk`
supports overriding the download location via the `BAZELISK_BASE_URL`
environment variable.
(https://github.com/bazelbuild/bazelisk?tab=readme-ov-file#where-does-bazelisk-get-bazel-from)

Now `bazelisk` install snippets benefit from a single point of
maintenance, `setup/bazel.sh`, it's time to address a duly noted review
comment on #951:
> Looks good to me, however to cut down on build times I would much
> prefer that we use the standalone binaries they release.

This is now way closer to what was done for macOS runners:
https://github.com/DataDog/ci-platform-machine-images/pull/395/files

We don't expect to bump the `bazelisk` version very often. The main
scenario where this might be required is if `bazel` deliverables stop
being published at their usual URLs. Even in that case, `bazelisk`
supports overriding the download location via the `BAZELISK_BASE_URL`
environment variable or its or its counterpart in `.bazeliskrc`.
(https://github.com/bazelbuild/bazelisk?tab=readme-ov-file#where-does-bazelisk-get-bazel-from)

### What does this PR do?
Now `bazelisk` is available on all our CI executors
([macOS](DataDog/ci-platform-machine-images#395)
runners,
[Linux](DataDog/datadog-agent-buildimages#951) &
[Windows](DataDog/datadog-agent-buildimages#953)
containers), this change secures an initial part of our `bazel` setup
while providing reusable job templates to ease caching in CI.

Practically, it adds a handful of jobs focusing on **building `bazel`
dependencies** in the corresponding GitLab `deps_build` stage.

Overall, it consists in:
1. verifying **`bazelisk` properly bootstraps `bazel` across all
platforms**, (primary scope of the PR)
2. extracting initial `bazel:`-prefixed CI job templates and dogfooding
them, (i.e. by building deps)
3. binding `bazelisk`/`bazel` caches to GitLab caching capabilities
(runner-based as of now): installed binaries, "repository cache", "repo
contents cache", and "disk cache" are all saved/restored correctly
to/from GitLab while honoring OS/architecture boundaries,
4. marking in-workspace cache in both `.bazelignore` and `.gitignore`,
5. adjusting code/job ownership accordingly.

### Motivation
Securing some initial `bazel` configuration in CI :
- ensures all platforms are kept in the radar over next iterations,
- prevents regressions over next iterations,
- establishes foundations for future jobs.

### Possible Drawbacks / Trade-offs
- on Windows, compilation errors made be descope dependencies being
built to **only `bzip2`**. This will be of course addressed in a
subsequent PR,
- GitLab caching is still runner-based for the time being, which can be
revisited later. (like everything)

### Additional Notes
Main addressed challenges:
- cache location conflicts[^1]: externalized "repo contents cache"
through `tools/bazel*` wrappers:
  - bazelbuild/bazel#26384
  - bazelbuild/bazel#26522
  - bazelbuild/bazel#26773
  - bazelbuild/bazel#26802
- macOS runners:
  - `bzip2` dependency: fetch from a reachable source:
    - [x] #40219
- Windows:
- `bazel` spawn strategy: fallback to a permissive strategy since
`sandboxed` is unsupported
    - [x] #40328
- `tools/bazel` wrapper: fallback to batch (`tools/bazel.bat`), since
`bash` is discouraged by `bazel` in this case and `tools/bazel.ps1`
poses detection problems,
  - long paths were supported in containers but not on runners:
    - [x] DataDog/ci-platform-machine-images#429
  - recursive symlinks: use `robocopy` instead of `copy`/`move`/`xcopy`.
  
[^1]:
> ERROR: The repo contents cache
[/path/to/datadog-agent/.cache/repo_contents] is inside the workspace
[/path/to/datadog-agent]. This can cause spurious failures. Disable the
repo contents cache with `--repo_contents_cache=`, or specify
`--repo_contents_cache=<path outside the workspace>`.