[8.4.0] Disable the repo contents cache by default#26802
Merged
Wyverald merged 2 commits intorelease-8.4.0from Aug 20, 2025
Merged
[8.4.0] Disable the repo contents cache by default#26802Wyverald merged 2 commits intorelease-8.4.0from
Wyverald merged 2 commits intorelease-8.4.0from
Conversation
fmeum
approved these changes
Aug 20, 2025
fmeum
reviewed
Aug 20, 2025
iancha1992
approved these changes
Aug 20, 2025
3 tasks
rdesgroppes
added a commit
to DataDog/datadog-agent
that referenced
this pull request
Sep 2, 2025
### What does this PR do? Now `bazelisk` is available on all our CI executors ([macOS](DataDog/ci-platform-machine-images#395) runners, [Linux](DataDog/datadog-agent-buildimages#951) & [Windows](DataDog/datadog-agent-buildimages#953) containers), this change secures an initial part of our `bazel` setup while providing reusable job templates to ease caching in CI. Practically, it adds a handful of jobs focusing on **building `bazel` dependencies** in the corresponding GitLab `deps_build` stage. Overall, it consists in: 1. verifying **`bazelisk` properly bootstraps `bazel` across all platforms**, (primary scope of the PR) 2. extracting initial `bazel:`-prefixed CI job templates and dogfooding them, (i.e. by building deps) 3. binding `bazelisk`/`bazel` caches to GitLab caching capabilities (runner-based as of now): installed binaries, "repository cache", "repo contents cache", and "disk cache" are all saved/restored correctly to/from GitLab while honoring OS/architecture boundaries, 4. marking in-workspace cache in both `.bazelignore` and `.gitignore`, 5. adjusting code/job ownership accordingly. ### Motivation Securing some initial `bazel` configuration in CI : - ensures all platforms are kept in the radar over next iterations, - prevents regressions over next iterations, - establishes foundations for future jobs. ### Possible Drawbacks / Trade-offs - on Windows, compilation errors made be descope dependencies being built to **only `bzip2`**. This will be of course addressed in a subsequent PR, - GitLab caching is still runner-based for the time being, which can be revisited later. (like everything) ### Additional Notes Main addressed challenges: - cache location conflicts[^1]: externalized "repo contents cache" through `tools/bazel*` wrappers: - bazelbuild/bazel#26384 - bazelbuild/bazel#26522 - bazelbuild/bazel#26773 - bazelbuild/bazel#26802 - macOS runners: - `bzip2` dependency: fetch from a reachable source: - [x] #40219 - Windows: - `bazel` spawn strategy: fallback to a permissive strategy since `sandboxed` is unsupported - [x] #40328 - `tools/bazel` wrapper: fallback to batch (`tools/bazel.bat`), since `bash` is discouraged by `bazel` in this case and `tools/bazel.ps1` poses detection problems, - long paths were supported in containers but not on runners: - [x] DataDog/ci-platform-machine-images#429 - recursive symlinks: use `robocopy` instead of `copy`/`move`/`xcopy`. [^1]: > ERROR: The repo contents cache [/path/to/datadog-agent/.cache/repo_contents] is inside the workspace [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the workspace>`.
|
Why was this done? Just to preserve existing 8.x series behavior? |
Collaborator
|
The cache was buggy in 8.x and still has some issues that will be fixed in 9.0.1. |
rdesgroppes
added a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`
to `--repo_contents_cache=` (empty = disabled, matching Bazel 8.4+
default), incompatible with in-workspace cache path used with ephemeral
runners (both GitLab and GitHub actions want in-workspace cache paths).
### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory, leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```
`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0.
It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.
The present change therefore makes sure this remains disabled until
Bazel figures out a way to honor the `.cache` exclusion in
`.bazelignore`.
rdesgroppes
added a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`
to `--repo_contents_cache=` (empty = disabled, matching Bazel 8.4+
default), incompatible with in-workspace cache path used with ephemeral
runners (both GitLab and GitHub actions want in-workspace cache paths).
### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory, leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```
`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0.
It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.
The present change therefore makes sure this remains disabled until
Bazel figures out a way to honor the `.cache` exclusion in
`.bazelignore`.
rdesgroppes
added a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 23, 2026
…nt cache
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`:
- `--config=ci` is now also injected on GitHub Actions (previously the
outer condition excluded GitHub Actions entirely, preventing the
`--repo_contents_cache=` neutralization below from ever firing there)
- `--repo_contents_cache=` (empty = disabled) is injected whenever
`XDG_CACHE_HOME` has fallen back to the in-repo `.cache` default,
meaning no user-configured persistent cache is available
### Motivation
`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0 (PR
bazelbuild/bazel#27706, cherry-picked as bazelbuild/bazel#27898) — it
had been disabled by default (PR bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory and is discarded at the end of the job, defeating its
purpose while adding overhead. Passing `--repo_contents_cache=` mirrors
what Bazel 8.4.0 did at the flag level.
### Describe how you validated your changes
CI passes on both GitLab (DDCI) and GitHub Actions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rdesgroppes
added a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 23, 2026
…nt cache
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`:
- `--config=ci` is now also injected on GitHub Actions (previously the
outer condition excluded GitHub Actions entirely, preventing the
`--repo_contents_cache=` neutralization below from ever firing there)
- `--repo_contents_cache=` (empty = disabled) is injected whenever
`XDG_CACHE_HOME` has fallen back to the in-repo `.cache` default,
meaning no user-configured persistent cache is available
### Motivation
`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0 (PR
bazelbuild/bazel#27706, cherry-picked as bazelbuild/bazel#27898) — it
had been disabled by default (PR bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory and is discarded at the end of the job, defeating its
purpose while adding overhead. Passing `--repo_contents_cache=` mirrors
what Bazel 8.4.0 did at the flag level.
### Describe how you validated your changes
CI passes on both GitLab (DDCI) and GitHub Actions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rdesgroppes
added a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`
to `--repo_contents_cache=` (empty = disabled, matching Bazel 8.4+
default), incompatible with in-workspace cache path used with ephemeral
runners (both GitLab and GitHub actions want in-workspace cache paths).
### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory, leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```
`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0.
It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.
The present change therefore makes sure this remains disabled until
Bazel figures out a way to honor the `.cache` exclusion in
`.bazelignore`.
gh-worker-dd-mergequeue-cf854d bot
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).
### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```
`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.
It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.
The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.
Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
aya-shimizu24
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 24, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).
### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```
`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.
It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.
The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.
Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
StephenWakely
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 27, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).
### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```
`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.
It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.
The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.
Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
StephenWakely
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Mar 27, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).
### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```
`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.
It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.
The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.
Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cc @fmeum @meteorcloudy