Skip to content

[8.4.0] Disable the repo contents cache by default#26802

Merged
Wyverald merged 2 commits intorelease-8.4.0from
wyv-840-disable-repo-contents-cache
Aug 20, 2025
Merged

[8.4.0] Disable the repo contents cache by default#26802
Wyverald merged 2 commits intorelease-8.4.0from
wyv-840-disable-repo-contents-cache

Conversation

@Wyverald
Copy link
Copy Markdown
Member

@Wyverald Wyverald requested a review from a team as a code owner August 20, 2025 19:08
@github-actions github-actions bot added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. awaiting-review PR is awaiting review from an assigned reviewer labels Aug 20, 2025
@Wyverald Wyverald enabled auto-merge August 20, 2025 19:26
@Wyverald Wyverald added this pull request to the merge queue Aug 20, 2025
@iancha1992 iancha1992 added this to the 8.4.0 release blockers milestone Aug 20, 2025
Merged via the queue into release-8.4.0 with commit 6dfb2d6 Aug 20, 2025
49 checks passed
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Aug 20, 2025
@Wyverald Wyverald deleted the wyv-840-disable-repo-contents-cache branch August 20, 2025 20:12
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Sep 2, 2025
### What does this PR do?
Now `bazelisk` is available on all our CI executors
([macOS](DataDog/ci-platform-machine-images#395)
runners,
[Linux](DataDog/datadog-agent-buildimages#951) &
[Windows](DataDog/datadog-agent-buildimages#953)
containers), this change secures an initial part of our `bazel` setup
while providing reusable job templates to ease caching in CI.

Practically, it adds a handful of jobs focusing on **building `bazel`
dependencies** in the corresponding GitLab `deps_build` stage.

Overall, it consists in:
1. verifying **`bazelisk` properly bootstraps `bazel` across all
platforms**, (primary scope of the PR)
2. extracting initial `bazel:`-prefixed CI job templates and dogfooding
them, (i.e. by building deps)
3. binding `bazelisk`/`bazel` caches to GitLab caching capabilities
(runner-based as of now): installed binaries, "repository cache", "repo
contents cache", and "disk cache" are all saved/restored correctly
to/from GitLab while honoring OS/architecture boundaries,
4. marking in-workspace cache in both `.bazelignore` and `.gitignore`,
5. adjusting code/job ownership accordingly.

### Motivation
Securing some initial `bazel` configuration in CI :
- ensures all platforms are kept in the radar over next iterations,
- prevents regressions over next iterations,
- establishes foundations for future jobs.

### Possible Drawbacks / Trade-offs
- on Windows, compilation errors made be descope dependencies being
built to **only `bzip2`**. This will be of course addressed in a
subsequent PR,
- GitLab caching is still runner-based for the time being, which can be
revisited later. (like everything)

### Additional Notes
Main addressed challenges:
- cache location conflicts[^1]: externalized "repo contents cache"
through `tools/bazel*` wrappers:
  - bazelbuild/bazel#26384
  - bazelbuild/bazel#26522
  - bazelbuild/bazel#26773
  - bazelbuild/bazel#26802
- macOS runners:
  - `bzip2` dependency: fetch from a reachable source:
    - [x] #40219
- Windows:
- `bazel` spawn strategy: fallback to a permissive strategy since
`sandboxed` is unsupported
    - [x] #40328
- `tools/bazel` wrapper: fallback to batch (`tools/bazel.bat`), since
`bash` is discouraged by `bazel` in this case and `tools/bazel.ps1`
poses detection problems,
  - long paths were supported in containers but not on runners:
    - [x] DataDog/ci-platform-machine-images#429
  - recursive symlinks: use `robocopy` instead of `copy`/`move`/`xcopy`.
  
[^1]:
> ERROR: The repo contents cache
[/path/to/datadog-agent/.cache/repo_contents] is inside the workspace
[/path/to/datadog-agent]. This can cause spurious failures. Disable the
repo contents cache with `--repo_contents_cache=`, or specify
`--repo_contents_cache=<path outside the workspace>`.
@markasoftware-tc
Copy link
Copy Markdown

Why was this done? Just to preserve existing 8.x series behavior?

@fmeum
Copy link
Copy Markdown
Collaborator

fmeum commented Feb 25, 2026

The cache was buggy in 8.x and still has some issues that will be fixed in 9.0.1.

rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`
to `--repo_contents_cache=` (empty = disabled, matching Bazel 8.4+
default), incompatible with in-workspace cache path used with ephemeral
runners (both GitLab and GitHub actions want in-workspace cache paths).

### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory, leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```

`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0.

It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.

The present change therefore makes sure this remains disabled until
Bazel figures out a way to honor the `.cache` exclusion in
`.bazelignore`.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`
to `--repo_contents_cache=` (empty = disabled, matching Bazel 8.4+
default), incompatible with in-workspace cache path used with ephemeral
runners (both GitLab and GitHub actions want in-workspace cache paths).

### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory, leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```

`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0.

It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.

The present change therefore makes sure this remains disabled until
Bazel figures out a way to honor the `.cache` exclusion in
`.bazelignore`.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Mar 23, 2026
…nt cache

### What does this PR do?

Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`:

- `--config=ci` is now also injected on GitHub Actions (previously the
  outer condition excluded GitHub Actions entirely, preventing the
  `--repo_contents_cache=` neutralization below from ever firing there)
- `--repo_contents_cache=` (empty = disabled) is injected whenever
  `XDG_CACHE_HOME` has fallen back to the in-repo `.cache` default,
  meaning no user-configured persistent cache is available

### Motivation

`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0 (PR
bazelbuild/bazel#27706, cherry-picked as bazelbuild/bazel#27898) — it
had been disabled by default (PR bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.

When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory and is discarded at the end of the job, defeating its
purpose while adding overhead. Passing `--repo_contents_cache=` mirrors
what Bazel 8.4.0 did at the flag level.

### Describe how you validated your changes

CI passes on both GitLab (DDCI) and GitHub Actions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Mar 23, 2026
…nt cache

### What does this PR do?

Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`:

- `--config=ci` is now also injected on GitHub Actions (previously the
  outer condition excluded GitHub Actions entirely, preventing the
  `--repo_contents_cache=` neutralization below from ever firing there)
- `--repo_contents_cache=` (empty = disabled) is injected whenever
  `XDG_CACHE_HOME` has fallen back to the in-repo `.cache` default,
  meaning no user-configured persistent cache is available

### Motivation

`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0 (PR
bazelbuild/bazel#27706, cherry-picked as bazelbuild/bazel#27898) — it
had been disabled by default (PR bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.

When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory and is discarded at the end of the job, defeating its
purpose while adding overhead. Passing `--repo_contents_cache=` mirrors
what Bazel 8.4.0 did at the flag level.

### Describe how you validated your changes

CI passes on both GitLab (DDCI) and GitHub Actions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel` and `tools/bazel.bat`
to `--repo_contents_cache=` (empty = disabled, matching Bazel 8.4+
default), incompatible with in-workspace cache path used with ephemeral
runners (both GitLab and GitHub actions want in-workspace cache paths).

### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory
(e.g. on an ephemeral CI runner), the cache lands under the repo-scoped
`.cache` directory, leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```

`--repo_contents_cache` stores a workspace-independent copy of fetched
external repository trees under `{--repository_cache}/contents` and
defaults to `"null"` (enabled) as of Bazel 9.0.0.

It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4.0
precisely because of the issues described in bazelbuild/bazel#26384.

The present change therefore makes sure this remains disabled until
Bazel figures out a way to honor the `.cache` exclusion in
`.bazelignore`.
gh-worker-dd-mergequeue-cf854d bot pushed a commit to DataDog/datadog-agent that referenced this pull request Mar 23, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).

### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```

`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.

It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.

The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.

Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
aya-shimizu24 pushed a commit to DataDog/datadog-agent that referenced this pull request Mar 24, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).

### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```

`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.

It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.

The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.

Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
StephenWakely pushed a commit to DataDog/datadog-agent that referenced this pull request Mar 27, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).

### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```

`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.

It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.

The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.

Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
StephenWakely pushed a commit to DataDog/datadog-agent that referenced this pull request Mar 27, 2026
### What does this PR do?
Extends the CI argument injection in `tools/bazel*` beyond just `--config=ci` to `--repo_contents_cache=` (disabled, matching Bazel 8.4+ default) when using an in-workspace cache path (i.e. on Linux ephemeral runners).

### Motivation
When `XDG_CACHE_HOME` is not explicitly set to a persistent directory (i.e. on Linux ephemeral CI runners), the cache lands under the repo-scoped `.cache` directory (both GitLab and GitHub Actions want in-workspace cache paths), leading to:
```
ERROR: The repo contents cache [/path/to/repos/v1/contents] is inside the main repo [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with `--repo_contents_cache=`, or specify `--repo_contents_cache=<path outside the main repo>`.
```

`--repo_contents_cache` stores a workspace-independent copy of fetched external repository trees under `{--repository_cache}/contents` and defaults to `"null"` (enabled) as of Bazel 9.0.

It had been disabled by default (bazelbuild/bazel#26802) in Bazel 8.4+ precisely because of the issues described in bazelbuild/bazel#26384.

The present change therefore keeps it disabled in this very case **until Bazel figures out a way to honor the `.cache` exclusion in `.bazelignore`**.

Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants