[ABLD-135] Add initial `bazel` jobs by rdesgroppes · Pull Request #40153 · DataDog/datadog-agent

rdesgroppes · 2025-08-22T14:28:50Z

What does this PR do?

Now bazelisk is available on all our CI executors (macOS runners, Linux & Windows containers), this change secures an initial part of our bazel setup while providing reusable job templates to ease caching in CI.

Practically, it adds a handful of jobs focusing on building bazel dependencies in the corresponding GitLab deps_build stage.

Overall, it consists in:

verifying bazelisk properly bootstraps bazel across all platforms, (primary scope of the PR)
extracting initial bazel:-prefixed CI job templates and dogfooding them, (i.e. by building deps)
binding bazelisk/bazel caches to GitLab caching capabilities (runner-based as of now): installed binaries, "repository cache", "repo contents cache", and "disk cache" are all saved/restored correctly to/from GitLab while honoring OS/architecture boundaries,
marking in-workspace cache in both .bazelignore and .gitignore,
adjusting code/job ownership accordingly.

Motivation

Securing some initial bazel configuration in CI :

ensures all platforms are kept in the radar over next iterations,
prevents regressions over next iterations,
establishes foundations for future jobs.

Possible Drawbacks / Trade-offs

on Windows, compilation errors made be descope dependencies being built to only bzip2. This will be of course addressed in a subsequent PR,
GitLab caching is still runner-based for the time being, which can be revisited later. (like everything)

Additional Notes

Main addressed challenges:

cache location conflicts¹: externalized "repo contents cache" through tools/bazel* wrappers:
macOS runners:
- bzip2 dependency: fetch from a reachable source:
  - [ABLD-135] Make bazel retrieve bzip2 on AWS-hosted runners #40219
Windows:
- bazel spawn strategy: fallback to a permissive strategy since sandboxed is unsupported
  - [ABLD-135] Use available bazel spawn strategy on Windows #40328
- tools/bazel wrapper: fallback to batch (tools/bazel.bat), since bash is discouraged by bazel in this case and tools/bazel.ps1 poses detection problems,
- long paths were supported in containers but not on runners:
  - DataDog/ci-platform-machine-images#429
- recursive symlinks: use robocopy instead of copy/move/xcopy.

ERROR: The repo contents cache [/path/to/datadog-agent/.cache/repo_contents] is inside the workspace [/path/to/datadog-agent]. This can cause spurious failures. Disable the repo contents cache with --repo_contents_cache=, or specify --repo_contents_cache=<path outside the workspace>.

↩

agent-platform-auto-pr · 2025-08-22T15:17:50Z

Gitlab CI Configuration Changes

Added Jobs

.bazel:build-deps

.bazel:build-deps:
  cache:
  - key:
      files:
      - .bazelversion
      prefix: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazelisk
    - .cache/bazel/install
    policy: pull-push
    when: on_success
  - key: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazel/cache
    - .cache/bazel/disk
    - .cache/bazel/repo-contents
    policy: pull-push
    when: on_success
  needs: []
  rules:
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - when: on_success
  stage: deps_build
  variables:
    BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
    BAZEL_DISK_CACHE: $BAZEL_OUTPUT_USER_ROOT/disk
    BAZEL_OUTPUT_USER_ROOT: $XDG_CACHE_HOME/bazel
    BAZEL_REPO_CONTENTS_CACHE: $BAZEL_OUTPUT_USER_ROOT/repo-contents
    XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache

.bazel:defs

.bazel:defs:
  cache:
  - key:
      files:
      - .bazelversion
      prefix: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazelisk
    - .cache/bazel/install
    policy: pull-push
    when: on_success
  - key: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazel/cache
    - .cache/bazel/disk
    - .cache/bazel/repo-contents
    policy: pull-push
    when: on_success
  needs: []
  variables:
    BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
    BAZEL_DISK_CACHE: $BAZEL_OUTPUT_USER_ROOT/disk
    BAZEL_OUTPUT_USER_ROOT: $XDG_CACHE_HOME/bazel
    BAZEL_REPO_CONTENTS_CACHE: $BAZEL_OUTPUT_USER_ROOT/repo-contents
    XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache

.bazel:ext:windows

.bazel:ext:windows:
  script:
  - '$FailFast = @''

    $ErrorActionPreference = ''Stop''

    $PSNativeCommandUseErrorActionPreference = $true

    Set-StrictMode -Version 3.0

    ''@'
  - Invoke-Expression $FailFast
  - (($FailFast, $SCRIPT) -join [Environment]::NewLine) | docker run --env=BAZELISK_HOME
    --env=BAZEL_DISK_CACHE --env=BAZEL_OUTPUT_USER_ROOT --env=BAZEL_REPO_CONTENTS_CACHE
    --env=CI_PROJECT_DIR --env=RUNNER_TEMP_PROJECT_DIR --interactive --rm --volume="${CI_PROJECT_DIR}:$CI_PROJECT_DIR"
    --volume="${RUNNER_TEMP_PROJECT_DIR}:$RUNNER_TEMP_PROJECT_DIR" --workdir="$CI_PROJECT_DIR"
    "$WINBUILDIMAGE" powershell -Command -
  tags:
  - windows-v2:2022
  variables:
    GIT_DEPTH: 2
    OVERRIDE_GIT_STRATEGY: clone
    WINBUILDIMAGE: registry.ddbuild.io/ci/datadog-agent-buildimages/windows_ltsc2022_${ARCH}${CI_IMAGE_WIN_LTSC2022_X64_SUFFIX}:${CI_IMAGE_WIN_LTSC2022_X64}

bazel:build-deps:linux-amd64

bazel:build-deps:linux-amd64:
  cache:
  - key:
      files:
      - .bazelversion
      prefix: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazelisk
    - .cache/bazel/install
    policy: pull-push
    when: on_success
  - key: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazel/cache
    - .cache/bazel/disk
    - .cache/bazel/repo-contents
    policy: pull-push
    when: on_success
  image: registry.ddbuild.io/ci/datadog-agent-buildimages/linux$CI_IMAGE_LINUX_SUFFIX:$CI_IMAGE_LINUX
  needs: []
  rules:
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - when: on_success
  script:
  - bazel build //deps/...
  stage: deps_build
  tags:
  - arch:amd64
  - specific:true
  variables:
    BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
    BAZEL_DISK_CACHE: $BAZEL_OUTPUT_USER_ROOT/disk
    BAZEL_OUTPUT_USER_ROOT: $XDG_CACHE_HOME/bazel
    BAZEL_REPO_CONTENTS_CACHE: $BAZEL_OUTPUT_USER_ROOT/repo-contents
    XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache

bazel:build-deps:linux-arm64

bazel:build-deps:linux-arm64:
  cache:
  - key:
      files:
      - .bazelversion
      prefix: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazelisk
    - .cache/bazel/install
    policy: pull-push
    when: on_success
  - key: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazel/cache
    - .cache/bazel/disk
    - .cache/bazel/repo-contents
    policy: pull-push
    when: on_success
  image: registry.ddbuild.io/ci/datadog-agent-buildimages/linux$CI_IMAGE_LINUX_SUFFIX:$CI_IMAGE_LINUX
  needs: []
  rules:
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - when: on_success
  script:
  - bazel build //deps/...
  stage: deps_build
  tags:
  - arch:arm64
  - specific:true
  variables:
    BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
    BAZEL_DISK_CACHE: $BAZEL_OUTPUT_USER_ROOT/disk
    BAZEL_OUTPUT_USER_ROOT: $XDG_CACHE_HOME/bazel
    BAZEL_REPO_CONTENTS_CACHE: $BAZEL_OUTPUT_USER_ROOT/repo-contents
    XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache

bazel:build-deps:macos-amd64

bazel:build-deps:macos-amd64:
  cache:
  - key:
      files:
      - .bazelversion
      prefix: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazelisk
    - .cache/bazel/install
    policy: pull-push
    when: on_success
  - key: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazel/cache
    - .cache/bazel/disk
    - .cache/bazel/repo-contents
    policy: pull-push
    when: on_success
  needs: []
  rules:
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - when: on_success
  script:
  - bazel build //deps/...
  stage: deps_build
  tags:
  - macos:ventura-amd64
  - specific:true
  variables:
    BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
    BAZEL_DISK_CACHE: $BAZEL_OUTPUT_USER_ROOT/disk
    BAZEL_OUTPUT_USER_ROOT: $XDG_CACHE_HOME/bazel
    BAZEL_REPO_CONTENTS_CACHE: $BAZEL_OUTPUT_USER_ROOT/repo-contents
    XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache

bazel:build-deps:macos-arm64

bazel:build-deps:macos-arm64:
  cache:
  - key:
      files:
      - .bazelversion
      prefix: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazelisk
    - .cache/bazel/install
    policy: pull-push
    when: on_success
  - key: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazel/cache
    - .cache/bazel/disk
    - .cache/bazel/repo-contents
    policy: pull-push
    when: on_success
  needs: []
  rules:
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - when: on_success
  script:
  - bazel build //deps/...
  stage: deps_build
  tags:
  - macos:ventura-arm64
  - specific:true
  variables:
    BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
    BAZEL_DISK_CACHE: $BAZEL_OUTPUT_USER_ROOT/disk
    BAZEL_OUTPUT_USER_ROOT: $XDG_CACHE_HOME/bazel
    BAZEL_REPO_CONTENTS_CACHE: $BAZEL_OUTPUT_USER_ROOT/repo-contents
    XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache

bazel:build-deps:windows-amd64

bazel:build-deps:windows-amd64:
  cache:
  - key:
      files:
      - .bazelversion
      prefix: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazelisk
    - .cache/bazel/install
    policy: pull-push
    when: on_success
  - key: bazel-$CI_RUNNER_DESCRIPTION
    paths:
    - .cache/bazel/cache
    - .cache/bazel/disk
    - .cache/bazel/repo-contents
    policy: pull-push
    when: on_success
  needs: []
  rules:
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - when: on_success
  script:
  - '$FailFast = @''

    $ErrorActionPreference = ''Stop''

    $PSNativeCommandUseErrorActionPreference = $true

    Set-StrictMode -Version 3.0

    ''@'
  - Invoke-Expression $FailFast
  - (($FailFast, $SCRIPT) -join [Environment]::NewLine) | docker run --env=BAZELISK_HOME
    --env=BAZEL_DISK_CACHE --env=BAZEL_OUTPUT_USER_ROOT --env=BAZEL_REPO_CONTENTS_CACHE
    --env=CI_PROJECT_DIR --env=RUNNER_TEMP_PROJECT_DIR --interactive --rm --volume="${CI_PROJECT_DIR}:$CI_PROJECT_DIR"
    --volume="${RUNNER_TEMP_PROJECT_DIR}:$RUNNER_TEMP_PROJECT_DIR" --workdir="$CI_PROJECT_DIR"
    "$WINBUILDIMAGE" powershell -Command -
  stage: deps_build
  tags:
  - windows-v2:2022
  variables:
    ARCH: x64
    BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
    BAZEL_DISK_CACHE: $BAZEL_OUTPUT_USER_ROOT/disk
    BAZEL_OUTPUT_USER_ROOT: $XDG_CACHE_HOME/bazel
    BAZEL_REPO_CONTENTS_CACHE: $BAZEL_OUTPUT_USER_ROOT/repo-contents
    GIT_DEPTH: 2
    OVERRIDE_GIT_STRATEGY: clone
    SCRIPT: "echo \"\U0001F7E1 TODO(regis): compilation errors remain - limiting to\
      \ a working subset for the time being\"\nbazel build @bzip2//..."
    WINBUILDIMAGE: registry.ddbuild.io/ci/datadog-agent-buildimages/windows_ltsc2022_${ARCH}${CI_IMAGE_WIN_LTSC2022_X64_SUFFIX}:${CI_IMAGE_WIN_LTSC2022_X64}
    XDG_CACHE_HOME: $CI_PROJECT_DIR/.cache

Changes Summary

Removed	Modified	Added	Renamed
0	0	8	0

ℹ️ Diff available in the job log.

cit-pr-commenter · 2025-08-22T15:57:15Z

Regression Detector

Regression Detector Results

Metrics dashboard
Target profiles
Run ID: 3ad9f071-1e5b-4b90-b2f9-6bf368ef9994

Baseline: febb18c
Comparison: c148634
Diff

Optimization Goals: ✅ No significant changes detected

Experiments ignored for regressions

Regressions in experiments with settings containing erratic: true are ignored.

perf	experiment	goal	Δ mean %	Δ mean % CI	trials	links
➖	docker_containers_cpu	% cpu utilization	+1.89	[-1.24, +5.01]	1	Logs

Fine details of change detection per experiment

perf	experiment	goal	Δ mean %	Δ mean % CI	trials	links
➖	docker_containers_cpu	% cpu utilization	+1.89	[-1.24, +5.01]	1	Logs
➖	docker_containers_memory	memory utilization	+0.60	[+0.39, +0.80]	1	Logs
➖	quality_gate_logs	% cpu utilization	+0.27	[-2.51, +3.05]	1	Logs bounds checks dashboard
➖	uds_dogstatsd_20mb_12k_contexts_20_senders	memory utilization	+0.16	[+0.12, +0.20]	1	Logs
➖	file_to_blackhole_1000ms_latency	egress throughput	+0.15	[-0.48, +0.78]	1	Logs
➖	otlp_ingest_logs	memory utilization	+0.12	[-0.02, +0.27]	1	Logs
➖	file_to_blackhole_500ms_latency	egress throughput	+0.06	[-0.56, +0.68]	1	Logs
➖	file_tree	memory utilization	+0.01	[-0.03, +0.06]	1	Logs
➖	file_to_blackhole_0ms_latency	egress throughput	+0.01	[-0.62, +0.64]	1	Logs
➖	tcp_dd_logs_filter_exclude	ingress throughput	-0.01	[-0.04, +0.02]	1	Logs
➖	uds_dogstatsd_to_api	ingress throughput	-0.02	[-0.08, +0.04]	1	Logs
➖	file_to_blackhole_100ms_latency	egress throughput	-0.08	[-0.67, +0.51]	1	Logs
➖	ddot_logs	memory utilization	-0.09	[-0.20, +0.02]	1	Logs
➖	ddot_metrics	memory utilization	-0.18	[-0.37, +0.01]	1	Logs
➖	quality_gate_idle_all_features	memory utilization	-0.26	[-0.29, -0.23]	1	Logs bounds checks dashboard
➖	otlp_ingest_metrics	memory utilization	-0.32	[-0.49, -0.14]	1	Logs
➖	quality_gate_idle	memory utilization	-0.51	[-0.54, -0.47]	1	Logs bounds checks dashboard
➖	quality_gate_metrics_logs	memory utilization	-0.55	[-0.87, -0.23]	1	Logs bounds checks dashboard
➖	tcp_syslog_to_blackhole	ingress throughput	-2.43	[-2.48, -2.38]	1	Logs

Bounds Checks: ❌ Failed

perf	experiment	bounds_check_name	replicates_passed	links
✅	docker_containers_cpu	simple_check_run	10/10
✅	docker_containers_memory	memory_usage	10/10
❌	docker_containers_memory	simple_check_run	9/10
✅	file_to_blackhole_0ms_latency	lost_bytes	10/10
✅	file_to_blackhole_0ms_latency	memory_usage	10/10
✅	file_to_blackhole_1000ms_latency	memory_usage	10/10
✅	file_to_blackhole_100ms_latency	lost_bytes	10/10
✅	file_to_blackhole_100ms_latency	memory_usage	10/10
✅	file_to_blackhole_500ms_latency	lost_bytes	10/10
✅	file_to_blackhole_500ms_latency	memory_usage	10/10
✅	quality_gate_idle	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_idle	memory_usage	10/10	bounds checks dashboard
✅	quality_gate_idle_all_features	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_idle_all_features	memory_usage	10/10	bounds checks dashboard
✅	quality_gate_logs	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_logs	lost_bytes	10/10	bounds checks dashboard
✅	quality_gate_logs	memory_usage	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	cpu_usage	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	lost_bytes	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	memory_usage	10/10	bounds checks dashboard

Explanation

Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%

Performance changes are noted in the perf column of each table:

✅ = significantly better comparison variant performance
❌ = significantly worse comparison variant performance
➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
Its configuration does not mark it "erratic".

CI Pass/Fail Decision

✅ Passed. All Quality Gates passed.

quality_gate_logs, bounds check lost_bytes: 10/10 replicas passed. Gate passed.
quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
quality_gate_metrics_logs, bounds check lost_bytes: 10/10 replicas passed. Gate passed.
quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.

agent-platform-auto-pr · 2025-08-22T16:23:40Z

Static quality checks

✅ Please find below the results from static quality gates
Comparison made with ancestor 97d3978

Successful checks

Info

	Quality gate	Delta	On disk size (MiB)	Delta	On wire size (MiB)
✅	agent_deb_amd64	$${+0.06}$$	$${701.65}$$ < $${704.84}$$	$${-0.03}$$	$${177.4}$$ < $${178.58}$$
✅	agent_deb_amd64_fips	$${+0.06}$$	$${688.92}$$ < $${703.09}$$	$${-0.03}$$	$${174.3}$$ < $${178.12}$$
✅	agent_heroku_amd64	$${+0.01}$$	$${350.81}$$ < $${355.37}$$	$${+0}$$	$${94.25}$$ < $${95.72}$$
✅	agent_msi	$${+0.06}$$	$${977.45}$$ < $${980.61}$$	$${+0.02}$$	$${150.28}$$ < $${152.67}$$
✅	agent_rpm_amd64	$${+0.06}$$	$${701.64}$$ < $${704.83}$$	$${-0.01}$$	$${179.6}$$ < $${180.22}$$
✅	agent_rpm_amd64_fips	$${+0.06}$$	$${688.91}$$ < $${703.08}$$	$${+0.01}$$	$${175.8}$$ < $${179.85}$$
✅	agent_rpm_arm64	$${+0.06}$$	$${690.61}$$ < $${694.74}$$	$${+0.04}$$	$${161.85}$$ < $${163.96}$$
✅	agent_rpm_arm64_fips	$${+0.06}$$	$${678.72}$$ < $${693.05}$$	$${+0.03}$$	$${158.45}$$ < $${163.0}$$
✅	agent_suse_amd64	$${+0.06}$$	$${701.64}$$ < $${704.83}$$	$${-0.01}$$	$${179.6}$$ < $${180.22}$$
✅	agent_suse_amd64_fips	$${+0.06}$$	$${688.91}$$ < $${703.08}$$	$${+0.01}$$	$${175.8}$$ < $${179.85}$$
✅	agent_suse_arm64	$${+0.06}$$	$${690.61}$$ < $${694.74}$$	$${+0.04}$$	$${161.85}$$ < $${163.96}$$
✅	agent_suse_arm64_fips	$${+0.06}$$	$${678.72}$$ < $${693.05}$$	$${+0.03}$$	$${158.45}$$ < $${163.0}$$
✅	docker_agent_amd64	$${+0.06}$$	$${784.06}$$ < $${788.65}$$	$${+0.01}$$	$${270.27}$$ < $${272.01}$$
✅	docker_agent_arm64	$${+0.06}$$	$${796.36}$$ < $${802.0}$$	$${+0.01}$$	$${257.0}$$ < $${259.7}$$
✅	docker_agent_jmx_amd64	$${+0.06}$$	$${974.93}$$ < $${979.84}$$	$${+0.02}$$	$${338.91}$$ < $${340.95}$$
✅	docker_agent_jmx_arm64	$${+0.06}$$	$${975.83}$$ < $${981.8}$$	$${+0.02}$$	$${321.62}$$ < $${324.65}$$
✅	docker_cluster_agent_amd64	$${-0}$$	$${212.68}$$ < $${214.5}$$	$${-0}$$	$${72.22}$$ < $${73.51}$$
✅	docker_cluster_agent_arm64	$${+0}$$	$${228.6}$$ < $${230.33}$$	$${+0}$$	$${68.48}$$ < $${69.77}$$
✅	docker_cws_instrumentation_amd64	$${0}$$	$${7.07}$$ < $${7.12}$$	$${-0}$$	$${2.95}$$ < $${3.29}$$
✅	docker_cws_instrumentation_arm64	$${0}$$	$${6.69}$$ < $${6.92}$$	$${+0}$$	$${2.71}$$ < $${3.07}$$
✅	docker_dogstatsd_amd64	$${+0}$$	$${38.33}$$ < $${39.57}$$	$${-0}$$	$${14.81}$$ < $${15.76}$$
✅	docker_dogstatsd_arm64	$${-0}$$	$${36.99}$$ < $${38.2}$$	$${-0}$$	$${14.26}$$ < $${14.83}$$
✅	dogstatsd_deb_amd64	$${0}$$	$${29.56}$$ < $${31.4}$$	$${+0}$$	$${7.8}$$ < $${8.95}$$
✅	dogstatsd_deb_arm64	$${0}$$	$${28.15}$$ < $${29.97}$$	$${-0}$$	$${6.75}$$ < $${7.89}$$
✅	dogstatsd_rpm_amd64	$${0}$$	$${29.56}$$ < $${31.4}$$	$${-0}$$	$${7.81}$$ < $${8.96}$$
✅	dogstatsd_suse_amd64	$${0}$$	$${29.56}$$ < $${31.4}$$	$${-0}$$	$${7.81}$$ < $${8.96}$$
✅	iot_agent_deb_amd64	$${0}$$	$${54.56}$$ < $${54.97}$$	$${-0}$$	$${13.76}$$ < $${14.45}$$
✅	iot_agent_deb_arm64	$${0}$$	$${51.86}$$ < $${51.9}$$	$${-0}$$	$${11.89}$$ < $${12.63}$$
✅	iot_agent_deb_armhf	$${0}$$	$${51.43}$$ < $${51.84}$$	$${+0}$$	$${11.96}$$ < $${12.74}$$
✅	iot_agent_rpm_amd64	$${0}$$	$${54.56}$$ < $${54.97}$$	$${+0}$$	$${13.77}$$ < $${14.47}$$
✅	iot_agent_suse_amd64	$${0}$$	$${54.56}$$ < $${54.97}$$	$${+0}$$	$${13.77}$$ < $${14.47}$$

On the one hand, `/tmp` is a problematic (ambiguous) path under Windows (`c:/tmp`, `c:\\tmp` and even `\\tmp` would all denote absolute paths). On the other hand, keeping the default disk cache path makes the cache persistent: - across reboots on developer machines, - in GitLab CI. (#40153) In case someone asks: disk and remote caching work well together: `bazel` forms a combined cache, reading from and writing to both, which reduces remote load and speeds up builds, see: https://www.buildbuddy.io/blog/bazels-remote-caching-and-remote-execution-explained/#flags

On the one hand, `/tmp` is a problematic (ambiguous) path under Windows (`c:/tmp`, `c:\\tmp` and even `\\tmp` would all denote absolute paths). On the other hand, keeping the default disk cache path allows to persist the cache: - across reboots on developer machines, - in GitLab CI. (#40153) In case someone asks: disk and remote caching work well together: `bazel` forms a combined cache, reading from and writing to both, which reduces remote load and speeds up builds, see: https://www.buildbuddy.io/blog/bazels-remote-caching-and-remote-execution-explained/#flags

.github/CODEOWNERS

JSGette · 2025-09-01T09:57:48Z

.gitlab/bazel/include.yaml

+      when: on_success
+    - key: bazel-$CI_RUNNER_DESCRIPTION
+      paths:
+        - .cache/bazel/cache


Do we really want to have this? CI Infra team is working on persistent runners this sprint so we should be able to reuse directories on the workers, sharing disk cache via Gitlab will actually add to the Runner's startup time.

https://datadoghq.atlassian.net/browse/CIEXE-1222

Do we really want to have this?

Yes.
tl;dr: I approach my PRs with a “stage objective” in mind.

This probably comes from my habit of working in environments where it’s not acceptable to deliver intermediate results that aren’t self-contained, even if we expect them to be replaced soon. In my experience, “soon” often stretches out when we rely on other teams who naturally have different priorities.

That’s why I prefer to avoid situations where, even for a few days, the CI experience is degraded (for example, downloading dependencies on every single run). To me, it’s worth having this temporary stage in place while waiting for the final solution.

JSGette · 2025-09-01T10:01:31Z

.gitlab/deps_build/deps_build.yml

@@ -122,3 +122,41 @@ build_processed_btfhub_archive:
    - dda inv -- -e system-probe.process-btfhub-archive --branch $BTFHUB_ARCHIVE_BRANCH


I would rather have a separate yaml file to include it to keep the current deps_build.yml clean. Something like deps_bazel_build.yml

✅ Done in second commit.

to keep the current deps_build.yml clean.

... not only the said file is kept clean, but the extraction also allows to bother fewer reviewers.

JSGette · 2025-09-01T10:02:32Z

.gitlab/deps_build/deps_build.yml

+  variables:
+    ARCH: x64
+    SCRIPT: |-
+      bazel build @bzip2//...


Why is the target different for Windows? I would at least write a comment or add a todo hinting that Windows support isn't complete yet

Possible Drawbacks / Trade-offs

on Windows, compilation errors made be descope dependencies being built to only bzip2. This will be of course addressed in a subsequent PR,

✅ Answered in 3rd commit, which adds the following message: (better than a comment to raise awareness)

🟡 TODO(regis): compilation errors remain to be addressed - limiting to a working subset for the time being

JSGette

It's a banger!

This is to address: #40153 (comment)

rdesgroppes added changelog/no-changelog No changelog entry needed qa/no-code-change No code change in Agent code requiring validation labels Aug 22, 2025

github-actions bot added the short review PR is simple enough to be reviewed quickly label Aug 22, 2025

rdesgroppes force-pushed the regis.desgroppes/add-initial-bazel-jobs branch from 8673f3a to 78e3ae7 Compare August 22, 2025 15:14

rdesgroppes force-pushed the regis.desgroppes/add-initial-bazel-jobs branch 2 times, most recently from 3f758ff to 3126bb9 Compare August 25, 2025 07:05

github-actions bot added medium review PR review might take time and removed short review PR is simple enough to be reviewed quickly labels Aug 25, 2025

rdesgroppes force-pushed the regis.desgroppes/add-initial-bazel-jobs branch 19 times, most recently from 2a2ce30 to 96dbab2 Compare August 26, 2025 22:58

rdesgroppes force-pushed the regis.desgroppes/add-initial-bazel-jobs branch 9 times, most recently from 1c27424 to b39069f Compare August 27, 2025 17:55

rdesgroppes mentioned this pull request Aug 27, 2025

[ABLD-133] Stop using OS-specific temp dir for bazel disk cache #40323

Merged

rdesgroppes force-pushed the regis.desgroppes/add-initial-bazel-jobs branch 6 times, most recently from 7a0f243 to b3f95eb Compare August 27, 2025 20:43

CelianR approved these changes Sep 1, 2025

View reviewed changes

.github/CODEOWNERS Show resolved Hide resolved

rdesgroppes mentioned this pull request Sep 1, 2025

[ABLD-144] Build openssl via bazel #40095

Merged

JSGette reviewed Sep 1, 2025

View reviewed changes

JSGette approved these changes Sep 1, 2025

View reviewed changes

rdesgroppes added 3 commits September 2, 2025 14:00

Add initial bazel jobs

0a7b2f3

Extract bazel:deps-build to own file to keep deps_build.yml clean

acff6b4

This is to address: #40153 (comment)

Show a reason why it's a different target on Windows

c148634

This is to address: #40153 (comment)

alopezz mentioned this pull request Sep 12, 2025

Disable bazel deps build on Windows #40897

Merged

dd-octo-sts bot mentioned this pull request Sep 16, 2025

[Backport 7.71.x] Disable bazel deps build on Windows #40991

Merged

rdesgroppes mentioned this pull request Mar 27, 2026

Make Bazel ignore build directories #48475

Merged

		@@ -122,3 +122,41 @@ build_processed_btfhub_archive:
		- dda inv -- -e system-probe.process-btfhub-archive --branch $BTFHUB_ARCHIVE_BRANCH

Conversation

rdesgroppes commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Possible Drawbacks / Trade-offs

Additional Notes

Footnotes

Uh oh!

agent-platform-auto-pr bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Gitlab CI Configuration Changes

Added Jobs

Changes Summary

Uh oh!

cit-pr-commenter bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression Detector

Regression Detector Results

Optimization Goals: ✅ No significant changes detected

Experiments ignored for regressions

Fine details of change detection per experiment

Bounds Checks: ❌ Failed

Explanation

CI Pass/Fail Decision

Uh oh!

agent-platform-auto-pr bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Static quality checks

Info

Uh oh!

Uh oh!

JSGette Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rdesgroppes Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JSGette Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

rdesgroppes Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JSGette Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

rdesgroppes Sep 1, 2025

Choose a reason for hiding this comment

Possible Drawbacks / Trade-offs

Uh oh!

rdesgroppes Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

JSGette left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rdesgroppes commented Aug 22, 2025 •

edited

Loading

agent-platform-auto-pr bot commented Aug 22, 2025 •

edited

Loading

cit-pr-commenter bot commented Aug 22, 2025 •

edited

Loading

agent-platform-auto-pr bot commented Aug 22, 2025 •

edited

Loading

JSGette Sep 1, 2025 •

edited

Loading

rdesgroppes Sep 1, 2025 •

edited

Loading

rdesgroppes Sep 1, 2025 •

edited

Loading