Add Pipeline to deploy custom agent image for FIPS testing by michel-laterman · Pull Request #8035 · elastic/elastic-agent

michel-laterman · 2025-04-29T18:18:08Z

What does this PR do?

Add a new buildkite pipeline to build a custom agent image and use it in an ECH deployment for testing.

Why is it important?

FIPS integration tests will require a custom agent running in the CFT region.

Checklist

I have read and understood the pull request guidelines of this project.
My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~I have made corresponding changes to the documentation~~
~~I have made corresponding change to the default configuration files~~
~~I have added tests that prove my fix is effective or that my feature works~~
~~I have added an entry in ./changelog/fragments using the changelog tool~~
I have added an integration test or an E2E test

Disruptive User Impact

N/A

michel-laterman · 2025-05-01T18:52:00Z

buildkite test this

michel-laterman · 2025-05-02T16:41:11Z

@v1v @pazone, can you take a look at this, we need permissions in order to push a custom image to use in the CFT region

denied: requested access to the resource is denied
--
  | Error: Failed pushing docker image: running "docker image push docker.elastic.co/observability-ci/elastic-agent-fips:git-b84b80343415" failed with exit code 1

v1v · 2025-05-08T19:39:12Z

| Error: Failed pushing docker image: running "docker image push docker.elastic.co/observability-ci/elastic-agent-fips:git-b84b80343415" failed with exit code 1

Can you share the URL link to the error?

I'm not familiar with the current user and namespace, but as far as I see, those details are stored at https://github.com/elastic/elastic-agent/blob/aa224536eadf49f8b9b962df240c0caa4861970e/.buildkite/hooks/pre-command#l17.

~~However, I think you need to configure the pre-command hook to run for the new BK pipelines:~~

elastic-agent/.buildkite/hooks/pre-command

Lines 39 to 47 in aa22453

    
           if [[ "$BUILDKITE_PIPELINE_SLUG" == "elastic-agent-package" ]]; then 
        
             if [[ "$BUILDKITE_STEP_KEY" == "package_elastic-agent" ]]; then 
        
               docker_login 
        
             fi 
        
             if [[ "$BUILDKITE_STEP_KEY" == "dra-publish" || "$BUILDKITE_STEP_KEY" == "bk-api-publish-independent-agent" ]]; then 
        
               release_manager_login 
        
             fi 
        
           fi

is the settings for the elastic-agent-package BK pipeline.

I see you have enabled the pre-command for the new step

v1v · 2025-06-09T15:29:40Z

Updatecli tests

$ gh pr checkout 8035
$ git checkout -b test/fips-ech-updatecli
$ git push origin test/fips-ech-updatecli
$ # modify .ci/updatecli/values.d/scm.yml to use my fork
$ GITHUB_TOKEN=$(gh auth token) \                      
  GITHUB_ACTOR=v1v \
  BRANCH_NAME=test/fips-ech-updatecli updatecli apply \ 
    --config .ci/updatecli/updatecli-bump-vm-images.yml \
    --values .ci/updatecli/values.d/scm.yml

diff --git a/.ci/updatecli/values.d/scm.yml b/.ci/updatecli/values.d/scm.yml
index 2708262728..c01cb89d50 100644
--- a/.ci/updatecli/values.d/scm.yml
+++ b/.ci/updatecli/values.d/scm.yml
@@ -1,8 +1,8 @@
 scm:
   enabled: true
-  owner: elastic
+  owner: v1v

Produced v1v#9

v1v

LGTM, with some minor comments

v1v

LGTM, the questions I asked earlier are not blockers, but likely something to be done in follow-up

v1v

LGTM, the questions I asked earlier are not blockers, but likely something to be done in follow-up

elasticmachine · 2025-06-09T18:40:21Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 4ef376f

Failed CI Steps

History

💛 Build #21990 was flaky aa44551
💔 Build #21979 failed 33647be
💔 Build #21971 failed 33647be
💔 Build #21922 failed 645c8b6
💛 Build #21907 was flaky afb5d73

cc @michel-laterman

elastic-sonarqube · 2025-06-09T18:45:46Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

Add a new buildkite pipeline to build a custom agent image and use it in an ECH deployment for testing. Run FIPS integration tests on VMs with a FIPS provider. (cherry picked from commit 5908ed2) # Conflicts: # testing/integration/fleetserver_fips_test.go

Add a new buildkite pipeline to build a custom agent image and use it in an ECH deployment for testing. Run FIPS integration tests on VMs with a FIPS provider. (cherry picked from commit 5908ed2)

… FIPS testing (#8415) * Add Pipeline to deploy custom agent image for FIPS testing (#8035) Add a new buildkite pipeline to build a custom agent image and use it in an ECH deployment for testing. Run FIPS integration tests on VMs with a FIPS provider. (cherry picked from commit 5908ed2) * Fix pre-command to support extended testing (#8418) (cherry picked from commit 97246c1) --------- Co-authored-by: Michel Laterman <82832767+michel-laterman@users.noreply.github.com>

…ts-oblt-cli * feature/serverless-its-oblt-cli: (51 commits) as agreed let's move the group to the kb.integration pipeline Update .github/workflows/serverless-project.yml ci: invoke serverless integration tests package from BK github-actions: create a serverless project daily [tests] split up serverless and resource leaks integration tests (#8396) chore: Update to elastic/beats@dfdc12e33de0 (#8446) Lock upgrade marker (#8254) build(deps): bump github.com/elastic/elastic-agent-system-metrics from 0.11.12 to 0.11.13 (#8420) Add docker image name template and renamed fips cloud specs (#8429) buildkite(scripts): refactor common scripts (#8365) Use require.Eventually to try and address flakiness (#8421) Fix pre-command to support extended testing (#8418) [Automation] Bump Golang version to 1.24.4 (#8384) chore: Update to elastic/beats@aed2a8b768bd (#8423) [main][Automation] Update versions (#8425) Add Pipeline to deploy custom agent image for FIPS testing (#8035) ci: automatically update OTel components (#8288) [Automation] Bump VM Image version to 1749258065 (#8390) fix: increase context timeout to accommodate for slower machines in Test_checkForUnprivilegedVault (#8374) otel: add cumulativetodeltaprocessor to EDOT collector (#8372) ...

Add Pipeline to deploy custom agent image for FIPS testing

ea416ea

michel-laterman added enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team backport-8.19 Automated backport to the 8.19 branch labels Apr 29, 2025

mergify bot assigned michel-laterman Apr 29, 2025

Add new pipeline as triggered step

c9d4023

michel-laterman added the skip-changelog label Apr 29, 2025

michel-laterman and others added 5 commits April 29, 2025 13:28

Use normal bk agent

f8d343b

Add bk agent provider and machinetype

46ec87a

Add FIPS support to cloud:push

cd7f72a

Fix magefile

3ae5f92

Merge branch 'main' into fips-ech

b84b803

michel-laterman mentioned this pull request May 6, 2025

[Integration Testing] Allow tests to declare themselves as needing a FIPS environment #8083

Merged

8 tasks

michel-laterman added 5 commits May 6, 2025 09:21

Push image to beats-ci, add docker login hook

838cdbd

Fix pipeline

7515f8f

Add FIPS env check to buildkite-integration-tests.sh

8ad48a3

Remove integration.fips addition

0ace521

Merge branch 'main' into fips-ech

005746a

ycombinator mentioned this pull request May 7, 2025

[Integration Test] Ensure that upgrading a FIPS-capable Agent results in a FIPS-capable Agent #7804

Merged

8 tasks

Fix label names

bd6119d

mergify bot mentioned this pull request May 8, 2025

[DO NOT MERGE] Trying out some fixes for PR: #8035 #8122

Closed

revert change to integration tests

85ae7fa

michel-laterman and others added 4 commits May 8, 2025 14:36

Pass FIPS

11b8ada

Merge branch 'main' into fips-ech

3ddecb1

Add integration test with FIPS: true in requirements

775bd58

Remove extra test

5fb8569

michel-laterman marked this pull request as ready for review May 9, 2025 18:48

michel-laterman force-pushed the fips-ech branch from d5abeac to a2c3227 Compare June 9, 2025 15:15

Merge branch 'main' into fips-ech

59fa434

michel-laterman requested review from pchila, pkoutsovasilis and ycombinator June 9, 2025 15:16

v1v reviewed Jun 9, 2025

View reviewed changes

Comment thread .buildkite/bk.integration-fips.pipeline.yml Outdated

Comment thread .buildkite/hooks/pre-command

Comment thread .buildkite/bk.integration-fips.pipeline.yml

Comment thread .buildkite/bk.integration-fips.pipeline.yml

Remove key for aggregate-reports-fips

4ef376f

ycombinator reviewed Jun 9, 2025

View reviewed changes

Comment thread .ci/updatecli/updatecli-bump-vm-images.yml

v1v approved these changes Jun 9, 2025

View reviewed changes

ycombinator reviewed Jun 9, 2025

View reviewed changes

Comment thread .buildkite/bk.integration-fips.pipeline.yml Outdated

ycombinator approved these changes Jun 9, 2025

View reviewed changes

michel-laterman mentioned this pull request Jun 9, 2025

updatecli-bump-vm-images.yml improvements #8408

Open

michel-laterman merged commit 5908ed2 into elastic:main Jun 9, 2025
11 checks passed

mergify bot mentioned this pull request Jun 9, 2025

[8.19] (backport #8035) Add Pipeline to deploy custom agent image for FIPS testing #8415

Merged

4 tasks

michel-laterman deleted the fips-ech branch June 9, 2025 19:44

michel-laterman mentioned this pull request Jun 9, 2025

Fix pre-command to support extended testing #8418

Merged

This was referenced Jun 12, 2025

github-actions: updatecli bump for specific branches #8466

Merged

mergify: support auto-merge for bk.integration-fips.pipeline.yml #8468

Merged

pkoutsovasilis mentioned this pull request Jun 30, 2025

ci: pin elastic-agent version #8736

Merged

8 tasks

This was referenced Jun 30, 2025

[8.17](backport #8736) ci: pin elastic-agent version #8748

Merged

[8.18](backport #8736) ci: pin elastic-agent version #8749

Merged

[9.0](backport #8736) ci: pin elastic-agent version #8750

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Pipeline to deploy custom agent image for FIPS testing#8035

Add Pipeline to deploy custom agent image for FIPS testing#8035
michel-laterman merged 60 commits intoelastic:mainfrom
michel-laterman:fips-ech

michel-laterman commented Apr 29, 2025 •

edited

Loading

Uh oh!

michel-laterman commented May 1, 2025

Uh oh!

michel-laterman commented May 2, 2025

Uh oh!

v1v commented May 8, 2025 •

edited

Loading

Uh oh!

v1v commented Jun 9, 2025

Uh oh!

v1v left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

v1v left a comment

Uh oh!

v1v left a comment

Uh oh!

Uh oh!

elasticmachine commented Jun 9, 2025 •

edited

Loading

Uh oh!

elastic-sonarqube bot commented Jun 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Conversation

michel-laterman commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Disruptive User Impact

Uh oh!

michel-laterman commented May 1, 2025

Uh oh!

michel-laterman commented May 2, 2025

Uh oh!

v1v commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

v1v commented Jun 9, 2025

Updatecli tests

Uh oh!

v1v left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

v1v left a comment

Choose a reason for hiding this comment

Uh oh!

v1v left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticmachine commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💛 Build succeeded, but was flaky

Failed CI Steps

History

Uh oh!

elastic-sonarqube bot commented Jun 9, 2025

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

michel-laterman commented Apr 29, 2025 •

edited

Loading

v1v commented May 8, 2025 •

edited

Loading

elasticmachine commented Jun 9, 2025 •

edited

Loading