[CI] Add support for running tests with stack 9.0.0 by mrodm · Pull Request #11138 · elastic/integrations

mrodm · 2024-09-16T13:52:05Z

Proposed commit message

Add support for running tests with stack 9.0.0.
Add new steps in daily and weekly pipelines for testing packages with stack 9.0.0 (without reporting issues for now).

Author's Checklist

Test pipeline with 9.0.0-SNAPSHOT.
Add new step to test against stack 9.0.0-SNAPSHOT in daily and weekly pipelines.
Ensure daily job for 9.0.0-SNAPSHOT does not report failing tests.
- Checked here: https://buildkite.com/elastic/integrations/builds/16084
Add required configuration for updatecli.
Update CI docs with latest changes.

Related issues

Part of EPR returns almost no packages when kibana.version of 9.0.0 is provided package-registry#1226

elastic-vault-github-plugin-prod · 2024-09-16T15:58:09Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2024-09-17T08:58:18Z

💔 Build Failed

Buildkite Build
Commit: 7b2fe316efc84b31eb8532c670243fca23f33e07

Failed CI Steps

Check integrations mysql

History

💔 Build #15937 failed c1b56af5106f2b3d44b0fac65b4cf7e30aa2de0a
💔 Build #15936 failed e9e46863d0b67ad73425ded247f879d3c5086eac
💚 Build #15935 succeeded d64f32a0c14651425b4a6dc80d87c95268509a95
💔 Build #15934 failed 0801a94fb685c8f570278c54bf861ce8d8090b18
💚 Build #15933 succeeded 15040d6a5ecbcc6327f42294f33a255205c7cdd9

cc @mrodm

mrodm · 2024-09-17T13:48:01Z

.buildkite/pipeline.schedule-daily.yml

+    depends_on:
+      - step: "check"
+        allow_failure: false


Would it be interesting to run this step after finishing the tests with 8.x ?
It would be updating the step defined in the depends_on field.

To avoid creating another build to test 290 packages at the same time (not sure if that could be an issue). WDYT?

Well, this is going to be temporary, I guess that some time after 9.0 is released, 7.x will become unmaintained and we will remove its stage.

If it depends on 8.x, will it be executed if 8.x fails? We want both to be executed even if the other fails, what is pretty common 😅

Yes, it should be able to do that by setting depends_on as:

depends_on: - step: "local-8-stack" allow_failure: true

Once it is removed 7.x, it can be reverted the depends_on configuration too.

I'll add this for now at least for daily

But is it a problem to run so many stages at the same time? It looks like we are fixing an issue we don't have (yet?).

I don't have a strong preference with any of the options, but it looks simpler to just run everything at the same time if this is not a problem by now.

That's true, I'm just moving ahead and probably there would not be any issue.
I'll change it back, and if there are issues we can do this change then.

mrodm · 2024-09-17T13:50:09Z

.buildkite/pipeline.schedule-weekly.yml

+    depends_on:
+      - step: "check"
+        allow_failure: false


Same as in the daily job? Would it be interesting to trigger this after the other step testing 8.x has finished? (update depends_on field).

mrodm · 2024-09-17T13:51:20Z

.github/workflows/updatecli/updatecli.d/bump-latest-9x-snapshot-version.yml

+      matchpattern: '(STACK_VERSION:) 9\.[^\s]*\.[^\s]*'
+      replacepattern: '$1 {{ source "latestSnapshot" }}'
+
+  update-snapshot-label-weekly:


These steps are now failing in the current check, but they should work once this PR is merged and all the changes are in main branch.

Thanks for adding the updatecli config too 👍

jsoriano

LGTM, thanks!

jsoriano · 2024-09-17T19:10:57Z

.buildkite/pipeline.schedule-daily.yml

+    depends_on:
+      - step: "check"
+        allow_failure: false


Well, this is going to be temporary, I guess that some time after 9.0 is released, 7.x will become unmaintained and we will remove its stage.

If it depends on 8.x, will it be executed if 8.x fails? We want both to be executed even if the other fails, what is pretty common 😅

jsoriano · 2024-09-17T19:13:46Z

.buildkite/pipeline.yml

+    # skipped for now notification for tests related to STACK_VERSION: 9
    if: >
-      build.source == "trigger_job" &&  build.env('BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG') == "integrations-schedule-daily"
+      build.source == "trigger_job" &&  build.env('BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG') == "integrations-schedule-daily" && build.env('STACK_VERSION') !~ /^9/


jsoriano · 2024-09-17T19:15:14Z

.github/workflows/updatecli/updatecli.d/bump-latest-9x-snapshot-version.yml

+      matchpattern: '(STACK_VERSION:) 9\.[^\s]*\.[^\s]*'
+      replacepattern: '$1 {{ source "latestSnapshot" }}'
+
+  update-snapshot-label-weekly:


Thanks for adding the updatecli config too 👍

jsoriano

Thanks! 👍

elasticmachine · 2024-09-18T17:35:47Z

⏳ Build in-progress, with failures

Buildkite Build
Commit: 26db499

Failed CI Steps

History

💛 Build #16064 was flaky 9954b20
💔 Build #16039 failed c36dd31
💔 Build #15987 failed e6c4f01
💔 Build #15980 failed 7b2fe316efc84b31eb8532c670243fca23f33e07
💔 Build #15937 failed c1b56af5106f2b3d44b0fac65b4cf7e30aa2de0a

cc @mrodm

elastic-sonarqube · 2024-09-18T19:14:57Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

Add support for 9.0 stack in Buildkite scripts as well as adding the steps required in daily and weekly pipelines to start testing packages with a local stack 9.0.0-SNAPSHOT.

This PR removes the workaround to test packages using Elastic stack with version 9.x if packages were just supporting up to stack version "^8.x.y" introduced in #11138.

mrodm self-assigned this Sep 16, 2024

mrodm mentioned this pull request Sep 16, 2024

EPR returns almost no packages when kibana.version of 9.0.0 is provided elastic/package-registry#1226

Closed

4 tasks

mrodm added 13 commits September 17, 2024 11:32

Add support for 9.0 stack

2348f2b

Test STACK_VERSION 9.0.0-SNAPSHOT

90042ff

Allow to test packages with constraint ^8.x for STACK_VERSION 9.0.0

40f864d

Fix comments

ea11883

Test again just 9.0.0 stack

4d3ba1e

Restore allowing constraint 8.0.0 when testing with 9.0.0 stack

be020b3

Update pipelines

5538d14

Update label for weekly step

0dcfdcc

Remove STACK_VERSION env var

c4f13fd

Update CI docs

a0cbd5b

Update cron links to daily and weekly pipelines

72f4e8d

Update stack version references

3455508

Update docs ci

e6c4f01

mrodm force-pushed the update-ci-9.0 branch from 7b2fe31 to e6c4f01 Compare September 17, 2024 10:16

mrodm commented Sep 17, 2024

View reviewed changes

mrodm marked this pull request as ready for review September 17, 2024 13:56

mrodm requested a review from a team as a code owner September 17, 2024 13:56

jsoriano approved these changes Sep 17, 2024

View reviewed changes

mrodm added 3 commits September 18, 2024 10:45

Update depends relationships in daily job

5163e39

Update cron for weekly job

c36dd31

Revert depends_on changes

9954b20

jsoriano approved these changes Sep 18, 2024

View reviewed changes

Test skipping reporting in 9.0

441f7e8

mrodm force-pushed the update-ci-9.0 branch from f688f46 to 441f7e8 Compare September 18, 2024 16:31

mrodm added 6 commits September 18, 2024 18:37

Update condition

846b407

Update condition

19ea0ef

Test without refspec

db0bfed

restore regex

c8ed84b

Test with other pipeline

bbeef79

Remove testing

26db499

mrodm merged commit 99a22db into elastic:main Sep 19, 2024

mrodm deleted the update-ci-9.0 branch September 19, 2024 08:37

mrodm mentioned this pull request Apr 10, 2025

[CI] Remove workaround test stack 9 packages in daily CI jobs #13491

Merged

3 tasks

Conversation

mrodm commented Sep 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Author's Checklist

Related issues

Uh oh!

elastic-vault-github-plugin-prod bot commented Sep 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Uh oh!

elasticmachine commented Sep 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

History

Uh oh!

mrodm Sep 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrodm Sep 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Sep 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⏳ Build in-progress, with failures

Failed CI Steps

History

Uh oh!

elastic-sonarqube bot commented Sep 18, 2024

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mrodm commented Sep 16, 2024 •

edited

Loading

elastic-vault-github-plugin-prod bot commented Sep 16, 2024 •

edited

Loading

elasticmachine commented Sep 17, 2024 •

edited

Loading

mrodm Sep 17, 2024 •

edited

Loading

mrodm Sep 18, 2024 •

edited

Loading

elasticmachine commented Sep 18, 2024 •

edited

Loading