Skip to content

[8.19] [Build] Split tasks by arch for increased parallelism (#223471)#226892

Merged
Ikuni17 merged 2 commits intoelastic:8.19from
Ikuni17:backport/8.19/pr-223471
Jul 17, 2025
Merged

[8.19] [Build] Split tasks by arch for increased parallelism (#223471)#226892
Ikuni17 merged 2 commits intoelastic:8.19from
Ikuni17:backport/8.19/pr-223471

Conversation

@Ikuni17
Copy link
Copy Markdown
Contributor

@Ikuni17 Ikuni17 commented Jul 7, 2025

Backport

This will backport the following commits from main to 8.19:

Questions ?

Please refer to the Backport tool documentation

## Summary
Closes elastic/kibana-operations#110

This is a continuation of elastic#217929. Previously a `Task` created two
artifacts of a specific flavor, one for `x64` and one for `ARM64`. These
were still running serially, even though the `Task` itself was
parallelized. This PR splits each architecture into its own `Task` so we
can increase parallelism. The result is a 14% further decrease in time
taken for the build step; 53 min down to 46 min. Between the two PRs,
the artifact build time is reduced 57%!

### Testing
https://buildkite.com/elastic/kibana-artifacts-snapshot/builds/6413

### Next steps
- Moving the ARM artifact builds to an ARM native machine would likely
see the next biggest reduction because they take 2-3 times longer than
their x86 counterparts. Though this would likely be a decent amount of
work.
- Some of the `global` tasks could be grouped into parallel tasks, but
there is some dependency between those tasks and the gains would be
minimal.

(cherry picked from commit b51fab2)

# Conflicts:
#	src/dev/build/tasks/os_packages/create_os_package_tasks.ts
@Ikuni17 Ikuni17 requested a review from kibanamachine as a code owner July 7, 2025 19:57
@Ikuni17 Ikuni17 added the backport This PR is a backport of another PR label Jul 7, 2025
@Ikuni17 Ikuni17 enabled auto-merge (squash) July 7, 2025 19:57
@Ikuni17
Copy link
Copy Markdown
Contributor Author

Ikuni17 commented Jul 7, 2025

buildkite test this

@Ikuni17 Ikuni17 disabled auto-merge July 7, 2025 20:20
@Ikuni17 Ikuni17 self-assigned this Jul 7, 2025
@Ikuni17 Ikuni17 requested a review from a team July 7, 2025 20:20
@elasticmachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

✅ unchanged

cc @Ikuni17

@Ikuni17 Ikuni17 merged commit c955c81 into elastic:8.19 Jul 17, 2025
8 checks passed
@Ikuni17 Ikuni17 deleted the backport/8.19/pr-223471 branch July 17, 2025 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR ci:build-os-packages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants