Skip to content

[Fleet] Chunk asset installation during package install#189045

Merged
xcrzx merged 2 commits intoelastic:mainfrom
xcrzx:memory-optimization-3
Jul 25, 2024
Merged

[Fleet] Chunk asset installation during package install#189045
xcrzx merged 2 commits intoelastic:mainfrom
xcrzx:memory-optimization-3

Conversation

@xcrzx
Copy link
Copy Markdown
Contributor

@xcrzx xcrzx commented Jul 24, 2024

Resolves: #189043

Summary

This PR limits the number of saved objects installed in a single request. During saved object installation a lot of auxiliary objects are created in memory. Chunking allows for the garbage collection of memory objects that are not needed for response.

Memory consumption before

Screenshot 2024-07-12 at 12 44 58

After
Screenshot 2024-07-18 at 11 53 30

@xcrzx xcrzx added technical debt Improvement of the software architecture and operational architecture release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area v8.16.0 labels Jul 24, 2024
@xcrzx xcrzx self-assigned this Jul 24, 2024
@xcrzx xcrzx requested a review from a team as a code owner July 24, 2024 10:01
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 24, 2024

Pinging @elastic/fleet (Team:Fleet)

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 24, 2024

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 24, 2024

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 24, 2024

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

juliaElastic
juliaElastic approved these changes Jul 24, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kibana-ci
Copy link
Copy Markdown

kibana-ci commented Jul 24, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Jest Integration Tests #3 / telemetry tasks detection-rules should send task metrics

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @xcrzx

@xcrzx xcrzx merged commit b7e66eb into elastic:main Jul 25, 2024
@kibanamachine kibanamachine added the backport:skip This PR does not require backporting label Jul 25, 2024
@xcrzx xcrzx deleted the memory-optimization-3 branch July 25, 2024 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@juliaElastic juliaElastic juliaElastic approved these changes

Assignees

@xcrzx xcrzx

Labels

backport:skip This PR does not require backporting Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team:Fleet Team label for Observability Data Collection Fleet team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. technical debt Improvement of the software architecture and operational architecture v8.16.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Fleet] Split asset installation into chunks to avoid high memory pressure

5 participants

@xcrzx @elasticmachine @kibana-ci @juliaElastic @kibanamachine