Skip to content

Increase Xmx used by JRuby during Rake execution to 4Gb#16911

Merged
andsel merged 1 commit intoelastic:mainfrom
andsel:fix/increase_memory_for_dra_tasks
Jan 19, 2025
Merged

Increase Xmx used by JRuby during Rake execution to 4Gb#16911
andsel merged 1 commit intoelastic:mainfrom
andsel:fix/increase_memory_for_dra_tasks

Conversation

@andsel
Copy link
Copy Markdown
Member

@andsel andsel commented Jan 19, 2025

Release notes

[rn:skip]

What does this PR do?

Increase to 4Gb the max heap memory used by JVM to run JRuby's Rake during DRA.

@andsel andsel self-assigned this Jan 19, 2025
@elastic-sonarqube
Copy link
Copy Markdown

elastic-sonarqube bot commented Jan 19, 2025

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jan 19, 2025

💚 Build Succeeded

cc @andsel

@andsel andsel marked this pull request as ready for review January 19, 2025 15:35
dliappis
dliappis approved these changes Jan 19, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@dliappis dliappis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

dliappis added a commit to dliappis/logstash that referenced this pull request Jan 19, 2025
@dliappis
Cherry pick from elastic#16911
a89235b
@dliappis
Copy link
Copy Markdown
Contributor

dliappis commented Jan 19, 2025

@andsel mostly curious, any idea what changed that suddenly makes us require more heap for builds from main? Kind related, how far back should it be backported?

@andsel andsel merged commit 58e6dac into elastic:main Jan 19, 2025
@andsel
Copy link
Copy Markdown
Member Author

andsel commented Jan 19, 2025

@dliappis looking at last part of the execption:

rake aborted!
--
  | Java::JavaLang::OutOfMemoryError: Java heap space
  | org.jruby.dist/org.jruby.ext.zlib.JZlibDeflate.run(JZlibDeflate.java:323)
  | org.jruby.dist/org.jruby.ext.zlib.JZlibDeflate.append(JZlibDeflate.java:280)
  | org.jruby.dist/org.jruby.ext.zlib.JZlibDeflate.deflate(JZlibDeflate.java:297)
  | org.jruby.dist/org.jruby.ext.zlib.JZlibDeflate.deflate(JZlibDeflate.java:232)

the problems comes from JRuby JZip deflater that tries to allocate triple the space it allocate before https://github.com/jruby/jruby/blob/97d04a057681b893705f2d08c5c9cf8ccca729c5/core/src/main/java/org/jruby/ext/zlib/JZlibDeflate.java#L323, but that code is untouched for years. So I don't think that's the source.

I don't know which changes could have generated this, maybe some other Ruby Gems or jars that increased in size, but my is just speculation.

@jsvd do you have better guess on this?

@andsel
Copy link
Copy Markdown
Member Author

andsel commented Jan 20, 2025

@logstashmachine backport 8.x

github-actions bot pushed a commit that referenced this pull request Jan 20, 2025
@andsel @logstashmachine
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911)
ce5ace5 
(cherry picked from commit 58e6dac)
@github-actions github-actions bot mentioned this pull request Jan 20, 2025
Merged
andsel added a commit that referenced this pull request Jan 20, 2025
@github-actions @andsel
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911) (#16914
094724c 
)

(cherry picked from commit 58e6dac)

Co-authored-by: Andrea Selva <selva.andre@gmail.com>
@andsel
Copy link
Copy Markdown
Member Author

andsel commented Jan 24, 2025

@logstashmachine backport 8.17

github-actions bot pushed a commit that referenced this pull request Jan 24, 2025
@andsel @logstashmachine
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911)
aed8ab0 
(cherry picked from commit 58e6dac)
@github-actions github-actions bot mentioned this pull request Jan 24, 2025
Merged
@andsel
Copy link
Copy Markdown
Member Author

andsel commented Jan 24, 2025

@logstashmachine backport 8.16

github-actions bot pushed a commit that referenced this pull request Jan 24, 2025
@andsel @logstashmachine
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911)
60341f4 
(cherry picked from commit 58e6dac)
@github-actions github-actions bot mentioned this pull request Jan 24, 2025
Merged
@andsel
Copy link
Copy Markdown
Member Author

andsel commented Jan 24, 2025

@logstashmachine backport 7.17

github-actions bot pushed a commit that referenced this pull request Jan 24, 2025
@andsel @logstashmachine
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911)
59df814 
(cherry picked from commit 58e6dac)
@github-actions github-actions bot mentioned this pull request Jan 24, 2025
Merged
andsel added a commit that referenced this pull request Jan 24, 2025
@github-actions @andsel
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911) (#16943
b9b5b95 
)

(cherry picked from commit 58e6dac)

Co-authored-by: Andrea Selva <selva.andre@gmail.com>
andsel added a commit that referenced this pull request Jan 24, 2025
@github-actions @andsel
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911) (#16944
f2a7b30 
)

(cherry picked from commit 58e6dac)

Co-authored-by: Andrea Selva <selva.andre@gmail.com>
andsel added a commit that referenced this pull request Jan 24, 2025
@github-actions @andsel
Increase Xmx used by JRuby during Rake execution to 4Gb (#16911) (#16945
d15b366 
)

(cherry picked from commit 58e6dac)

Co-authored-by: Andrea Selva <selva.andre@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@dliappis dliappis dliappis approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@andsel andsel

Labels

backport needed v7.17.28 v8.16.4 v8.17.2 v8.18.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@andsel @elasticmachine @dliappis