Only use MemorySegments on Java 25+#15338
Conversation
|
Thanks @chrisvest ! |
|
JBoss marshalling is getting caught with its fingers in the reflection jar by JDK 25 EA. I'll try upgrading it from 2.0.5 to 2.2.3, but I'm not sure that'll do much. I think it needs to catch |
This was indeed fixed to use permitted APIs since 2.2.0, so upgrading should get you around this problem. |
|
Thanks, David! |
Motivation: We wish to use memory segments for managing direct memory, as Unsafe is going away. However, we cannot use this feature unless the application is running on Java 25 or better due to the following JDK bugs: - https://bugs.openjdk.org/browse/JDK-8357145 - https://bugs.openjdk.org/browse/JDK-8357268 In Netty, we can work around those bugs. However, our downstream users are also getting `ByteBuffers` out of our `ByteBuf` instances, and making their own calls into the JDK APIs, and can thus also run into these bugs. Effectively, this breaks their systems when we enable memory segments on Java 24. Modification: Change CleanerJava24 to CleanerJava25, and only enable it on Java 25+. Add a build to the PR matrix for Java 25 Early Access. Add a nightly workflow that checks the latest Java 25 Early Access build version, which will fail if a new version is released. This workflow will help us stay up to date on the latest Java 25 Early Access version. We can remove it or disable it once Java 25 is released. Maybe we want something similar for Java 26 and so on, in the future. Result: We only use memory segments on Java versions where they don't run into buggy interactions. Fixes netty#15324
This changed in 2.2.0, presumably.
e91b612 to
17c00cc
Compare
|
Rather mysteriously, compilation fails in the |
|
The error message from I think we have to bump our marshalling dependency generally, and not just during compilation. Otherwise downstream users will run into this error, but in reverse. |
|
@chrisvest you also need to add a profile for |
|
@normanmaurer We're gonna have to skip blockhound entirely for Java 25 EA, at least for the current build 26. I think there's a JDK bug preventing it from working. |
|
@chrisvest judging from the failing build output I'd suspect that you need to upgrade blockhound to at least |
|
Let's merge this for now... @derklaro we can open a new PR and try your suggestion there. |
|
Thanks for taking care of this. Hadn't considered there was a new version of Blockhound. |
…ing to Netty change ### What changes were proposed in this pull request? This PR removes `-Dio.netty.noUnsafe=true` from the default JVM args in `values.yaml`. ### Why are the changes needed? Netty 4.2.2+ have a correct implementation instead of `Unsafe` usage for Java 25+ like the following instead of the legacy `Unsafe` operation. - netty/netty#15231 - netty/netty#15338 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs with the existing tests. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (claude-4-opus) Closes #593 from dongjoon-hyun/SPARK-56229. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Motivation:
We wish to use memory segments for managing direct memory, as Unsafe is going away. However, we cannot use this feature unless the application is running on Java 25 or better due to the following JDK bugs:
In Netty, we can work around those bugs.
However, our downstream users are also getting
ByteBuffersout of ourByteBufinstances, and making their own calls into the JDK APIs, and can thus also run into these bugs. Effectively, this breaks their systems when we enable memory segments on Java 24.Modification:
Change CleanerJava24 to CleanerJava25, and only enable it on Java 25+.
Add a build to the PR matrix for Java 25 Early Access.
Add a nightly workflow that checks the latest Java 25 Early Access build version, which will fail if a new version is released. This workflow will help us stay up to date on the latest Java 25 Early Access version. We can remove it or disable it once Java 25 is released. Maybe we want something similar for Java 26 and so on, in the future.
Result:
We only use memory segments on Java versions where they don't run into buggy interactions.
Fixes #15324