Graduate the QUIC codec from incubator to Netty Core#14979
Merged
Graduate the QUIC codec from incubator to Netty Core#14979
Conversation
Motivation: Let's use dependencies which have not known CVEs Modifications: Upgrade bouncycastle and logback Result: Cleanup
Motivation: There are new releases of java. Modifications: Upgrade java versions to latest release Result: Use latest versions on the CI
Motivation: A new version of netty was released Modifications: Upgrade to latest version Result: Use latest netty release
Motivation: When client auth is optional we must not fail the handshake on the client side when no keymanager was configured on the client side. Modifications: - Fix handling on the client-side - Add testcase - Fix test Result: Fixes netty/netty-incubator-codec-quic#566
Motivation: We are a bit behind, lets update to latest quiche sha. Modifications: Update to 5ddba81a3ab03c0fd4c262d444bded4208013669 Result: Use latest quiche code
Motivation: Let's use the commit sha of quiche for the 0.18.0 release. Modifications: - Change quiche sha - Skip building of apps as its not needed Result: Use sha that represent the latest quiche release
…etty#572) Motivation: When caching fails during workflow execution we should still continue the build as its just an optimization. Also we should use the new fork for docker layer caching Modifications: - Update action that does the docker layer caching to the new fork - Ignore errors during caching Result: More stable builds
Motivation: We need quite some diskspace due our ue of docker, let's ensure we have enough by free up what we dont need on the runner. Modifications: Use action that frees up diskspace Result: More stable build
Motivation: We should use weak references to hold global references to our own classes as otherwise it will be not possible to unload the classloader. Modifications: Use weak references for the classes in JNI. Result: Related to netty#13480
Motivation: There were updates to the chromium-stable branch of boringssl. Modifications: Update to latest sha Result: Use more recent boringssl version
Motivation: Since version 0.0.42.Final the connection from an android client was not working anymore, caused by a QuicClosedChannelException. After inspecting the changes I found that 885a18a introduced the bug. Modifications: The call to Quiche.memoryAddress used the capacity of the wrong buffer. The bug only appeared on Android because on most other platforms the buf.hasMemoryAddress() is true, and the len parameter is not used therefore. Result: Make netty-incubator-codec-quic work on Android again
Motivation: The javaModuleNameClassifier was missing in the android profile and therefore the `AndroidManifest.xml` contained `package="io.netty.incubator.codec.quic.linux.x86_64"` Modifications: Added javaModuleNameClassifier property in pom.xml Result: Show the right package name in the `AndroidManifest.xml`
netty#579) … and Datagram extension is used Motivation: We can't assume we can directly access the memoryAddress of the ByteBuf as it might not be possible on all platforms or even explicit disabled by the end user Modifications: Use static helper method that will always be able to return the correct memory address Result: Code works as expected on all platforms
Motivation: We recently had a bug which only showed up when the memoryAddress of a direct buffer could not be accessed directly. We didnt catch this as the memory address always was accessible by unsafe. Let's add a PR build job which explicit disable unsafe. Modifications: - Add profile which disables unsafe during build - Add job which builds with unsafe disabled Result: Better test coverage
Motivation: Docker images can be quite large, lets try to cleanup things we not need. Modifications: - Cleanup things during docker image creation - Disable docker layer caching as it most of the times not work anyway Result: Docker image needs less disk-space
Motivation: There was a new maven release Modifications: Upgrade to 3.9.4 Result: Use latest maven release via maven wrapper
…s it is not needed anymore (netty#586) Motivation: Recently GraalVM added support for Apple M1. Modifications: - Remove the comment from resource-config.json file - Update the version for native-maven-plugin Result: The comment in resource-config.json is not needed anymore.
netty#587) …eference Motivation: We missed to delete the local reference in some cases due early returns. Modifications: Move the delete calls close to the creation calls and so ensure we not miss to delete local references Result: More correct code
…ty#588) Motivation: The code introduced as part of netty/netty-incubator-codec-quic#574 contained a bug which could crash the JVM. This only manifested in JDK21+ as earlier versions did not contain the related assert. Modifications: Remove left-over call that could crash the JVM Result: Correct code that also works on JDK21
Motivation: Now that JDK21 is released we should also build with it on our CI Modifications: Add docker-compose config for JDK21 and use it in the PR workflow Result: Test with latest LTS release
Motivation: We should run our testsuite with '-Xcheck:jni' to ensure we catch bugs in our JNI code which could later cause issues like crashes Modifications: - Add -Xcheck:jni - Add -ea Result: Testsuite will be able to catch more JNI bugs
Motivation: A new netty release is out. Modifications: Upgrade to latest release Result: Use latest release
Motivation: We should update all our used actions to the latest version Modifications: Update all actions Result: Cleanup
This was referenced Apr 2, 2025
…available The '$JAVA_HOME/bin/gu' binary is no longer part of the graalce distribution.
Motivation: We only link against the quiche package so we only need to build it with cargo Modifications: Only build quiche package Result: Speedup build and cleanup
Motivation: We should link against the latest quiche release Modifications: Update to 0.23.4 Result: Use sha that matches latest quiche release
Motivation: Let's ensure we can build with latest netty 4.2.0 release Modifications: Change version to latest release Result: Validate everything works with latest netty 4.2.0 release
# Conflicts: # docker/docker-compose.centos-7.yaml
…ing on Linux Also remove some duplicate test dependencies
|
Will every change in QUIC codec be applied to incubator first ? |
Member
Author
No, I think it'll be supported at about the same level as the rest of 4.1. That is, only bug and compatibility fixes from now on. |
dongjoon-hyun
added a commit
to apache/spark
that referenced
this pull request
Dec 5, 2025
…` license ### What changes were proposed in this pull request? This PR aims to update `NOTICE-binary` with `Netty` 4.2.7 license. - https://github.com/netty/netty/blob/netty-4.2.7.Final/NOTICE.txt ### Why are the changes needed? It seems that we updated `Netty Notice` at Apache Spark `3.0.0-preview` with `Netty 4.1.30.Final`. - #25544 Since there are many changes like the following, we need to update it by simply copying and pasting. - Netty 4.1.38.Final - netty/netty#9344 - Netty 4.1.44.Final - netty/netty#9161 - Netty 4.1.54.Final - netty/netty#10773 - Netty 4.1.66.Final - netty/netty#11256 - netty/netty#11437 - Netty 4.1.108.Final - netty/netty#13864 - Netty 4.2.1.Final - netty/netty#14979 - Netty 4.2.7.Final - netty/netty#15658 Additionally, I also double-checked newly added transitive license through ASF [LEGAL-700](https://issues.apache.org/jira/browse/LEGAL-700). We are good to go. - **Apple Public Source License 2.0** - https://spdx.org/licenses/APSL-2.0.html ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53335 from dongjoon-hyun/SPARK-54602. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun
added a commit
to apache/spark
that referenced
this pull request
Dec 5, 2025
…` license ### What changes were proposed in this pull request? This PR aims to update `NOTICE-binary` with `Netty` 4.2.7 license. - https://github.com/netty/netty/blob/netty-4.2.7.Final/NOTICE.txt ### Why are the changes needed? It seems that we updated `Netty Notice` at Apache Spark `3.0.0-preview` with `Netty 4.1.30.Final`. - #25544 Since there are many changes like the following, we need to update it by simply copying and pasting. - Netty 4.1.38.Final - netty/netty#9344 - Netty 4.1.44.Final - netty/netty#9161 - Netty 4.1.54.Final - netty/netty#10773 - Netty 4.1.66.Final - netty/netty#11256 - netty/netty#11437 - Netty 4.1.108.Final - netty/netty#13864 - Netty 4.2.1.Final - netty/netty#14979 - Netty 4.2.7.Final - netty/netty#15658 Additionally, I also double-checked newly added transitive license through ASF [LEGAL-700](https://issues.apache.org/jira/browse/LEGAL-700). We are good to go. - **Apple Public Source License 2.0** - https://spdx.org/licenses/APSL-2.0.html ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53335 from dongjoon-hyun/SPARK-54602. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 191ce4c) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
QUIC is the underlying transport protocol for HTTP/3.
The prior HTTP versions are all core protocols, so naturally HTTP/3 should be too.
Modification:
Copy over the source code from netty/netty-incubator-codec-quic@1a4b5dc.
Adjust the pom.xml files to fit into the project as a proper Netty submodule.
Result:
The QUIC module has graduated from incubator.
Fixes netty/netty-incubator-codec-quic#580