Java 17 and Jetty 12 upgrade#24866
Merged
Merged
Conversation
75ec372 to
2cd9557
Compare
c514a3e to
363b4c5
Compare
Contributor
|
@ethanyzhang imported this issue into IBM GitHub Enterprise |
129bef4 to
621b86d
Compare
This could be done in a single file, but by splitting this up it will allows contributors and maintainers to incrementally re- enable the modernizer on each module since it will cause lots of code change. It will also introduce lots of opportunities for new contributors to get involved with Presto.
- Spark requires certain versions of jersey, javax.rs.ws-api and javax.serlet-api in order to run. This change ensures that we use the correct library versions at runtime. The versions chosen correspond to those used in Spark 2.4.4 which is the spark version inside of Presto's spark-core dependency - Use java 17 bytecode for PoS integration tests
This change expands the OVERRIDE_JDK_DIR environment configuration to the tempto runner in addition to the existing behavior for the presto coordinator and workers inside of the product tests suite. run_tempto.sh was updated to detect the overridden jdk volume when executing in order to make the tests run on a compatible JDK since product tests compiles to java 17 bytecode by default
This makes URI compliance checks in the REST catalog server more lax during tests in order to comply with the request URIs generated by the Iceberg rest catalog spec
Due to plugin and dependency updates, some modules' surefire execution picks up a junit runner instead of the testng one. This changes forces the use of testng in all presto modules.
presto-testing-mysql still depends on io.airlift:units instead of com.facebook.airlift:units. This causes issues for the tests which set MySqlOptions since the builder type only accepts units. Rather than include the io.airlift:units dependency, this commit removes the use of the builder options which accept a Duration type in order to allow the project to compile. This commit can be reverted if we release a new version of presto-testing-mysql with the new com.facebook.airlift:units dependency.
Airlift's HttpUriBuilder expects that uris passed to it are not yet encoded. However, because we use a semicolon ';' to delimit function names and parameters in the function server the semicolon which is encoded to %3B gets double-encoded to %253B by the URI builder (% encodes to %25). Older versions of Jetty allow this sequence in the URI. Newer servers however reject this request encoded with %25xxx with an error stating "AMBIGUOUS_PATH_PARAMETER". If we avoid using the HttpUriBuilder, we don't double-encode the semicolon, and the requests will work against the new server
- remove airline from classpath by excluding it from com.teradata.tpcds:tpcds - force presto-product-tests to shade airline by disabling the generation of the dependency-reduced-pom - change message() to body() in StatementClient to get correct error messages
downgrade guice to 6.0.0 for Java 8 compat
Downgrade guice to 6.0.0. replace all jakarta.inject.Provider with javax.inject.Provider. Update airlift version to one with guice 6.0.0. This makes it easier to integrate libraries that use javax with Presto.
This diff contains multiple changes that go together 1. We remove the dependency on presto-main module as it is targetted to compile for JDK17 but presto-on-spark* modules compile to JDK8/11. 2. Removing dependency on presto-main means that we have to figure out a solution to provide the missing functionality on which we depended. The only way to to do this is to duplicate the necessary classes and their dependencies 3. Next, we now have to override http-client,http-server and discovery to be compatible with JDK8/11. For this we pin the airlift version to 0.216 and the corresponding jetty version to 9.8.45.vXXXXX 4. Next we still need to do some amount of migration of classes as we moved from using javax -> jakarta and from io.airlift to com.facebook.airlift. Note that because for some of the airlift jars we still use 0.216 so it expects old io.airlift classes. So we retain both io.airlift and com.facebook.airlift 5. To manage the duplication we also exclude some classes from duplicate class check as the signature of the classes is the same
tdcmeehan
approved these changes
Aug 12, 2025
Closed
This was referenced Sep 11, 2025
This was referenced Sep 24, 2025
17 tasks
17 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change brings in the latest dependencies for airbase, airlift, drift, discovery-server, airline, and units. The latest airlift utilizes Jetty 12. In order to perform this upgrade in Presto we need to move a lot of dependencies from javax to jakarta namespace since Jetty 12 uses those dependencies. This necessitated a large migration across the codebase. There aren't too many complicated changes here other than some test fixes due to the upgrades. Mostly it is renaming and re-organizing imports.
Modules which rely on jetty through the airlift HTTP server and HTTP client dependencies have been updated to compile to Java 17 bytecode through the the property
<project.build.targetJdk>in each modules' respective pom.xml files. Client-facing dependencies have been kept on Java 8 bytecode such as presto-common, presto-spi, presto-client, and presto-jdbc. Using Java 17 bytecode means that Java 17 is now required to build and run Presto after this PR is merged.See this document for details about upgrading Presto deployments after this change has been merged
Motivation and Context
Jetty 12 has performance enhancements over our previous Jetty 9.X line and resolves many security vulnerabilities. It also helps modernize Presto and bring it closer to the most recent Java versions.
Impact
--add-opensarguments need to be added to users'jvm.configfilesTest Plan
Contributor checklist
Release Notes