KAFKA-19664 test#2
Closed
dejan2609 wants to merge 14 commits into
Closed
Conversation
…rsion (i.e. file `unixStartScript.txt`) was being referenced
details:
- gradle: 8.14.3 -->> 9.0.0
- https://github.com/gradle/gradle/releases/tag/v9.0.0
- https://gradle.org/whats-new/gradle-9
- https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#changes_major_9
- shadow plugin: 8.3.6 -->> 9.0.2
- https://github.com/GradleUp/shadow/releases/tag/9.0.0
- https://github.com/GradleUp/shadow/releases/tag/9.0.1
- https://github.com/GradleUp/shadow/releases/tag/9.0.2
refactoring (performed in order to accomodate Gradle 9 breaking changes):
- tasks `siteDocsTar` and `releaseTarGz` are moved out of `core` (and into a new dummy module: `distribution`); note that distribution destination folder stays as-is (:core)
- `preserveFileTimestamps` Tar taks property value explicitly set to `true` (default values: Gradle 8: `true`, Gradle 9: 'false')
- files and folders permissions are introduced
- git placeholder file (.gitkeep) is added in order to preserve `distribution` folder (Apache Rat check for that file is skipped)
- `wrapper.gradle` changes: no need to check number of dots for Gradle versions >=9 anymore
- redundant 'gradleVersion' ext property is removed (in favor of 'versions.gradle' usage)
… dependencies/plugins and GitHub Actions details: - `setup-gradle` GitHub Action: 4.3.0 -->> 4.4.3 (note: this version was tested against Gradle 9) - grgit: 4.1.1 -->> 5.3.0 - gradle versions plugin: 0.48.0 -->> 0.52.0 - owasp dependency check : 8.2.1 -->> 12.1.3 - spotbugs: 6.2.3 -->> 6.2.5 - spotless: 6.25.0 -->> 7.2.1 - Zinc Scala compiler: 1.9.2 -->> 1.10.8
… a `upgrade-system-tests*` submodules related link: https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#test_task_fails_when_no_tests_are_discovered
…hen no test are discovered (i.e. solution for KAFKA-16801 is expanded)
details: support for Java 25: https://docs.gradle.org/9.1.0/release-notes.html#support-for-java-25
note: spotbugs related Gradle tasks temporarily are disabled for Java 25
…o more places) note: `build / JUnit tests Java 25-ea` Github Action results with a previous commit: https://github.com/apache/kafka/actions/runs/17853628508/job/50769215024?pr=20561 ``` Gradle task had a failure exit code. Failing this script. 25373 tests cases run in 1h17m30s. 17471 PASSED ✅, 5676 FAILED ❌, 0 FLAKY⚠️ , 0 SKIPPED 🙈, 0 QUARANTINED 😷, and 0 errors. Error: Process completed with exit code 1. Error: $GITHUB_STEP_SUMMARY upload aborted, supports content up to a size of 1024k, got 6145k. For more information see: https://docs.github.com/actions/using-workflows/workflow-commands-for-github-actions#adding-a-markdown-summary ```
…ed on two more places)" This reverts commit ff1add5.
Scala 2.13.17 is published on Maven Central: https://contributors.scala-lang.org/t/scala-2-13-17-release-planning/6994/17
dejan2609
pushed a commit
that referenced
this pull request
Feb 12, 2026
…ker provisioning (apache#21394) ## Summary Fixes bugs where `--jdk-version` and `--jdk-arch` parameters were ignored during system test worker provisioning, and refactors `vagrant/base.sh` to support flexible JDK versions without code changes. --- ## Problem The Vagrant provisioning script (`vagrant/base.sh`) had two bugs that caused JDK version parameters to be ignored: | Bug | Problem | |-----|---------| | **#1: `--jdk-version` ignored** | `JDK_FULL` was hardcoded to `17-linux-x64`, so passing `--jdk-version 25` still downloaded JDK 17 | | **#2: `--jdk-arch` ignored** | Architecture parameter was passed but never used in the S3 download URL | --- ## Solution - Validate `JDK_MAJOR` and `JDK_ARCH` input parameters with regex - Dynamically construct `JDK_FULL` from `JDK_MAJOR` and `JDK_ARCH` - Update S3 path to use `/jdk/` subdirectory - Add logging for debugging --- ## Changes ### `vagrant/base.sh` | Change | Description | |--------|-------------| | **Input validation** | Added regex validation for `JDK_MAJOR` and `JDK_ARCH` with sensible defaults | | **Dynamic construction** | `JDK_FULL` is now constructed from `JDK_MAJOR` and `JDK_ARCH` if not explicitly provided | | **Updated S3 path** | Changed URL from `/kafka-packages/jdk-{version}.tar.gz` to `/kafka-packages/jdk/jdk-{version}.tar.gz` | | **Logging** | Added debug output for JDK configuration | | **Backward compatibility** | Vagrantfile can still pass `JDK_FULL` directly; the script validates and uses it if valid | --- ## S3 Path Change ### Old Path ``` s3://kafka-packages/jdk-{version}.tar.gz ``` ### New Path ``` s3://kafka-packages/jdk/jdk-{version}.tar.gz ``` ### Available JDKs in `s3://kafka-packages/jdk/` | File | Version | Architecture | |------|---------|--------------| | `jdk-7u80-linux-x64.tar.gz` | 7u80 | x64 | | `jdk-8u144-linux-x64.tar.gz` | 8u144 | x64 | | `jdk-8u161-linux-x64.tar.gz` | 8u161 | x64 | | `jdk-8u171-linux-x64.tar.gz` | 8u171 | x64 | | `jdk-8u191-linux-x64.tar.gz` | 8u191 | x64 | | `jdk-8u202-linux-x64.tar.gz` | 8u202 | x64 | | `jdk-11.0.2-linux-x64.tar.gz` | 11.0.2 | x64 | | `jdk-17-linux-x64.tar.gz` | 17 | x64 | | `jdk-18.0.2-linux-x64.tar.gz` | 18.0.2 | x64 | | `jdk-21.0.1-linux-x64.tar.gz` | 21.0.1 | x64 | | `jdk-21.0.1-linux-aarch64.tar.gz` | 21.0.1 | aarch64 | | `jdk-25-linux-x64.tar.gz` | 25 | x64 | | `jdk-25-linux-aarch64.tar.gz` | 25 | aarch64 | | `jdk-25.0.1-linux-x64.tar.gz` | 25.0.1 | x64 | | `jdk-25.0.1-linux-aarch64.tar.gz` | 25.0.1 | aarch64 | | `jdk-25.0.2-linux-x64.tar.gz` | 25.0.2 | x64 | | `jdk-25.0.2-linux-aarch64.tar.gz` | 25.0.2 | aarch64 | --- ## Future JDK Releases > **IMPORTANT: No code changes required for future Java major/minor releases!** The validation regex supports all version formats: - **Major versions**: `17`, `25`, `26` - **Minor versions**: `25.0.1`, `25.0.2`, `26.0.1` - **Legacy format**: `8u144`, `8u202` ### Adding New JDK Versions To add support for a new JDK version (e.g., JDK 26, 25.0.3): 1. Download the JDK tarball from Oracle/Adoptium 2. Rename to follow naming convention: `jdk-{VERSION}-linux-{ARCH}.tar.gz` 3. Upload to S3: `aws s3 cp jdk-{VERSION}-linux-{ARCH}.tar.gz s3://kafka-packages/jdk/` 4. Use in tests: `--jdk-version {VERSION} --jdk-arch {ARCH}` No modifications to `base.sh` or any other scripts are needed. --- ## Benefits | Before | After | |--------|-------| | `--jdk-version` ignored | ✅ Correctly uses specified version | | `--jdk-arch` ignored | ✅ Correctly uses specified architecture | | Only major version support | ✅ Full version support (e.g., `25.0.2`) | | Code change needed for new JDK | ✅ Just upload to S3 and pass version | --- ## Testing Tested with different JDK versions to confirm the fix works correctly: | Test | JDK_MAJOR | Expected | Actual | Result | Test Report | |------|-----------|----------|--------|--------|-------------| | JDK 17 | `17` | javac 17.0.4 | javac 17.0.4 | ✅ | | JDK 25 | `25` | javac 25.0.2 | javac 25.0.2 | ✅ | --- ## Backward Compatibility - **Vagrantfile**: Continues to work as before - **Existing workflows**: Default behavior unchanged (JDK 17 on x64 architecture) - **No breaking changes**: All existing configurations continue to work --- Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a PR on my own fork.