Skip to content

Consolidate simdvec source sets and restore warnings-as-errors #144797

@ChrisHegarty

Description

@ChrisHegarty

The simdvec module has accumulated significant build and code complexity through its multi-release JAR (MR-JAR) layout, which splits the codebase across main, main21, and main22 source sets. This means 29+ source files live in main21, a handful of stub classes in main exist solely to satisfy the compiler (they throw UnsupportedOperationException and are never executed), and developers must reason about which source set a file belongs to and how the MR-JAR classloader selects between them. The main21 source set also compiles without -Werror due to an unsuppressible incubating module warning - a workaround that weakens build hygiene.

This meta-issue tracks simplifying and consolidating the simdvec codebase: suppressing the incubating warning centrally so -Werror can be restored, merging all source sets into main, removing stub classes, and eliminating the MR-JAR plugin entirely.


The primary reason main21 exists as a separate source set is that it uses jdk.incubator.vector, which emits an unsuppressible warning on JDK 21:

warning: using incubating module(s): jdk.incubator.vector

This warning cannot be individually suppressed on JDK 21 because the -Xlint:-incubating flag was only added in JDK 22 (JDK-8187591 (​https://bugs.openjdk.org/browse/JDK-8187591​)). Since the Elasticsearch build enforces -Werror, the workaround has been to remove -Werror for the main21 compile task. This is undesirable since it means all the source files in main21 compile without warnings-as-errors. The same workaround exists in swisshash/build.gradle.

The -Xlint:none trick

On JDK 21, the incubating module warning is a "mandatory" warning that sits outside the named -Xlint category system. It is emitted by default and with -Xlint:all, but it is suppressed by -Xlint:none. Crucially, when -Xlint:none is followed by -Xlint:all,..., javac re-enables only the named lint categories, while the unnamed incubating warning stays suppressed. So the compiler args:

  -Xlint:none -Xlint:all,-path,-serial,-options,-deprecation,-try,-removal -Werror

suppress the incubating warning while still failing on all other warnings.

Verified behavior across JDK versions:

JDK -Xlint:none then -Xlint:all,... Incubating suppressed? Real warnings caught?
21 Yes — incubating is unnamed, stays suppressed Yes Yes
22–24 Yes — incubating is named but not re-enabled by all after none Yes Yes
25 No — incubating is re-enabled by all No Yes

For JDK 25+, -incubating must be added to the -Xlint exclusion list (the flag is available from JDK 22 onwards, but is an error on JDK 21). The fix should therefore be version-aware: always prepend -Xlint:none, and additionally exclude -incubating when the compiler JDK is >= 22.

Motivation

  • Restore warnings-as-errors for all simdvec source files
  • Simplify the build — remove MR-JAR complexity (separate source sets, stub classes, MR-JAR classloader interactions)
  • Simplify the code — eliminate "compile-time only" stub classes in main that throw UnsupportedOperationException and are never executed at runtime
  • Reduce developer friction — no more confusion about which source set a file belongs to, or why the same class exists in multiple source sets

Tasks

  • 1. Suppress incubating warning in ElasticsearchJavaBasePlugin

    Prepend -Xlint:none before -Xlint:all,... in ElasticsearchJavaBasePlugin.configureCompile. Add version-aware -incubating exclusion for JDK 22+ compilers. Remove the options.compilerArgs -= '-Werror'
    workarounds from simdvec/build.gradle and swisshash/build.gradle. Verify both modules compile cleanly with -Werror active. (Separate PR — small, low-risk, independently valuable.) Suppress incubating module warning to retain -Werror #144798

  • 2. Merge main21 source set into main in simdvec

    Move all 29 files from src/main21/java/ into src/main/java/. Replace stub classes in main (e.g., VectorScorerFactoryImpl, ESVectorizationProvider) with their real main21 implementations. Add
    --add-modules=jdk.incubator.vector and --add-reads to the main compileJava task. Move test21/IndexInputUtilsTests.java into test/. Update build.gradle to remove the main21 task configuration. (Depends
    on task 1.) Merge main21 source set into main in simdvec #144921

  • 3. Merge main22 source set into main in simdvec

    The 6 files in main22 override main21 versions (e.g., ByteVectorScorer, Int7SQVectorScorer). Merge these into main, using runtime version checks where JDK 22-specific behavior differs (e.g., SUPPORTS_HEAP_SEGMENTS). Merge main22 source set into main in simdvec #145005

  • 4. Remove MR-JAR plugin from simdvec

    Once no mainNN source sets remain, remove apply plugin: 'elasticsearch.mrjar' from simdvec/build.gradle. Remove the Multi-Release: true manifest attribute. Clean up any MR-JAR-specific test
    infrastructure. (Depends on tasks 2 and 3.)

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions