[Native] Gradle-related tweaks to improve handling of the simdvec native library#144539
[Native] Gradle-related tweaks to improve handling of the simdvec native library#144539ldematte merged 3 commits intoelastic:mainfrom
Conversation
…epend on native libraries.
|
Pinging @elastic/es-delivery (Team:Delivery) |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThis change refactors how native library dependencies are consumed and passed to build tasks. The native library dependency declarations are updated to use a new ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Comment Tip CodeRabbit can generate a title for your PR based on the changes.Add |
ChrisHegarty
left a comment
There was a problem hiding this comment.
LGTM. Thanks @ldematte
* upstream/main: (146 commits) Revert "[Native] Gradle-related tweaks to improve handling of the simdvec native library (elastic#144539)" Fix ArrayIndexOutOfBoundsException in fetch phase with partial results (elastic#144385) ESQL: Correctly manage NULL data type for SUM (elastic#144942) [ESQL] Fixes GroupedTopNBenchmark not executing (elastic#144944) Fix reader context leak when query response serialization fails (elastic#144708) Validate individual offset values in BULK_OFFSETS bounds checks (elastic#144643) Merge main21 source set into main in simdvec (elastic#144921) [TEST] Unmute TsidExtractingIdFieldMapperTests (elastic#144848) [Native] Gradle-related tweaks to improve handling of the simdvec native library (elastic#144539) Fix `ThreadedActionListenerTests#testRejectionHandling` (elastic#144795) Add new DLM Frozen Tier Transition execution plugin and service (elastic#144595) Prometheus: execute query_range via parsed EsqlStatement plan (elastic#144416) Investigate `testBulkIndexingRequestSplitting` failure (elastic#144766) Add test utility for wrapping directories in FilterDirectory layer (elastic#143563) Fix ES|QL decay tests with negative scale (elastic#144657) Fix circuit breaker leak in percolator query construction (elastic#144827) Use XPerFieldDocValuesFormat in AbstractTSDBSyntheticIdCodec (elastic#144744) [DOCS] Document how reindex work in CPS (elastic#144016) Fix Int4 vector library tests failing on Java 21 (elastic#144830) [DiskBBQ] Fix index sorting on flush (elastic#144938) ...
…ive library (elastic#144539) This PR introduces a couple of changes related to native libraries build; In the script used to build and publish the native vec library, force gradle to rebuild native intermediate files to ensure a clean and up-to-date binary. Under some conditions, we ended with object files that were not at the latest version. Probably these were extreme conditions (locally built libraries, experimenting different solutions), but better be safe. The library is tiny and takes seconds to build anyway. Update gradle build files of benchmarks and KnnIndexTester to explicitly depend on native-libraries. Most of the code in benchmarks and KnnIndexTester depends on the native simdvec library, and it happened in the past that we unknowingly run with older, not updated versions of the library. In particular, we want the :libs:native:native-libraries:extractLibs to appear in the task graph for both :benchmarks:run and :qa:vector:checkVec.
…dvec native library (elastic#144539)" This reverts commit 568254d.
…ive library (elastic#144539) This PR introduces a couple of changes related to native libraries build; In the script used to build and publish the native vec library, force gradle to rebuild native intermediate files to ensure a clean and up-to-date binary. Under some conditions, we ended with object files that were not at the latest version. Probably these were extreme conditions (locally built libraries, experimenting different solutions), but better be safe. The library is tiny and takes seconds to build anyway. Update gradle build files of benchmarks and KnnIndexTester to explicitly depend on native-libraries. Most of the code in benchmarks and KnnIndexTester depends on the native simdvec library, and it happened in the past that we unknowingly run with older, not updated versions of the library. In particular, we want the :libs:native:native-libraries:extractLibs to appear in the task graph for both :benchmarks:run and :qa:vector:checkVec.
…dvec native library (elastic#144539)" This reverts commit 568254d.
…ive library (elastic#144539) This PR introduces a couple of changes related to native libraries build; In the script used to build and publish the native vec library, force gradle to rebuild native intermediate files to ensure a clean and up-to-date binary. Under some conditions, we ended with object files that were not at the latest version. Probably these were extreme conditions (locally built libraries, experimenting different solutions), but better be safe. The library is tiny and takes seconds to build anyway. Update gradle build files of benchmarks and KnnIndexTester to explicitly depend on native-libraries. Most of the code in benchmarks and KnnIndexTester depends on the native simdvec library, and it happened in the past that we unknowingly run with older, not updated versions of the library. In particular, we want the :libs:native:native-libraries:extractLibs to appear in the task graph for both :benchmarks:run and :qa:vector:checkVec.
…dvec native library (elastic#144539)" This reverts commit 568254d.
This PR introduces a couple of changes related to native libraries build;
In the script used to build and publish the native vec library, force gradle to rebuild native intermediate files to ensure a clean and up-to-date binary. Under some conditions, we ended with object files that were not at the latest version. Probably these were extreme conditions (locally built libraries, experimenting different solutions), but better be safe. The library is tiny and takes seconds to build anyway.
Update gradle build files of benchmarks and KnnIndexTester to explicitly depend on native-libraries. Most of the code in benchmarks and KnnIndexTester depends on the native simdvec library, and it happened in the past that we unknowingly run with older, not updated versions of the library. In particular, we want the
:libs:native:native-libraries:extractLibsto appear in the task graph for both:benchmarks:runand:qa:vector:checkVec.