Add GPU vector indexing monitoring to _xpack/usage#141932
Merged
elasticsearchmachine merged 15 commits intoelastic:mainfrom Feb 18, 2026
Merged
Add GPU vector indexing monitoring to _xpack/usage#141932elasticsearchmachine merged 15 commits intoelastic:mainfrom
elasticsearchmachine merged 15 commits intoelastic:mainfrom
Conversation
Collaborator
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
56f957b to
726c7d6
Compare
Expose GPU vector indexing usage via the _xpack/usage and _xpack APIs
GET _xpack response (features section):
```json
{
"gpu_vector_indexing": {
"available": true,
"enabled": true
}
}
```
_xpack field semantics (local node only):
- available: License permits GPU indexing
- enabled: the node handling this request has GPU hardware and
hasn't disabled it via vectors.indexing.use_gpu=false
GET _xpack/usage response:
```json
{
"gpu_vector_indexing": {
"available": true,
"enabled": true,
"index_build_count": 30,
"nodes_with_gpu": 3,
"nodes": [
{ "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
"enabled": true, "index_build_count": 10 },
{ "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
"enabled": true, "index_build_count": 10 },
{ "type": "NVIDIA A100", "memory_in_bytes": 80000000000,
"enabled": true, "index_build_count": 10 }
]
}
}
```
_xpack/usage field semantics (cluster-wide):
- available: License permits GPU indexing
- enabled: at least one node has configured GPU hardware and hasn't disabled
it via vectors.indexing.use_gpu=false
- index_build_count: total GPU index builds across cluster
- nodes_with_gpu: count of data nodes with GPU support
- nodes[]: per-node GPU details (type, memory, enabled, build count)
726c7d6 to
74f965e
Compare
ChrisHegarty
reviewed
Feb 6, 2026
libs/gpu-codec/src/main/java/org/elasticsearch/gpu/codec/ES92GpuHnswVectorsWriter.java
Show resolved
Hide resolved
...re/src/test/java/org/elasticsearch/xpack/core/gpu/GpuVectorIndexingFeatureSetUsageTests.java
Show resolved
Hide resolved
x-pack/plugin/gpu/src/test/java/org/elasticsearch/xpack/gpu/GpuUsageTransportActionTests.java
Show resolved
Hide resolved
- Add no-GPU scenario to randomUsage() for better test coverage - Make buildUsage() static to enable direct unit testing - Simplify GpuUsageTransportActionTests by removing mock boilerplate - Add test for empty cluster response scenario - Fix licenseNotAvailable test: index_build_count must be 0 - Verify GPU names are reported even when setting is disabled
The GPU yaml REST tests call the _xpack/usage and _xpack/info APIs, which internally iterate over all registered XPack feature actions (XPackUsageFeatureAction.ALL / XPackInfoFeatureAction.ALL). With the INTEG_TEST distribution (the default), only x-pack-core and x-pack-security modules are loaded, so action handlers for other features (e.g. aggregate_metric) are missing, causing a 500 error. Switch to DistributionType.DEFAULT so the full set of XPack modules is available and the usage/info endpoints can query all features.
Introduces gpu_vector_indexing_telemetry transport version for proper serialization of GpuVectorIndexingFeatureSetUsage in mixed clusters. Without a proper transport version, sending GPU usage data to older nodes that don't have the NamedWriteable registered would cause deserialization failures. With this change, GPU telemetry is filtered out when responding to nodes older than 9.3. Transport version IDs: - 9.4: 9277000 - 9.3: 9250004
Contributor
Author
|
@ChrisHegarty Do you have any additional feedback? |
Remove per-node index_build_count assertion since with 2 nodes and 1 shard, the indexing node may not be nodes.0. The cluster-level index_build_count assertion already validates indexing occurred. Also use filter_path to return only gpu_vector_indexing from xpack.usage.
Contributor
Author
|
@elasticsearchmachine run |
Collaborator
💔 Backport failedThe backport operation could not be completed due to the following error: You can use sqren/backport to manually backport by running |
mayya-sharipova
added a commit
to mayya-sharipova/elasticsearch
that referenced
this pull request
Feb 18, 2026
Expose GPU vector indexing usage via the _xpack/usage and _xpack APIs
GET _xpack response (features section):
```json
{
"gpu_vector_indexing": {
"available": true,
"enabled": true
}
}
```
_xpack field semantics (local node only): - available: License permits
GPU indexing - enabled: the node handling this request has GPU hardware
and hasn't disabled it via vectors.indexing.use_gpu=false
GET _xpack/usage response:
```json
{
"gpu_vector_indexing": {
"available": true,
"enabled": true,
"index_build_count": 30,
"nodes_with_gpu": 3,
"nodes": [
{ "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
"enabled": true, "index_build_count": 10 },
{ "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
"enabled": true, "index_build_count": 10 },
{ "type": "NVIDIA A100", "memory_in_bytes": 80000000000,
"enabled": true, "index_build_count": 10 }
]
}
}
```
_xpack/usage field semantics (cluster-wide): - available: License
permits GPU indexing - enabled: at least one node has configured GPU
hardware and hasn't disabled it via vectors.indexing.use_gpu=false -
index_build_count: total GPU index builds across cluster -
nodes_with_gpu: count of data nodes with GPU support - nodes[]: per-node
GPU details (type, memory, enabled, build count)
Backport for elastic#141932
elasticsearchmachine
pushed a commit
that referenced
this pull request
Feb 18, 2026
Expose GPU vector indexing usage via the _xpack/usage and _xpack APIs
GET _xpack response (features section):
```json
{
"gpu_vector_indexing": {
"available": true,
"enabled": true
}
}
```
_xpack field semantics (local node only): - available: License permits
GPU indexing - enabled: the node handling this request has GPU hardware
and hasn't disabled it via vectors.indexing.use_gpu=false
GET _xpack/usage response:
```json
{
"gpu_vector_indexing": {
"available": true,
"enabled": true,
"index_build_count": 30,
"nodes_with_gpu": 3,
"nodes": [
{ "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
"enabled": true, "index_build_count": 10 },
{ "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
"enabled": true, "index_build_count": 10 },
{ "type": "NVIDIA A100", "memory_in_bytes": 80000000000,
"enabled": true, "index_build_count": 10 }
]
}
}
```
_xpack/usage field semantics (cluster-wide): - available: License
permits GPU indexing - enabled: at least one node has configured GPU
hardware and hasn't disabled it via vectors.indexing.use_gpu=false -
index_build_count: total GPU index builds across cluster -
nodes_with_gpu: count of data nodes with GPU support - nodes[]: per-node
GPU details (type, memory, enabled, build count)
Backport for #141932
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Feb 19, 2026
…on-sliced-reindex * upstream/main: (120 commits) [Fleet] Add OpAMP field mappings to fleet-agents (elastic#142550) Clarify `expectedSize` behaviour of `ReleasableBytesStreamOutput` (elastic#142451) Refactor KnnIndexTester to tidy up some options (elastic#142651) Fixed with elastic#142638 already (elastic#142655) Change *OverTimeTests to extend AbstractAggregationTestCase (elastic#142659) Fix byteRefBlockHashSize for release mode (elastic#142668) Mute org.elasticsearch.xpack.esql.tree.EsqlNodeSubclassTests testTransform {class org.elasticsearch.xpack.esql.plan.logical.MMR} elastic#142674 Fix PAUSED_FOR_NODE_REMOVAL shard blocking QUEUED promotion (elastic#142637) Mute org.elasticsearch.xpack.logsdb.RandomizedRollingUpgradeIT testIndexingStandardSource elastic#142670 Revert "[ESQL] Introduce pluggable external datasource framework (elastic#141678) (elastic#142663) Mute org.elasticsearch.xpack.esql.spatial.SpatialPushDownGeoShapeIT testQuantizedXY elastic#141234 PromQL: infer start/end from query DSL filters (elastic#142580) Add GPU vector indexing monitoring to _xpack/usage (elastic#141932) Fix testTrackerClearShutdown: use non-zero startTimeMillis for DONE status (elastic#142646) Mute org.elasticsearch.xpack.esql.qa.single_node.GenerativeIT test elastic#142426 ESQL_ Move time_zone to GA (elastic#142287) Mute org.elasticsearch.xpack.esql.qa.multi_node.GenerativeIT test elastic#142426 DOCS: Convert Painless diagrams to mermaid (elastic#141851) ES|QL: fix validation in generative tests (elastic#142638) Unmute tests that do not reproduce failures (elastic#141712) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Expose GPU vector indexing usage via the _xpack/usage and _xpack APIs
GET _xpack response (features section):
{ "gpu_vector_indexing": { "available": true, "enabled": true } }_xpack field semantics (local node only):
GET _xpack/usage response:
{ "gpu_vector_indexing": { "available": true, "enabled": true, "index_build_count": 30, "nodes_with_gpu": 3, "nodes": [ { "type": "NVIDIA L4", "memory_in_bytes": 24000000000, "enabled": true, "index_build_count": 10 }, { "type": "NVIDIA L4", "memory_in_bytes": 24000000000, "enabled": true, "index_build_count": 10 }, { "type": "NVIDIA A100", "memory_in_bytes": 80000000000, "enabled": true, "index_build_count": 10 } ] } }_xpack/usage field semantics (cluster-wide):