PromQL: infer start/end from query DSL filters#142580
PromQL: infer start/end from query DSL filters#142580felixbarny merged 7 commits intoelastic:mainfrom
Conversation
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
| return switch (filter) { | ||
| case RangeQueryBuilder range when TIMESTAMP_FIELD.equals(range.fieldName()) -> range; | ||
| case BoolQueryBuilder bool -> { | ||
| for (QueryBuilder clause : bool.filter()) { |
There was a problem hiding this comment.
Should we combine the range filters from boolean clauses (e.g., timestamps < .. AND timestamps > ...)? Also, if a boolean query contains two complete range queries, we currently pick a random filter from them. Would that cause issues?
There was a problem hiding this comment.
I think we can keep it simple for now. This is targeted at extracting filters that Kibana adds and they always add a single range query lower and upper bounds rather than separate boolean filters.
In a perfect world, I'd prefer the request object to have first class support for start/end time filtering so that Kibana doesn't need to use an opaque Query DSL filter. But that would mean a lot of changes for Kibana and introducing a dependency on that work to be completed. Still something we may want to do long-term.
There was a problem hiding this comment.
I think we can keep it simple for now.
Yes, I am okay with this.
dnhatn
left a comment
There was a problem hiding this comment.
One question, but this looks great. Thanks Felix.
…on-sliced-reindex * upstream/main: (120 commits) [Fleet] Add OpAMP field mappings to fleet-agents (elastic#142550) Clarify `expectedSize` behaviour of `ReleasableBytesStreamOutput` (elastic#142451) Refactor KnnIndexTester to tidy up some options (elastic#142651) Fixed with elastic#142638 already (elastic#142655) Change *OverTimeTests to extend AbstractAggregationTestCase (elastic#142659) Fix byteRefBlockHashSize for release mode (elastic#142668) Mute org.elasticsearch.xpack.esql.tree.EsqlNodeSubclassTests testTransform {class org.elasticsearch.xpack.esql.plan.logical.MMR} elastic#142674 Fix PAUSED_FOR_NODE_REMOVAL shard blocking QUEUED promotion (elastic#142637) Mute org.elasticsearch.xpack.logsdb.RandomizedRollingUpgradeIT testIndexingStandardSource elastic#142670 Revert "[ESQL] Introduce pluggable external datasource framework (elastic#141678) (elastic#142663) Mute org.elasticsearch.xpack.esql.spatial.SpatialPushDownGeoShapeIT testQuantizedXY elastic#141234 PromQL: infer start/end from query DSL filters (elastic#142580) Add GPU vector indexing monitoring to _xpack/usage (elastic#141932) Fix testTrackerClearShutdown: use non-zero startTimeMillis for DONE status (elastic#142646) Mute org.elasticsearch.xpack.esql.qa.single_node.GenerativeIT test elastic#142426 ESQL_ Move time_zone to GA (elastic#142287) Mute org.elasticsearch.xpack.esql.qa.multi_node.GenerativeIT test elastic#142426 DOCS: Convert Painless diagrams to mermaid (elastic#141851) ES|QL: fix validation in generative tests (elastic#142638) Unmute tests that do not reproduce failures (elastic#141712) ...
This change lets PromQL range queries in ES|QL infer missing
start/endbounds from the request-level Query DSL filter when those bounds are not explicitly provided in the command. The goal is to reduce friction for users who already constrain time in the filter and currently need to repeat the same window in PromQL parameters in order to take advantage of the auto-bucketing introduced in #142582.At planning time,
EsqlSessionnow checks for aPromqlCommandthat is a range query without a time range, extracts@timestampbounds fromrequest.filter(), and rewrites the plan with inferred literal bounds. For example, a request filtered by@timestampfrom2025-01-01T00:00:00Zto2025-01-02T00:00:00Zwill now automatically execute PromQL with those boundaries.Implementation adds
QueryDslTimestampBoundsExtractor, which currently supports a directRangeQueryBuilderon@timestampand recursive lookup throughBoolQueryBuilderfilter/mustclauses. Parsing is format-aware via the range query format and is intentionally defensive: invalid or unsupported values return no bounds instead of failing the request. It is designed to work with the Query DSL filter Kibana adds to the request.User impact is improved usability and lower risk of mismatched time windows between DSL filters and PromQL arguments, while preserving existing behavior when explicit PromQL bounds are set.
Current limitation: date math expressions such as
now-15mare not inferred yet; they safely fall back to existing behavior and can be addressed in follow-up work.This is in context of #142578 where automatically determining start/end also enables to automatically determine the step based on a target bucket count.