[PERF] PromQL: Split rangeEval for binary operations#16698
Draft
bboreham wants to merge 6 commits intoprometheus:mainfrom
Draft
[PERF] PromQL: Split rangeEval for binary operations#16698bboreham wants to merge 6 commits intoprometheus:mainfrom
bboreham wants to merge 6 commits intoprometheus:mainfrom
Conversation
Contributor
|
I've raised #16797 extending the perf gains due to concrete types, without the code duplication of splitting |
Member
Author
|
A quick scan suggests this one is faster. Did I miss something? |
Contributor
|
Can you run the full benchmark please? Since this PR only updates the binops, other operations should be unaffected. I will run mine with |
Member
Author
|
I ran the full set of benchmarks; there were some regressions like Happy to accept #16797, maybe with some tweaks, but I still want to break out |
bboreham
pushed a commit
that referenced
this pull request
Jul 23, 2025
Currently, the promql functions take the interface slice []parser.Value as an argument, which is being implemented by the conrete types Vector, Matrix etc. This PR replaces the interface with the concrete types, resulting in improved performance. The inspiration for this PR came from #16698 which does this for binops. I extended the idea to all promql functions Signed-off-by: darshanime <deathbullet@gmail.com> * pass single Matrix Signed-off-by: darshanime <deathbullet@gmail.com> --------- Signed-off-by: darshanime <deathbullet@gmail.com>
tcp13equals2
pushed a commit
to tcp13equals2/prometheus
that referenced
this pull request
Aug 18, 2025
) Currently, the promql functions take the interface slice []parser.Value as an argument, which is being implemented by the conrete types Vector, Matrix etc. This PR replaces the interface with the concrete types, resulting in improved performance. The inspiration for this PR came from prometheus#16698 which does this for binops. I extended the idea to all promql functions Signed-off-by: darshanime <deathbullet@gmail.com> * pass single Matrix Signed-off-by: darshanime <deathbullet@gmail.com> --------- Signed-off-by: darshanime <deathbullet@gmail.com> Signed-off-by: Andrew Hall <andrew.hall@grafana.com>
To make subsequent changes easier. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
This means we can simplify the non-binop version that doesn't need the `prepSeries` function. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
This is more efficient as it avoids converting Vector arguments to parser.Value. Allow matching parameter to be nil to signal we don't need signatures. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
7bc5038 to
375727e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Update: the performance improvement got done #16797; this is now a refactor]
Binary operations generally involve matching left-hand series against right-hand series, for which we had the
prepSeriesfunction pre-computing signatures.The genesis of this refactor is that I didn't understand what was going on in #9577, so I tried to simplify the code leading up to it.
Along the way, some other functions got extracted from
rangeEval, and an annotation merge got hoisted.VectorAnd,VectorOrandVectorUnlessnow return nilAnnotationsso they can be used directly as the evaluation function;VectorscalarBinopshould also returnAnnotationsbut I'll leave that for another PR.Further work:
rangeEval: two taking no expressions, one taking one, and one for function calls. Probably there are benefits to making each case less generic.rangeEvalBinOpcould compute series matches (via a map) once, and then use them over and over at each time step. This is not trivial as the same LHS can match different RHS as series come and go, but I think it will be a big win for large queries.rangeEvalBinOpcould implement the intended optimisation from Optimise VectorAnd for step invariant expression #9577 where one or other side is time-invariant.benchmark results
(
go test -timeout 50m -count=6 -run xxx -bench RangeQuery ./promql)