Skip to content

promql: benchmark for join queries with more labels#17130

Merged
bwplotka merged 3 commits intoprometheus:mainfrom
linasm:benchmark-join-query
Oct 24, 2025
Merged

promql: benchmark for join queries with more labels#17130
bwplotka merged 3 commits intoprometheus:mainfrom
linasm:benchmark-join-query

Conversation

@linasm
Copy link
Contributor

@linasm linasm commented Sep 3, 2025 •
edited
Loading

I am working on a series of changes that will improve PromQL binary vector matching (AKA "joins") significantly. For evaluating the impact, I could not find any useful benchmarks.
The ones in bench_test.go only have one label l with values from 0 to 99:

prometheus/promql/bench_test.go

Lines 61 to 62 in 11c4915

metrics = append(metrics, labels.FromStrings("__name__", "a_hundred", "l", strconv.Itoa(i)))
metrics = append(metrics, labels.FromStrings("__name__", "b_hundred", "l", strconv.Itoa(i)))

But PromQL join performance heavily depends on the length of label sets, and in practice we see label sets that are at least hundreds of bytes long. Therefore I am submitting this new benchmark which I intend to use for evaluating join performance.

Does this PR introduce a user-facing change?

NONE

@linasm
promql: benchmark for join queries with more labels
eb6745c 
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
@linasm linasm requested a review from roidelapluie as a code owner September 3, 2025 06:37
@linasm
Copy link
Contributor Author

linasm commented Sep 3, 2025

Sample output:

BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	5092712708 ns/op	1580306584 B/op	  577787 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4857207125 ns/op	56548704 B/op	  562754 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4852032000 ns/op	56541232 B/op	  562739 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4831938500 ns/op	57055312 B/op	  562726 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4787814500 ns/op	56548600 B/op	  562720 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4692117417 ns/op	56542520 B/op	  562709 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4732971125 ns/op	57056192 B/op	  562719 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4886770834 ns/op	56538864 B/op	  562738 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4951542750 ns/op	56551416 B/op	  562742 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_+_rpc_request_error_total,steps=10000-10         	       1	4840886208 ns/op	57062504 B/op	  562730 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       1	1022672042 ns/op	82787360 B/op	  377614 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	1007236771 ns/op	83014016 B/op	  377553 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	 980386021 ns/op	82782516 B/op	  377610 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	 966500562 ns/op	83040232 B/op	  377694 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	 966786354 ns/op	83025608 B/op	  377611 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	 968533688 ns/op	83030076 B/op	  377588 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	1017864167 ns/op	83029864 B/op	  377578 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	 979216208 ns/op	82773764 B/op	  377553 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       2	1083539479 ns/op	83026588 B/op	  377629 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_AND_rpc_request_error_total{instance=~"0.*"},steps=10000-10         	       1	1006525625 ns/op	83279800 B/op	  377615 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2387791291 ns/op	1354795184 B/op	  641499 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2314223625 ns/op	885663072 B/op	  638940 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2311133584 ns/op	885650768 B/op	  638682 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2323486250 ns/op	885324944 B/op	  640115 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2307859791 ns/op	885217984 B/op	  639254 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2277869083 ns/op	885627584 B/op	  638521 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2268478583 ns/op	885614144 B/op	  638428 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2262297959 ns/op	885006512 B/op	  637839 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2265722041 ns/op	885164432 B/op	  638827 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_OR_rpc_request_error_total{instance=~"0.*"},steps=10000-10          	       1	2243000375 ns/op	885228336 B/op	  639380 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1756589291 ns/op	82844040 B/op	  377539 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1763694417 ns/op	83333800 B/op	  377418 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1791715542 ns/op	83313952 B/op	  377280 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1747951125 ns/op	83357608 B/op	  377530 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1803715625 ns/op	82852552 B/op	  377480 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1845421625 ns/op	83362192 B/op	  377615 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1752475833 ns/op	82847472 B/op	  377562 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1730626958 ns/op	83322880 B/op	  377342 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1801897625 ns/op	83365768 B/op	  377640 allocs/op
BenchmarkJoinQuery/expr=rpc_request_success_total_UNLESS_rpc_request_error_total{instance=~"0.*"},steps=10000-10      	       1	1764282667 ns/op	82857112 B/op	  377661 allocs/op
PASS
ok  	github.com/prometheus/prometheus/promql	201.377s

@linasm linasm mentioned this pull request Sep 3, 2025
Merged
@linasm
Add benchmark case for GROUP_LEFT
2a9ba3a 
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
@linasm linasm force-pushed the benchmark-join-query branch from beca574 to 2a9ba3a Compare September 4, 2025 09:32
@linasm linasm mentioned this pull request Sep 4, 2025
Open
bwplotka
bwplotka reviewed Oct 24, 2025
View reviewed changes
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epic! Some small questions / readability nits only, but generally great 💪🏽

bboreham
bboreham reviewed Oct 24, 2025
View reviewed changes
@linasm
Address PR feedback
dee96f7 
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
@linasm linasm force-pushed the benchmark-join-query branch from a032b05 to dee96f7 Compare October 24, 2025 11:17
@linasm
Copy link
Contributor Author

linasm commented Oct 24, 2025

@bboreham , @bwplotka - thanks, I have addressed your feedback.

@linasm linasm requested review from bboreham and bwplotka October 24, 2025 11:24
@bwplotka
Copy link
Member

bwplotka commented Oct 24, 2025

Thanks!

@bwplotka bwplotka merged commit 042fe9d into prometheus:main Oct 24, 2025
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bwplotka bwplotka bwplotka approved these changes

@roidelapluie roidelapluie Awaiting requested review from roidelapluie roidelapluie is a code owner

@bboreham bboreham Awaiting requested review from bboreham

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@linasm @bwplotka @bboreham