Skip to content

textparse: Refactored benchmark#15083

Merged
bwplotka merged 3 commits intomainfrom
textparsebench
Oct 7, 2024
Merged

textparse: Refactored benchmark#15083
bwplotka merged 3 commits intomainfrom
textparsebench

Conversation

@bwplotka
Copy link
Member

@bwplotka bwplotka commented Oct 3, 2024

For more realistic benchmarks to iterate over proto improvements we planned, #14965 and perhaps experimenting with CI use with https://github.com/benchmark-action/github-action-benchmark or so, I propose a refactor of the benchmark we want to persist in our textparse package.

To me we want to persist useful stuff to iterate on, but more importantly, something anyone should run before commiting new changes. For this reasons I killed those isolated cases for series, CT, series+metrics, series+metrics reuse. Instead we only have benchmark for realistic use of parser (as we use in scrape.append) AND opt-in (skipped by default) expfmt comparison. Anyone can comment out certain methods for local testing and comparisons.

Additionally:

  • I provided example CLI command
  • Fixed cases formatting to match official syntax (you can learn more in https://www.bwplotka.dev/2024/go-microbenchmarks-benchstat/).
  • Killed unnecessary sinks. With interfaces and internal state, I don't see any difference if I have sinks or not (that total var).
  • I changed operation to be = full scrape, not a single sample. I might miss why we did this per sample, but it feels unnecessary and complex. Each sample is different so perhaps it's simpler to think about ops = scrape (which we care about).
  • Removed gzip benchmark, I guess it was for some experiment, but not sure if worth to benchmark always not ours gzip implementation we don't implement.

Would love to know your opinion @beorn7 and @bboreham on this 🤗

Here is an example of what this benchmark enables us to do #14965 (comment)

@bwplotka
Copy link
Member Author

bwplotka commented Oct 4, 2024

cc @krajorama as it will be useful for nhcb work too 🤗

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Copy link
Member

@krajorama krajorama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job, I've got some nit comments. This should be easy to adopt to #15115 if it ever gets so far.

Signed-off-by: bwplotka <bwplotka@gmail.com>
Copy link
Member

@krajorama krajorama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bwplotka bwplotka merged commit 8d281c3 into main Oct 7, 2024
@bwplotka bwplotka deleted the textparsebench branch October 7, 2024 08:55
squat pushed a commit to squat/prometheus that referenced this pull request Nov 8, 2024
* textparse: Refactored benchmark.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Kill sinks, tested, nothing is inlined.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
squat pushed a commit to squat/prometheus that referenced this pull request Nov 8, 2024
* textparse: Refactored benchmark.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Kill sinks, tested, nothing is inlined.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
squat pushed a commit to squat/prometheus that referenced this pull request Nov 8, 2024
* textparse: Refactored benchmark.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Kill sinks, tested, nothing is inlined.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
squat pushed a commit to squat/prometheus that referenced this pull request Nov 25, 2024
* textparse: Refactored benchmark.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Kill sinks, tested, nothing is inlined.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
julienduchesne pushed a commit to julienduchesne/prometheus that referenced this pull request Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants