Testing: Run sync measure benchmarks in parallel#7113
Merged
dashpole merged 2 commits intoopen-telemetry:mainfrom Aug 6, 2025
Merged
Testing: Run sync measure benchmarks in parallel#7113dashpole merged 2 commits intoopen-telemetry:mainfrom
dashpole merged 2 commits intoopen-telemetry:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7113 +/- ##
=======================================
- Coverage 82.9% 82.9% -0.1%
=======================================
Files 262 262
Lines 24461 24461
=======================================
- Hits 20293 20291 -2
- Misses 3792 3794 +2
Partials 376 376 🚀 New features to boost your workflow:
|
pellared
approved these changes
Aug 4, 2025
Member
pellared
left a comment
There was a problem hiding this comment.
I think we can simply have the parallel benchmarks given making concurrent measurements is a typical use case.
MrAlias
approved these changes
Aug 5, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I am looking into https://promlabs.com/blog/2025/07/17/why-i-recommend-native-prometheus-instrumentation-over-opentelemetry/#comparing-counter-increment-performance, which seems to suggest the OTel metrics SDK performs poorly when a counter is incremented concurrently. It is potentially a bit of an artificial benchmark, but does suggest there is some contention beyond just the fact that they are incrementing an atomic integer...
Original benchmarks from the blog post: https://github.com/promlabs/prometheus-otel-benchmarks/blob/main/otel_test.go
Results are significantly worse (almost > 2x in some cases) with parallelism, but don't initially seem as bad as the blog post suggests. I only have 24 cores, so I can't test higher numbers.
Do we want to have parallel benchmarks in addition to our current non-parallel ones?