Skip to content

GH-39663: [C++] Ensure top-level benchmarks present informative metrics#40091

Merged
pitrou merged 1 commit intoapache:mainfrom
pitrou:gh39663-benchmark-metrics
Feb 19, 2024
Merged

GH-39663: [C++] Ensure top-level benchmarks present informative metrics#40091
pitrou merged 1 commit intoapache:mainfrom
pitrou:gh39663-benchmark-metrics

Conversation

@pitrou
Copy link
Copy Markdown
Member

@pitrou pitrou commented Feb 15, 2024

Rationale for this change

Some benchmarks may present only an iteration time, or not present sufficiently informative metrics.

What changes are included in this PR?

Add bytes/second and/or items/second metrics to top-level benchmarks where applicable.

This PR only tackles miscellaneous benchmarks from the top-level Arrow directory, as well as IO, IPC and utilities.

Are these changes tested?

Manually.

Are there any user-facing changes?

No.

… metrics

Add bytes/second and/or items/second metrics where applicable.
@github-actions
Copy link
Copy Markdown

⚠️ GitHub issue #39663 has been automatically assigned in GitHub to PR creator.

@pitrou
Copy link
Copy Markdown
Member Author

pitrou commented Feb 15, 2024

@ursabot please benchmark lang=C++

@pitrou pitrou requested a review from felipecrv February 15, 2024 10:24
@ursabot
Copy link
Copy Markdown

ursabot commented Feb 15, 2024

Benchmark runs are scheduled for commit c10547a. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

@conbench-apache-arrow
Copy link
Copy Markdown

Thanks for your patience. Conbench analyzed the 1 benchmarking run that has been run so far on PR commit c10547a.

There were 2 benchmark results indicating a performance regression:

The full Conbench report has more details.

Copy link
Copy Markdown
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Feb 15, 2024
@pitrou pitrou merged commit b224c58 into apache:main Feb 19, 2024
@pitrou pitrou removed the awaiting merge Awaiting merge label Feb 19, 2024
@pitrou pitrou deleted the gh39663-benchmark-metrics branch February 19, 2024 15:10
@conbench-apache-arrow
Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit b224c58.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++] Top-level Arrow benchmarks should present a items/s or bytes/s metric

4 participants