Skip to content

Improve report to show duration/i (e.g: 12.34 ns/i).#132

Merged
nateberkopec merged 1 commit intoevanphx:masterfrom
huacnlee:feat/report-duration-per-iter
Jan 2, 2024
Merged

Improve report to show duration/i (e.g: 12.34 ns/i).#132
nateberkopec merged 1 commit intoevanphx:masterfrom
huacnlee:feat/report-duration-per-iter

Conversation

@huacnlee
Copy link
Contributor

@huacnlee huacnlee commented Dec 20, 2023 •
edited
Loading

After update, the new report will look like:

ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [arm64-darwin23]
Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s

In some time we need to know how fast (in avg time), the before report only show the iterations/s, but we don't know how long of each iteration.

So I add the duration of each iteration in the report.

This may be useful to compare the performance with other languages, (e.g. Go, Rust etc.) they have the duration in per iteration.

For example Go:

Benchmark_string_alloc-8            	1000000000	         0.3361 ns/op

Use this code to test result:

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "benchmark-ips", github: "huacnlee/benchmark-ips", branch: "feat/report-duration-per-iter"
end

require "benchmark/ips"

text = "hello world"
part = "world"

Benchmark.ips do |x|
  x.config(time: 2, warmup: 1)

  x.report("string.new") do
    a = "hello world"
  end
  x.report("string.concat") do
    text + part
  end
  x.report("string.split") do
    text.split(" ")
  end
end

@huacnlee huacnlee force-pushed the feat/report-duration-per-iter branch from 81e8c7d to 58450fb Compare December 20, 2023 06:56
@huacnlee huacnlee changed the title Improve report to show duration/i (e.g: 12.24 ns/i). Improve report to show duration/i (e.g: 12.34 ns/i). Dec 20, 2023
@nateberkopec
Copy link
Collaborator

nateberkopec commented Jan 1, 2024 •
edited
Loading

I like the feature, but I'm not sure about the presentation of it.

I think I would prefer 22.780M (± 4.9%) i/s (43.90 ns/i) - 46.044M in 2.027883s)

@huacnlee
Copy link
Contributor Author

huacnlee commented Jan 2, 2024 •
edited
Loading

I have updated, and I left 15 width for (duration/i) for tidy print.

The new result:

Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s

@huacnlee
Improve report to show duration/i, e.g: 12.34 ns/i.
4e406c4 
```
Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s
```
@huacnlee huacnlee force-pushed the feat/report-duration-per-iter branch from 69dcf93 to 4e406c4 Compare January 2, 2024 02:56
nateberkopec
nateberkopec reviewed Jan 2, 2024
View reviewed changes
README.md
addition: 4955278.9 i/s - 4.85x slower
addition2: 36558904.5 i/s
addition3: 36359284.0 i/s - same-ish: difference falls within error
addition-test-long-label: 36135428.8 i/s - same-ish: difference falls within error
Copy link
Collaborator

@nateberkopec nateberkopec Jan 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating this!

Copy link

@Bodarobi Bodarobi Jan 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fasza

@nateberkopec nateberkopec merged commit 00c1428 into evanphx:master Jan 2, 2024
@huacnlee huacnlee deleted the feat/report-duration-per-iter branch July 10, 2024 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nateberkopec nateberkopec nateberkopec left review comments

+1 more reviewer

@Bodarobi Bodarobi Bodarobi left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@huacnlee @nateberkopec @Bodarobi