Please note that in #133 I said nothing about "quickly comparing methods on an object". Please don't make this take an obj argument that you send on. benchmark-ips is MUCH more than running microbenchmarks on Array to know that reduce is slower.

Some code needs more complex setup.
Some code is comparing that setup.
Or anything else.

This argument limits things and would require me to make a class with my benchmark methods to get around it. THE POINT of this method is to require less structure to get benchmarks up and running and get an easy comparison of anything.

ary =  [Object.new] * 969
ary += [Assertion.new] * 3
ary += [UnexpectedError.new] * 1
ary += [Skip.new] * 27

RESULTS = ary

def report_orig
  aggregate = RESULTS.group_by { |r| r.class }
  aggregate.default = [] # dumb. group_by should provide this

  [aggregate[Assertion].size, aggregate[UnexpectedError].size, aggregate[Skip].size]
end

def report_orig2
  aggregate = RESULTS.group_by { |r| r.class }.transform_values(&:size)
  aggregate.default = 0

  [aggregate[Assertion], aggregate[UnexpectedError], aggregate[Skip]]
end

def report_tally
  aggregate = RESULTS.map { |r| r.class }
  aggregate = if RUBY_VERSION > "3.1" then
                aggregate.tally(Hash.new 0)
              else
                aggregate.tally.tap { |r| r.default = 0 }
              end

  [aggregate[Assertion], aggregate[UnexpectedError], aggregate[Skip]]
end

def report_tally2
  aggregate = RESULTS.map { |r| r.class }
  aggregate = aggregate.tally.tap { |r| r.default = 0 }

  [aggregate[Assertion], aggregate[UnexpectedError], aggregate[Skip]]
end

def report_tally3
  aggregate = RESULTS.map { |r| r.class }.tally
  aggregate.default = 0

  [aggregate[Assertion], aggregate[UnexpectedError], aggregate[Skip]]
end

Correct, you didn't say anything about testing on an object. But without a receiver, who do you want the methods called on? just toplevel methods?

I moved quick_compare to Kernel so it's more flexible, more along the lines of your original version.

What's the problem with Benchmark::IPS.quick_compare(self, :report_orig, :report_orig2, ...)?
Having the object is useful if you have more state you need to pass in.
Then for instance this state could live in an object instance.
Or the methods would be defined as singleton methods of a module, to have the necessary constant scope, etc.

I am against polluting Kernel.
I think a benchmarking harness should not monkey-patch anything.

Also for 10+ years users of this gem managed just fine without this helper method, so maybe we should not add it.
At least I think we should get multiple opinions to not add something which only works for one person.
The existing API is also the same as stdlib Benchmark, it's pretty simple.

Can we use regular kwargs here?

How about using one of the faster forms, like the explicit while loop, or string form?

I wonder if having quick in the name would make people think that it runs faster than the normal version

Just saw the version bump here. Changing to 2.14.