⚠️ GitHub issue #41015 has been automatically assigned in GitHub to PR creator.

@ursabot please benchmark lang=JavaScript

Benchmark runs are scheduled for commit e5f6c93. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

Where else do we call perf from? Could we just change all the calls to run yarn perf / npm run perf?

As far as I know, everything that runs benchmarks just uses yarn perf.

Now we have the same command in two places

If this is a problem, perhaps we should remove the shebang then? The trouble with the shebang as written is that using /usr/bin/env -S is non-portable. So if we're using yarn perf everywhere, it should be fine to remove the shebang.

Sounds good to me. We have a few other places that use the same shebang and I think we should update them as well.

Rather than yarn, maybe we should use npm run since it's more portable. But that can be a separate change.

We have a few other places that use the same shebang and I think we should update them as well.

Just looked into this briefly and found places in archery that might fail if we remove the shebangs:

So we could go through and fix all these, but it might be more trouble than it's worth. Would you mind if I keep this PR scoped to just fixing the benchmarks' portability?

(For context, I'm trying to move the orchestration of the Arrow merge-commit JS benchmarks off a machine in someone's basement to an EC2 instance. This PR unblocks that work.)

So we could go through and fix all these, but it might be more trouble than it's worth. Would you mind if I keep this PR scoped to just fixing the benchmarks' portability?

Agreed. So to unblock you, can you run node --no-warnings --loader ts-node/esm/transpile-only perf/index.ts from the perf script? Otherwise, sure, let's merge this as is. I approve either way.

Thanks for your patience. Conbench analyzed the 1 benchmarking run that has been run so far on PR commit e5f6c93.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

I think that might be slightly difficult on the orchestration side. Cool, thanks for the review. 💜 would you mind merging? I don't have write access.

After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 02bc653.

There was 1 benchmark result indicating a performance regression:

• Commit Run on ursa-i9-9960x at 2024-04-05 21:57:32Z
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-02, scale_factor=1

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.