Some benchmarks had errors
Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit ca35643.
Benchmarks with errors
These are errors that were caught while running the benchmarks. You can click each link to go to the Conbench entry for that benchmark, which might have more information about what the error was.
-
Pull Request Run on
arm64-t4g-2xlarge-linuxat 2025-06-12 13:10:19Ztpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-01, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-05, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-01, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-02, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-06, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-08, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-04, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-06, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-08, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-04, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-05, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-02, scale_factor=1
-
Pull Request Run on
amd64-m5-4xlarge-linuxat 2025-06-12 10:50:47Ztpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-04, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-12, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-08, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-05, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-05, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-08, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-06, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-13, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-04, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-09, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-12, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-10, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-02, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-10, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-10, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-08, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-14, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-11, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-01, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-06, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-04, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-13, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-13, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-02, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-06, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-06, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-13, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-11, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-11, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-14, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-10, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-02, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-01, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-05, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-01, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-14, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-08, scale_factor=1tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-11, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-07, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-01, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-02, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-04, scale_factor=10tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1tpch(R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-05, scale_factor=10
Benchmarks with performance regressions
There were 18 possible performance regressions, according to the lookback z-score method.
Benchmarks with regressions:
-
Pull Request Run on
arm64-t4g-2xlarge-linuxat 2025-06-12 13:10:19ZBM_ByteStreamSplitEncode_Float_Neon(C++) with params=32768, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Neon(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Neon(C++) with params=65536, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Neon(C++) with params=4096, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Double_Neon(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Double_Neon(C++) with params=32768, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Double_Generic(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Generic(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmark
-
Pull Request Run on
test-mac-armat 2025-06-12 13:42:18ZBM_ByteStreamSplitEncode_Float_Neon(C++) with params=4096, source=cpp-micro, suite=parquet-encoding-benchmarkBinaryBitOp(C++) with source=cpp-micro, suite=arrow-decimal-benchmarkBM_ByteStreamSplitEncode_Float_Generic(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Neon(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmark
-
Pull Request Run on
amd64-c6a-4xlarge-linuxat 2025-06-12 10:14:14Z -
Pull Request Run on
amd64-m5-4xlarge-linuxat 2025-06-12 10:50:47ZBM_ByteStreamSplitEncode_Float_Sse2(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Generic(C++) with params=65536, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Generic(C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ByteStreamSplitEncode_Float_Sse2(C++) with params=65536, source=cpp-micro, suite=parquet-encoding-benchmark
All benchmark runs analyzed:
- Pull Request Run on
arm64-t4g-2xlarge-linuxat 2025-06-12 13:10:19Z - Pull Request Run on
test-mac-armat 2025-06-12 13:42:18Z - Pull Request Run on
amd64-c6a-4xlarge-linuxat 2025-06-12 10:14:14Z - Pull Request Run on
amd64-m5-4xlarge-linuxat 2025-06-12 10:50:47Z
Details
This report was generated using the lookback z-score method with a z-score threshold of 30.0.