Skip to content

Conversation

@norberttech
Copy link
Member

Change Log

Added

  • Parquet - DataPageV2 statistics

Fixed

Changed

Removed

Deprecated

Security


Description

Closes: #686

@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2023

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors
+-----------------------+-------------------+------+-----+------------------+-------------------+----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode              | rstdev         |
+-----------------------+-------------------+------+-----+------------------+-------------------+----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.140mb -0.02%  | 435.780ms +21.21% | ±0.24% -59.53% |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.790mb +0.02%   | 339.729ms +25.90% | ±0.34% +15.86% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.938mb +0.02%   | 653.813ms +12.51% | ±0.41% -31.05% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 233.527mb +0.00% | 927.935ms +17.98% | ±0.79% +26.72% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.784mb +0.02%   | 24.944ms +29.26%  | ±0.74% -64.46% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.784mb +0.02%   | 521.926ms +30.17% | ±0.45% +69.83% |
+-----------------------+-------------------+------+-----+------------------+-------------------+----------------+
Transformers
+-----------------------------+--------------------------+------+-----+-----------------+------------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak        | mode             | rstdev         |
+-----------------------------+--------------------------+------+-----+-----------------+------------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 87.050mb +0.00% | 62.021ms +26.19% | ±0.48% -33.64% |
+-----------------------------+--------------------------+------+-----+-----------------+------------------+----------------+
Loaders
+--------------------+----------------+------+-----+------------------+-------------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode              | rstdev         |
+--------------------+----------------+------+-----+------------------+-------------------+----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 93.291mb +0.00%  | 660.375ms +16.61% | ±0.33% +42.96% |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 46.067mb +0.00%  | 70.908ms +0.56%   | ±0.64% +7.93%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 88.569mb +0.00%  | 67.731ms +12.90%  | ±0.38% +7.74%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 284.040mb +0.00% | 1.486s +21.77%    | ±0.49% -20.05% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.560mb +0.01%  | 35.464ms -12.26%  | ±0.46% -64.52% |
+--------------------+----------------+------+-----+------------------+-------------------+----------------+
Building Blocks
+-------------------------+----------------------------+------+-----+-----------------+-------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak        | mode              | rstdev          |
+-------------------------+----------------------------+------+-----+-----------------+-------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 60.660mb +0.00% | 3.276ms +48.34%   | ±3.05% +157.81% |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 80.452mb +0.00% | 176.368ms +17.16% | ±0.44% -31.31%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 58.978mb +0.00% | 17.853ms +18.13%  | ±0.10% -90.22%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 59.798mb +0.00% | 3.038ms +74.16%   | ±3.71% +89.99%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 59.798mb +0.00% | 2.801ms +68.34%   | ±0.66% -57.89%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 59.012mb +0.00% | 3.795ms +44.86%   | ±0.99% +10.75%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 59.541mb +0.00% | 17.780ms +24.49%  | ±0.87% -49.03%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 59.540mb +0.00% | 17.637ms +25.14%  | ±0.38% -65.01%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 57.612mb +0.00% | 2.400μs +26.71%   | ±3.40% +34.72%  |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 57.612mb +0.00% | 0.500μs +25.00%   | ±0.00% -100.00% |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 65.845mb +0.00% | 13.542ms +30.84%  | ±2.25% -5.20%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 91.365mb +0.00% | 57.658ms +21.95%  | ±0.64% -34.16%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 60.061mb +0.00% | 2.873ms +52.67%   | ±1.02% -31.47%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 62.331mb +0.00% | 40.230ms +23.39%  | ±1.43% +239.04% |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 62.162mb +0.00% | 6.562ms +40.59%   | ±2.66% +318.88% |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 57.612mb +0.00% | 45.849ms +24.07%  | ±0.24% -16.69%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 57.612mb +0.00% | 45.695ms +21.57%  | ±0.42% -60.83%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 57.612mb +0.00% | 46.526ms +21.23%  | ±0.22% -86.49%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 59.886mb +0.00% | 9.689ms +30.25%   | ±0.27% -35.98%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 57.611mb +0.00% | 36.143ms +28.87%  | ±0.07% -39.00%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 57.612mb +0.00% | 20.734μs +63.10%  | ±0.99% +34.63%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 57.612mb +0.00% | 26.412μs +70.59%  | ±0.36% -74.28%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 80.453mb +0.00% | 179.332ms +12.86% | ±0.34% -31.69%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 91.738mb -0.00% | 142.177ms +19.95% | ±0.45% -80.59%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 47.596mb +0.00% | 73.505ms +27.32%  | ±0.60% +196.85% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 12.393mb +0.00% | 17.348ms +24.40%  | ±0.31% -85.69%  |
+-------------------------+----------------------------+------+-----+-----------------+-------------------+-----------------+

@norberttech norberttech merged commit e725f1e into flow-php:1.x Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parquet Statistics

1 participant