Skip to content

Conversation

@norberttech
Copy link
Member

Change Log

Added

  • ScalarFunction - arrayValues
  • ScalarFunction - arrayKeys
  • ScalarFunction - arrayKeep
  • ScalarFunction - arrayFilter

Fixed

Changed

Removed

Deprecated

Security


Description

@github-actions github-actions bot added size: M and removed size: S labels Jan 19, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jan 19, 2025

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors
+-----------------------+-------------------+------+-----+-----------------+------------------+----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev         |
+-----------------------+-------------------+------+-----+-----------------+------------------+----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.764mb +0.04%  | 535.911ms -0.43% | ±0.72% +3.81%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.831mb +0.04%  | 1.060s +0.93%    | ±0.41% -47.79% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 86.485mb +0.01% | 908.911ms +0.81% | ±0.36% +30.82% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.497mb +0.04%  | 34.269ms +1.48%  | ±1.31% +53.15% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.474mb +0.04%  | 599.043ms -1.11% | ±0.36% -43.16% |
+-----------------------+-------------------+------+-----+-----------------+------------------+----------------+
Transformers
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 108.492mb +0.00% | 59.888ms +0.35% | ±1.34% +50.11% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
Loaders
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev         |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.040mb +0.00%  | 100.417ms -0.81% | ±1.66% +87.43% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 76.800mb +0.00%  | 94.452ms +0.86%  | ±0.32% -34.39% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 166.984mb +0.00% | 20.742s -0.92%   | ±0.40% -41.20% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.083mb +0.01%  | 31.091ms -0.58%  | ±0.64% +64.19% |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
Building Blocks
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 80.257mb +0.00%  | 3.805ms +8.94%   | ±2.20% -23.25%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 97.535mb +0.01%  | 186.217ms -0.40% | ±0.57% +42.63%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 80.255mb +0.01%  | 18.811ms -0.35%  | ±0.67% -50.47%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 81.132mb +0.00%  | 1.824ms +13.33%  | ±2.73% +156.75% |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 81.132mb +0.00%  | 1.700ms +9.45%   | ±3.12% +225.67% |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 79.292mb +0.00%  | 3.798ms +3.18%   | ±1.62% +85.27%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 79.821mb +0.00%  | 15.064ms +0.23%  | ±0.26% -76.52%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 79.821mb +0.00%  | 15.119ms +1.30%  | ±0.26% -63.33%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 78.513mb +0.00%  | 1.900μs 0.00%    | ±0.00% 0.00%    |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 78.513mb +0.00%  | 0.400μs 0.00%    | ±0.00% 0.00%    |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.850mb +0.00%  | 12.753ms -1.66%  | ±0.67% -60.49%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 114.198mb +0.00% | 60.656ms -0.74%  | ±1.18% -35.71%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 80.341mb +0.00%  | 1.731ms +14.45%  | ±3.10% +75.84%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 83.635mb +0.01%  | 61.839ms -2.14%  | ±0.03% -97.16%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 81.394mb +0.00%  | 3.918ms +4.53%   | ±0.28% -80.23%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 78.797mb +0.01%  | 41.266ms -1.72%  | ±1.96% +168.83% |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 78.797mb +0.01%  | 41.331ms -3.47%  | ±0.18% -87.01%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 78.797mb +0.01%  | 41.860ms -4.93%  | ±0.74% +127.13% |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 80.953mb +0.00%  | 8.270ms -5.05%   | ±1.08% -64.60%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 78.703mb +0.00%  | 29.156ms -2.25%  | ±0.67% -56.10%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 78.513mb +0.00%  | 14.664μs -0.38%  | ±1.95% +67.94%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 78.513mb +0.00%  | 15.800μs -4.24%  | ±0.52% -47.78%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 97.536mb +0.01%  | 191.686ms -0.19% | ±2.00% +273.88% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 98.648mb +0.00%  | 439.212ms -1.46% | ±0.51% -3.45%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 51.479mb +0.00%  | 224.763ms +0.45% | ±0.36% -74.15%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 13.650mb +0.01%  | 48.789ms +1.13%  | ±0.16% -71.45%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 43.782mb +0.00%  | 361.746ms -0.52% | ±1.93% +94.90%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 11.592mb +0.02%  | 73.226ms +0.64%  | ±0.38% -78.90%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+

@codecov
Copy link

codecov bot commented Jan 19, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.56%. Comparing base (6d17959) to head (8debf7a).
Report is 2 commits behind head on 1.x.

Additional details and impacted files
@@            Coverage Diff             @@
##              1.x    #1391      +/-   ##
==========================================
+ Coverage   82.52%   82.56%   +0.03%     
==========================================
  Files         650      654       +4     
  Lines       17434    17472      +38     
==========================================
+ Hits        14388    14425      +37     
- Misses       3046     3047       +1     
Components Coverage Δ
etl 85.91% <100.00%> (+0.07%) ⬆️
cli 85.17% <ø> (ø)
lib-array-dot 94.53% <ø> (ø)
lib-azure-sdk 62.56% <ø> (ø)
lib-doctrine-dbal-bulk 97.36% <ø> (ø)
lib-filesystem 76.23% <ø> (ø)
lib-parquet 84.57% <ø> (ø)
lib-parquet-viewer 82.02% <ø> (ø)
lib-rdsl 87.09% <ø> (ø)
lib-snappy 90.69% <ø> (-0.47%) ⬇️
bridge-filesystem-async-aws 90.38% <ø> (ø)
bridge-filesystem-azure 89.92% <ø> (ø)
bridge-monolog-http 96.38% <ø> (ø)
symfony-http-foundation 77.10% <ø> (ø)
adapter-chartjs 86.45% <ø> (ø)
adapter-csv 89.49% <ø> (ø)
adapter-doctrine 90.14% <ø> (ø)
adapter-elasticsearch 97.19% <ø> (ø)
adapter-google-sheet 78.04% <ø> (ø)
adapter-http 59.15% <ø> (ø)
adapter-json 92.85% <ø> (ø)
adapter-logger 53.84% <ø> (ø)
adapter-meilisearch 97.75% <ø> (ø)
adapter-parquet 59.88% <ø> (ø)
adapter-text 84.44% <ø> (ø)
adapter-xml 83.15% <ø> (ø)

{
$array = (new Parameter($this->array))->asArray($row);

if (!\is_array($array)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!\is_array($array)) {
if (null === $array) {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it give anything?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strict null check will be cheaper, and asArray() can return either array or null.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a quick benchmark on 10mln iterations and the differences are almost invisible, looks like microoptimisation in one place won't generate enough impact.
It might be worth using rector/phpstan to enforce that everywhere

{
$array = (new Parameter($this->array))->asArray($row);

if (!\is_array($array)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!\is_array($array)) {
if (null === $array) {

Co-authored-by: Joseph Bielawski <stloyd@users.noreply.github.com>
@norberttech norberttech merged commit 52be2d7 into flow-php:1.x Jan 19, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants