Skip to content

Conversation

@stloyd
Copy link
Member

@stloyd stloyd commented Nov 11, 2023

Change Log

Added

  • Add `MapEntry`
  • Add `EnumType`

Fixed

Changed

  • Rework `StructureEntry` to use types
  • Rework `NativeEntryFactory` to use types

Removed

Deprecated

Security


Description

Closes #684, #775

@stloyd stloyd force-pushed the feature/struct-entry branch from 7df7723 to c29b51a Compare November 12, 2023 09:41
@github-actions github-actions bot added the docs label Nov 12, 2023
@stloyd stloyd force-pushed the feature/struct-entry branch from c29b51a to 95dfe83 Compare November 12, 2023 09:43
@github-actions
Copy link
Contributor

github-actions bot commented Nov 12, 2023

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors
+-----------------------+-------------------+------+-----+------------------+-------------------+----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode              | rstdev         |
+-----------------------+-------------------+------+-----+------------------+-------------------+----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 34.719mb +0.10%  | 1.148s +209.66%   | ±0.80% -25.73% |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.579mb +0.68%   | 301.940ms +13.84% | ±0.66% -27.82% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.751mb +0.56%   | 1.375s +135.87%   | ±0.46% -58.82% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.448mb +0.01% | 1.584s +105.75%   | ±0.43% -55.64% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.573mb -0.18%   | 25.074ms +29.54%  | ±1.15% +2.02%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.573mb -0.18%   | 403.969ms -0.26%  | ±0.50% -73.06% |
+-----------------------+-------------------+------+-----+------------------+-------------------+----------------+
Transformers
+-----------------------------+--------------------------+------+-----+------------------+------------------+-----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------------+--------------------------+------+-----+------------------+------------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 86.837mb +26.97% | 64.810ms +30.38% | ±2.46% +138.34% |
+-----------------------------+--------------------------+------+-----+------------------+------------------+-----------------+
Loaders
+--------------------+----------------+------+-----+------------------+-------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode              | rstdev          |
+--------------------+----------------+------+-----+------------------+-------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 94.959mb -0.24%  | 448.058ms -22.37% | ±2.15% +128.41% |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 45.792mb +19.35% | 70.690ms +2.39%   | ±0.79% +51.14%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 90.302mb +16.63% | 55.462ms -14.89%  | ±0.28% -69.64%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 292.144mb +9.81% | 1.435s -7.53%     | ±0.42% -9.13%   |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.352mb +7.60%  | 41.180ms +3.49%   | ±0.75% -44.21%  |
+--------------------+----------------+------+-----+------------------+-------------------+-----------------+
Building Blocks
+-------------------------+----------------------------+------+-----+------------------+--------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode               | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+--------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 60.436mb +26.25% | 2.237ms -17.25%    | ±1.87% -26.31%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 80.163mb +19.87% | 176.647ms +14.33%  | ±2.32% +156.65% |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 58.688mb +27.14% | 17.453ms +10.48%   | ±0.18% -92.55%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 59.575mb +26.63% | 1.625ms -15.27%    | ±1.34% +144.80% |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 59.575mb +26.63% | 1.674ms -12.59%    | ±0.80% -67.92%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 58.788mb +26.99% | 2.579ms -16.43%    | ±2.91% +493.91% |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 59.317mb +26.75% | 14.469ms -5.52%    | ±0.66% +23.79%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 59.316mb +26.75% | 14.233ms -7.59%    | ±1.80% +6.66%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 57.388mb +27.35% | 1.706μs -18.75%    | ±2.72% -30.05%  |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 57.388mb +27.35% | 0.300μs 0.00%      | ±0.00% 0.00%    |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 65.621mb +32.03% | 12.963ms +13.37%   | ±1.52% -21.93%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 91.141mb +27.28% | 66.671ms +33.61%   | ±2.74% +278.73% |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 59.837mb +26.51% | 1.775ms -21.36%    | ±3.32% -5.28%   |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 62.042mb +25.68% | 32.286ms -7.21%    | ±0.38% -23.24%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 61.938mb +25.61% | 4.358ms -16.42%    | ±0.92% -26.79%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 57.388mb +27.60% | 36.744ms -7.16%    | ±0.20% -79.87%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 57.388mb +27.60% | 36.956ms -6.97%    | ±2.23% +93.45%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 57.388mb +27.60% | 37.058ms -7.15%    | ±1.47% +185.34% |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 59.662mb +26.59% | 7.458ms +0.60%     | ±0.97% +24.71%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 57.387mb +27.35% | 28.720ms -4.84%    | ±0.94% +34.26%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 57.388mb +27.35% | 13.221μs 0.00%     | ±0.95% 0.00%    |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 57.388mb +27.35% | 15.870μs -0.78%    | ±1.50% +407.42% |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 80.163mb +19.87% | 182.550ms +14.46%  | ±0.48% -42.70%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 93.385mb +16.79% | 857.107ms +584.80% | ±0.60% -29.91%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 48.289mb +16.33% | 444.125ms +579.48% | ±0.76% -60.72%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 12.312mb +14.00% | 88.758ms +461.97%  | ±0.62% -53.16%  |
+-------------------------+----------------------------+------+-----+------------------+--------------------+-----------------+

@stloyd stloyd changed the title Rework StructureEntry Rework StructureEntry & add MapEntry Nov 12, 2023
@norberttech
Copy link
Member

One more additional thing, MapEntry should be used across the test suite to make sure that it's working well with different transformers/loaders

@stloyd stloyd force-pushed the feature/struct-entry branch from 0eceeef to 2ed0669 Compare November 12, 2023 13:14
@github-actions github-actions bot removed the docs label Nov 12, 2023
@stloyd stloyd force-pushed the feature/struct-entry branch from a553b1a to e49b36a Compare November 14, 2023 15:45
@stloyd stloyd marked this pull request as ready for review November 14, 2023 15:50
@stloyd stloyd mentioned this pull request Nov 14, 2023
@stloyd stloyd merged commit 3c1ec8e into flow-php:1.x Nov 14, 2023
@stloyd stloyd deleted the feature/struct-entry branch November 14, 2023 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MapEntry - new entry type

2 participants