Skip to content

Simplify casting entries to strings #582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
norberttech merged 2 commits into from
Oct 13, 2023
Merged

Simplify casting entries to strings #582

norberttech merged 2 commits into from
Oct 13, 2023

Conversation

@norberttech
Copy link
Member

@norberttech norberttech commented Oct 13, 2023

Change Log

Added

Fixed

Changed

  • Simplified casting entries to strings

Removed

Deprecated

Security

Description

Example:

(new Flow())
    ->read(Parquet::from(__DIR__ . '/orders_spark.parquet'))
    ->withEntry('order_id', ref('order_id')->cast('string'))
    ->withEntry('created_at', ref('created_at')->cast('string'))
    ->withEntry('updated_at', ref('updated_at')->cast('string'))
    ->withEntry('customer', ref('customer')->cast('json'))
    ->withEntry('order_lines', ref('order_lines')->cast('json'))
    ->withEntry('notes', ref('notes')->cast('json'))
    ->withEntry('address', ref('address')->cast('json'))
    ->write(CSV::to(__DIR__ . '/orders_spark.csv'))
    ->run();

Output: (this is all fake data, generated through faker library from python)

order_id,total_price,discount,created_at,updated_at,customer,address,order_lines,notes
a32e1d2b-3435-4d0c-9288-9895fa59300f,160.25,15,2023-10-09T20:58:44+00:00,2023-10-09T20:58:44+00:00,"{""customer_id"":""5efa50c0-d529-4785-a80c-530ffaadba5f"",""first_name"":""Charles"",""last_name"":""Donnelly"",""email"":""fae.grant@gmail.com""}","{""address_id"":""b4ed5cc4-5fce-490e-9737-824a4f8d662e"",""street"":""5273 Len Divide"",""city"":""East Carolshire"",""state"":""Mississippi"",""zip"":""39099-6974"",""country"":""French Southern Territories""}","[{""order_line_id"":""1fb573ba-e370-4113-b400-ea1d1bb3c61d"",""product_id"":""5bc8763b-020a-45a0-867d-f47428a130c9"",""quantity"":5,""price"":41.59},{""order_line_id"":""9a1d2c18-dffa-4f49-b871-1cffaf9d0793"",""product_id"":""3199f17d-e546-42a3-a922-664fe49055b4"",""quantity"":9,""price"":30.81},{""order_line_id"":""a5295a99-2107-4066-88a9-cd505db2a6fb"",""product_id"":""39de1255-6ea5-4390-a1c4-704f748516f2"",""quantity"":2,""price"":18.09}]","[{""note_id"":""d86edab1-e0a9-4351-b278-f2e3ce00c703"",""note_text"":""com.github.javafaker.Lorem@2e4ecdf8""}]"
0b8f57d5-9f1b-4be0-9831-c2fa60359fca,196.45,46.32,2023-10-09T20:58:44+00:00,2023-10-09T20:58:44+00:00,"{""customer_id"":""e3ab8ba7-74be-46d8-8bb8-213233e170ca"",""first_name"":""Zackary"",""last_name"":""Baumbach"",""email"":""melissia.schroeder@hotmail.com""}","{""address_id"":""589c15e7-d788-4b82-9808-934f9b95d5ac"",""street"":""5411 Steve Lake"",""city"":""South Rudyborough"",""state"":""Michigan"",""zip"":""63328-4878"",""country"":""Senegal""}","[{""order_line_id"":""9d9d75d7-01d9-42f4-ace3-c0da1266c2b3"",""product_id"":""8cd5e9d4-63af-48d1-be33-1a4ed2d13274"",""quantity"":6,""price"":35.05},{""order_line_id"":""3284599a-6bcf-44c4-963d-ee6fcd3ee5c4"",""product_id"":""79fd64a9-4be5-44a1-929a-913aa9564c54"",""quantity"":6,""price"":24.06},{""order_line_id"":""1b92a5b3-61cc-49b1-ac5c-43796ab7156a"",""product_id"":""c5606b0f-4816-40ff-a428-fabd045054de"",""quantity"":7,""price"":27.43},{""order_line_id"":""935f3b59-4d35-41ab-a3ce-df859d4a01c3"",""product_id"":""056889d7-9cf5-421a-88a4-fded7c1176bb"",""quantity"":8,""price"":38.92}]","[{""note_id"":""7929be55-e6d6-4040-aee6-7d8e8c80cc6e"",""note_text"":""com.github.javafaker.Lorem@2e4ecdf8""},{""note_id"":""8047b2f8-332e-4086-998f-0fe611215b00"",""note_text"":""com.github.javafaker.Lorem@2e4ecdf8""}]"
a9e1a097-7cc3-4216-bc1a-80244a903cf3,73.22,16.17,2023-10-09T20:58:44+00:00,2023-10-09T20:58:44+00:00,"{""customer_id"":""1fd00691-41da-4932-9af4-e5851d8618f7"",""first_name"":""Dalene"",""last_name"":""Kunde"",""email"":""granville.treutel@gmail.com""}","{""address_id"":""5b598a88-fe5d-495a-9352-ad4c207fcc49"",""street"":""7760 Bosco Knolls"",""city"":""Lake Theaberg"",""state"":""Colorado"",""zip"":""11132"",""country"":""Liechtenstein""}","[{""order_line_id"":""1091f3f2-0a8c-48c3-ba48-73a3111d3746"",""product_id"":""46dfe068-6126-41a3-b517-6c7b94c818c4"",""quantity"":7,""price"":24.86},{""order_line_id"":""e2261d88-02db-4774-bb36-aa0d1b48ca1c"",""product_id"":""36528683-fc92-454d-82ad-606dda418f11"",""quantity"":8,""price"":42.59}]","[{""note_id"":""c6f28ace-463c-422a-81c8-680ce84c046d"",""note_text"":""com.github.javafaker.Lorem@2e4ecdf8""},{""note_id"":""52d766c3-3d35-4931-ab00-4aac883f8e52"",""note_text"":""com.github.javafaker.Lorem@2e4ecdf8""}]"
c33c9a49-6783-4148-85ec-7167858f7b96,173.01,9.06,2023-10-09T20:58:44+00:00,2023-10-09T20:58:44+00:00,"{""customer_id"":""3fb468c1-ce45-42da-a55b-f50015e80bf3"",""first_name"":""Donita"",""last_name"":""Gislason"",""email"":""casey.pacocha@yahoo.com""}","{""address_id"":""55601a9e-37c6-4963-9c8a-1b2afec12bde"",""street"":""3478 Elroy Knolls"",""city"":""Bradlymouth"",""state"":""Maryland"",""zip"":""62147-9826"",""country"":""Vietnam""}","[{""order_line_id"":""7d427b8a-8633-45c7-b908-9c098cb6ac6c"",""product_id"":""44e67d8f-ef86-4042-b0d0-80877fa9523b"",""quantity"":6,""price"":16.56},{""order_line_id"":""b8e00e18-6a2d-4904-92af-5b45384bc44d"",""product_id"":""2b7eb674-d6a4-48ae-a8bb-d82daf0b8621"",""quantity"":7,""price"":8.11},{""order_line_id"":""a669c4e4-effc-46b6-b59a-03d8fd7bd87a"",""product_id"":""2e44b43b-94c9-4eb2-9dad-d88ef7e77a3a"",""quantity"":8,""price"":2.23},{""order_line_id"":""b26e8113-4e3e-4a2e-9167-1f74bcf0bf51"",""product_id"":""d0091188-935b-48f8-9743-44b6f16bd0ea"",""quantity"":6,""price"":37.4}]","[{""note_id"":""083f019a-3c7a-4971-8224-8d42c199abda"",""note_text"":""com.github.javafaker.Lorem@2e4ecdf8""},{""note_id"":""d07a680a-5b0c-4fb0-ae60-696d0ed24458"",""note_text"":""com.github.javafaker.Lorem@2e4ecdf8""}]"

@norberttech norberttech requested a review from stloyd October 13, 2023 16:51
stloyd
stloyd reviewed Oct 13, 2023
View reviewed changes
@norberttech @stloyd
Update src/core/etl/src/Flow/ETL/Row/Reference/Expression/Cast.php
ec10368 
Co-authored-by: Joseph Bielawski <stloyd@users.noreply.github.com>
@norberttech norberttech merged commit 90fab6a into flow-php:1.x Oct 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@stloyd stloyd stloyd approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

size: S

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@norberttech @stloyd