-
-
Notifications
You must be signed in to change notification settings - Fork 48
Description
Hello @norberttech, great tool :)
Working with Parquet lib, I am unable to convert dates before 1970-01-01, negative values are not properly converted for Date type.
Steps to reproduce
data_frame()
->read(
from_array(
[
[
'id' => 1,
'birth_date' => '1955-11-25', // date is buggy
'created_at' => '2025-05-12 23:59:12', // datetime is working
]
]
)
)
->write(
to_parquet(__DIR__ . '/test.parquet')
->withSchema(
schema(
int_schema('id'),
date_schema('birth_date'),
datetime_schema('created_at')
)
)
)
->run();I found that Int32DateConverter is using DateInterval which does not directly accept negative value, so probably to use invert propery depending on the given $data value.
The issue is same when reading parquet file, getting the following error.
Next Flow\Parquet\Exception\DataConversionException: Failed to convert data from parquet type for column 'birth_date'. Unknown or bad format (P-5555D)
Thank you in advance