Skip to content

[Python][C++] Parquet backwards compat for timestamps without timezone broken #22303

@asfimport

Description

@asfimport

When reading a parquet file which has timestamp fields they are read as a timestamp with timezone UTC if the parquet file was written by pyarrow 0.13.0 and/or 0.12.1.

Expected behavior would be that they are loaded as timestamps without any timezone information.

The attached files contain one row for all basic types and a few nested types, the timestamp fields are called datetime64 and datetime64_tz

see also https://github.com/JDASoftwareGroup/kartothek/tree/master/reference-data/arrow-compat

https://github.com/JDASoftwareGroup/kartothek/blob/c47e52116e2dc726a74d7d6b97922a0252722ed0/tests/serialization/test_arrow_compat.py#L31

 

Reporter: Florian Jetter / @fjetter
Assignee: TP Boudreau / @tpboudreau

Original Issue Attachments:

PRs and other links:

Note: This issue was originally created as ARROW-5889. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions