Skip to content

parquet_to_arrow_schema_by_columns Incorrectly Handles Nested Types #1654

@tustvold

Description

@tustvold

Describe the bug

parquet_to_arrow_schema_by_columns calls ArrowSchema::field_with_name with the name of the projected column from the parquet schema. In the case of a nested field, this will either fallback to deducing the type from the parquet schema, or potentially use the schema of a root-level field that happens to have the same name 😱

To Reproduce

Inspect code

Expected behavior

parquet_to_arrow_schema_by_columns should correctly handle arrow schema for nested types

Additional Context

Encountered whilst attempting to fix #1652 which in turn feeds into apache/datafusion#2439

Metadata

Metadata

Assignees

Labels

bugparquetChanges to the parquet crate

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions