Skip to content

Disallow creation of invalid PartitionSpec #12870

@devinrsmith

Description

@devinrsmith

Bug

The specification says in relationship to partitioning:

The source columns, selected by ids, must be a primitive type and cannot be contained in a map or list, but may be nested in a struct.

The java implementation provides the necessary safety checks around the "primitive type" part of this statement, but does not provide safety checks for the "contained in a map or list" part.

final Schema schema =
    new Schema(
        NestedField.required(
            2, "MyList", Types.ListType.ofRequired(1, Types.IntegerType.get())));
// Does not currently fail, but ideally should
PartitionSpec.builderFor(schema).identity("MyList.element").build();

I've got a branch with changes to TestPartitionSpecValidation.java that further demonstrates the current behavior.

Query engine

None

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementPR that improves existing functionality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions