Skip to content

NoSuchMethodError: PartitionedFileUtil$.splitFiles when running with Spark 3.5.4 #1572

@YanivKunda

Description

@YanivKunda

Describe the bug

The signature of PartitionedFileUtil$.splitFiles changed in Spark 3.5.5, and Comet was updated to use it in #1565, so it is now not compatible with 3.5.4.

This could be fixed with using different shims for the patch-level version (affecting release engineering) or by using reflection within the 3.5 shim.

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions