Description

Spark-4.0 has added new data type VariantType. The support for VariantType and variant shredded data has been added to parquet reader and writer.

Refer:

• https://issues.apache.org/jira/browse/SPARK-50614
• [SPARK-48495][SQL][DOCS] Describe shredding scheme for Variant
• [SPARK-50614][SQL] Add Variant shredding support for Parquet
• [SPARK-54454][SQL] Enable variant shredding and variant logical type annotation configs by default

This feature has been enabled by default in Spark. Currently, since Gluten does not support VariantType, need to add fallback.

Gluten version

None