Datafusion downcasts decimal loosing precision #13492
Description
Describe the bug
This cast throws an error
select cast(12345.6789 as decimal(6,2));
Arrow error: Invalid argument error: 1234568 is too large to store in a Decimal128 of precision 6. Max is 999999But if we cast it to a larger decimal and then cast it down to (6,2), then DataFusion cast works fine and looses precision. As shown below
> select cast( cast(12345.6789 as decimal(24,2)) as decimal(6,2));
+---------------------+
| Float64(12345.6789) |
+---------------------+
| 1234.56 |
+---------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.To Reproduce
try the above select query in datafusion-cli
Expected behavior
It should return null or throw an error as in this example
select cast(12345.6789 as decimal(6,2));
Arrow error: Invalid argument error: 1234568 is too large to store in a Decimal128 of precision 6. Max is 999999Additional context
while trying to enable decimal support in comet, Spark returns null in this cast but DataFusion returns a lowered precision value loosing the value of the decimal.