-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Closed
Labels
Description
Describe the bug
When validating a value for decimal, if error occurs the error message is formatted with the invalid value & highest/lowest allowed value; however this error message displays the values without accounting for scale, leading to a confusing error message.
To Reproduce
Example test (e.g. in arrow-cast/src/cast/mod.rs):
#[test]
fn test_123() {
let array = Int64Array::from(vec![1]);
dbg!(cast_with_options(
&array,
&DataType::Decimal32(1, 1),
&CastOptions {
safe: false,
format_options: FormatOptions::default(),
},
));
}Shows error message:
InvalidArgumentError("10 is too large to store in a Decimal32 of precision 1. Max is 9")
Expected behavior
Error message should be more like:
1.0 is too large to store in a Decimal32 of precision 1. Max is 0.9
- Perhaps add scale to the message too?
Additional context
Original DataFusion issue: apache/datafusion#3666
Relevant code snippet in arrow-rs:
arrow-rs/arrow-data/src/decimal.rs
Lines 1024 to 1043 in 1f77ac5
| pub fn validate_decimal32_precision(value: i32, precision: u8) -> Result<(), ArrowError> { | |
| if precision > DECIMAL32_MAX_PRECISION { | |
| return Err(ArrowError::InvalidArgumentError(format!( | |
| "Max precision of a Decimal32 is {DECIMAL32_MAX_PRECISION}, but got {precision}", | |
| ))); | |
| } | |
| if value > MAX_DECIMAL32_FOR_EACH_PRECISION[precision as usize] { | |
| Err(ArrowError::InvalidArgumentError(format!( | |
| "{value} is too large to store in a Decimal32 of precision {precision}. Max is {}", | |
| MAX_DECIMAL32_FOR_EACH_PRECISION[precision as usize] | |
| ))) | |
| } else if value < MIN_DECIMAL32_FOR_EACH_PRECISION[precision as usize] { | |
| Err(ArrowError::InvalidArgumentError(format!( | |
| "{value} is too small to store in a Decimal32 of precision {precision}. Min is {}", | |
| MIN_DECIMAL32_FOR_EACH_PRECISION[precision as usize] | |
| ))) | |
| } else { | |
| Ok(()) | |
| } | |
| } |
- And same for all other decimal types