Skip to content

Decimal precision validation displays value without accounting for scale #8382

@Jefffrey

Description

@Jefffrey

Describe the bug

When validating a value for decimal, if error occurs the error message is formatted with the invalid value & highest/lowest allowed value; however this error message displays the values without accounting for scale, leading to a confusing error message.

To Reproduce

Example test (e.g. in arrow-cast/src/cast/mod.rs):

    #[test]
    fn test_123() {
        let array = Int64Array::from(vec![1]);
        dbg!(cast_with_options(
            &array,
            &DataType::Decimal32(1, 1),
            &CastOptions {
                safe: false,
                format_options: FormatOptions::default(),
            },
        ));
    }

Shows error message:

InvalidArgumentError("10 is too large to store in a Decimal32 of precision 1. Max is 9")

Expected behavior

Error message should be more like:

1.0 is too large to store in a Decimal32 of precision 1. Max is 0.9

  • Perhaps add scale to the message too?

Additional context

Original DataFusion issue: apache/datafusion#3666

Relevant code snippet in arrow-rs:

pub fn validate_decimal32_precision(value: i32, precision: u8) -> Result<(), ArrowError> {
if precision > DECIMAL32_MAX_PRECISION {
return Err(ArrowError::InvalidArgumentError(format!(
"Max precision of a Decimal32 is {DECIMAL32_MAX_PRECISION}, but got {precision}",
)));
}
if value > MAX_DECIMAL32_FOR_EACH_PRECISION[precision as usize] {
Err(ArrowError::InvalidArgumentError(format!(
"{value} is too large to store in a Decimal32 of precision {precision}. Max is {}",
MAX_DECIMAL32_FOR_EACH_PRECISION[precision as usize]
)))
} else if value < MIN_DECIMAL32_FOR_EACH_PRECISION[precision as usize] {
Err(ArrowError::InvalidArgumentError(format!(
"{value} is too small to store in a Decimal32 of precision {precision}. Min is {}",
MIN_DECIMAL32_FOR_EACH_PRECISION[precision as usize]
)))
} else {
Ok(())
}
}

  • And same for all other decimal types

Metadata

Metadata

Assignees

Labels

arrowChanges to the arrow cratebug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions