Skip to content

Get precise precision for Decimal literal in FieldToDataType and DecimalField::getPrec #5007

@windtalker

Description

@windtalker

Enhancement

In FieldToDataType

DataTypePtr operator()(const DecimalField<T> & x) const
    {
        PrecType prec = maxDecimalPrecision<T>();
        return std::make_shared<DataTypeDecimal<T>>(prec, x.getScale());
    }

For a Decimal literal, its precision is always the max precision of the Decimal type.(9 for Decimal32, 18 for Decimal64, 38 for Decimal128 and 65 for Decimal256). But, for a decimal literal, we can always get its precise precision, no need to use the upper bound.

In DecimalField::getPrec

UInt32 getPrec() const
    {
        UInt32 cnt = 0;
        auto x = dec.value;
        while (x != 0)
        {
            x /= 10;
            cnt++;
        }
        if (cnt == 0)
            cnt = 1;
        return cnt;
    }

As we can see, if the dec.value == 0, the prec is 1, if the scale is 2, then the (prec, scale) will be (1,2) which is not valid, so we should return std::max(cnt, scale) as its prec.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/enhancementThe issue or PR belongs to an enhancement.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions