-
Notifications
You must be signed in to change notification settings - Fork 4k
ARROW-12074: [C++][Compute] Add scalar arithmetic kernels for decimal #10364
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
bkietz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this!
Please extract the decimal upscaling from the addition kernel into an implicit cast. This will simplify the addition kernel to stateless addition (IIUC) and give callers control over how to handle upscaling. See CommonNumeric() for existing promotion behavior; this should give you an idea of how to specify the upscaling as an implicit cast.
Thanks @bkietz , casting args (rescaling decimal inputs) implicitly before exec looks much better than my current implementation. Will try the approach. |
|
@bkietz , met with one problem, would like to hear your comments. Thanks. Decimal upscaling is operation dependent. E.g., Implicit args casting happens before kernel is created. Maybe add another callback |
|
if (auto type = CommonNumeric(*values)) {
ReplaceTypes(type, values);
} else if (auto type = CommonDecimal(name_, *values)) {
ReplaceTypes(type, values);
} |
|
Thanks @bkietz , it's almost done except one last catch. As the output type (precision, scale) is dependent on the inputs, I have a resolver object to calculate output type. The resolver is called with the casted input type, not the original type. It causes problem to division, as the output precision and scale should be calculated from original inputs. No trouble for add/subtract as the output precision/scale is the same for original and casted inputs (digit aligned). Multiply doesn't need cast. Does it make sense to pass both original input types and the casted types to the resolver [1][2]? We will have to update all existing custom resolver codes. [1] https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/function.cc#L196 EDIT: Pushed latest code to ease reviewing. Unit test fails due to this issue. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please open a follow up JIRA to make this public so it can be reused (for example in the comparison functions)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this could be the default case; it's what would be used for any comparison kernel, for example
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additionally, I think this could be defined for the varargs case (so it could be used in elementwise_min/elementwise_max/...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not done yet. Will follow up.
|
Rebased |
bkietz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is looking great, thanks again! Just a few more items
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please ensure we're not adding decimal kernels for functions which don't support decimals yet:
| AddDecimalBinaryKernels<Op>(name, &func); | |
| // TODO($FOLLOW_UP_JIRA) remove when decimal is supported for all arithmetic kernels | |
| if (name == "add" || name == "add_checked" || ...) { | |
| AddDecimalBinaryKernels<Op>(name, &func); | |
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please open a follow up JIRA for supporting decimals in the other arithmetic functions as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I moved AddDecimalBinaryKernels calls out of MakeArithmeticFunction and MakeArithmeticFunctionNotNull, and put it under each kernel supporting decimal operations.
https://github.com/apache/arrow/pull/10364/files#diff-3eafd7246f6a8c699f10d46e3276852fe44b6853b5517ef10396e561730c09f4R840
I'm afraid some arithmetic kernels (e.g., power) may not support decimals?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, that will work
I'm afraid some arithmetic kernels (e.g., power) may not support decimals?
This is one topic which the follow up JIRA would give us a better place to discuss
bkietz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will merge on green CI.
Thanks for doing this!
Add basic binary arithmetic (+,-,*,/) kernels for decimal types. Closes apache#10364 from cyb70289/decimal-arith Authored-by: Yibo Cai <yibo.cai@arm.com> Signed-off-by: Benjamin Kietzman <bengilgit@gmail.com>
Add basic binary arithmetic (+,-,*,/) kernels for decimal types.