-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and why for this feature, in addition to the what)
Related to PR #3309 which wants to add SQL support for tinyint and unsigned ints. The question is whether we want to support SQL access to underlying data types that Arrow supports.
Here's a table of integer data type support between Spark, Postgres, DataFusion, Arrow, and Parquet:

"SQL support" in this case means only 3 things, as far as I can tell:
- support for the data type in the
CREATE EXTERNAL TABLE x (schema..)syntax - support for the data type in the SQL
CASTfunction - no memory table support via
CREATE TABLE x AS SELECT 1::tinyint unsignedsyntax
Otherwise, the unsigned variants are already supported by DataFusion, because Arrow supports them. Unsigned integers can be in custom table providers, or in a parquet file.
Without SQL support for these data types, we won't be able to cast to unsigned types from SQL, nor load them from CSV/JSON.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.