[Python] Extend project to accept a list of types + add DuckDBPyType class#6777
[Python] Extend project to accept a list of types + add DuckDBPyType class#6777Mytherin merged 39 commits intoduckdb:masterfrom
project to accept a list of types + add DuckDBPyType class#6777Conversation
…t of DuckDBPyType
…e against a string - this makes this change backwards compatible, by supporting comparisons with for example, lists of stringified types
|
The implementation of the EnumType creation method is left as an exercise for the reader ;) |
…es for some of the creation methods, and made changes to the compare + constructor methods
Mytherin
left a comment
There was a problem hiding this comment.
Thanks for the PR! Looks great. I think adding support for types is a great idea - some comments below:
…int], dict['a': str, 'c2': bool]
|
Could you have a look at fixing the merge conflicts? |
|
The failure seems unrelated? |
|
Pandas provides this method as Do we maybe also want to add the |
|
I think providing both options is sensible. Perhaps we should just rename it to be fully compatible? Or have both |
|
I think I'll add |
|
Thanks! LGTM |
* Add resource requirements.
* Fix NOP memory race with placement new atomics.
* Fix missing include
* Pull out L1/L2 construction to tasks (still single threaded) * Convert IEJoinUnion to work on slices of L2 * Remove dead code. * Cherry pick block iterator locking fix. * Add resource requirements to test. * Fix NOP memory race with placement new atomics. * Fix missing include
Remove pointer indirection in ExtensionAccess (duckdb/duckdb#19529) fix: link error on linux with multiple definition of LogicalType::VARCHAR in `shell_renderer.cpp` (duckdb/duckdb#20096) Internal duckdb/duckdb#6777: IEJoin Unified L1/2 (duckdb/duckdb#20083)
Remove pointer indirection in ExtensionAccess (duckdb/duckdb#19529) fix: link error on linux with multiple definition of LogicalType::VARCHAR in `shell_renderer.cpp` (duckdb/duckdb#20096) Internal duckdb/duckdb#6777: IEJoin Unified L1/2 (duckdb/duckdb#20083) Co-authored-by: krlmlr <krlmlr@users.noreply.github.com>
This PR implements the feature request in #6706
projecton list of column typesThis allows you to provide a list of types to project on instead of column names, selecting all of the columns of the relation that match any of the provided types.
The implementation of this is likely temporarily, as discussed in the linked request.
Aiming to replace this when we have support for this functionality in core.
DuckDBPyType
I initially thought it was overkill to add a python class to represent our LogicalType, but we have thought of another use for this, so I have opted to add it anyways.
Primitives (defined on
duckdb.typing)Creation methods
sqltype(type_str: str)alias:
type,dtypecreate a type from parsing the
type_str, this can also be used for user or extension defined typesstring_type(collation: str = "")create VARCHAR type with optional collation
decimal_type(width: int, scale: int)create a DECIMAL type of the given width + scale
enum_type(name: str, type: DuckDBPyType, values: list)create en ENUM type from the 'values' list, cast astypeas underlying valuesarray_type(type: DuckDBPyType)alias:
list_typecreate a LIST type of the 'type' as child type
struct_type(fields: List[DuckDBPyType] | Dict[str, DuckDBPyType])alias:
row_typecreate a STRUCT type of the given field types (uses default names if given as List)
map_type(key: DuckDBPyType, value: DuckDBPyType)create a MAP type out of the 'key' and 'value' types
union_type(members: List[DuckDBPyType] | Dict[str, DuckDBPyType])create a UNION type of the given member types (similar to STRUCT)
Misc
Can be compared against strings
Converts implicitly from:
str,builtinstypes (str, bool, float etc..),list,dict,{'name', type, ..}dictionary (to STRUCT),numpybuiltin typesint64,bool_,float32etc.