fix: removal of pyarrow.lib.PyExtensionType in pyarrow 21#3581
fix: removal of pyarrow.lib.PyExtensionType in pyarrow 21#3581
pyarrow.lib.PyExtensionType in pyarrow 21#3581Conversation
|
@ianna @henryiii Is this the right fix here? if isinstance(storage_type, pyarrow.lib.PyExtensionType):
raise ValueError(
"Arrow arrays containing pickled Python objects can't be converted into Awkward Arrays"
)
elif isinstance(storage_type, pyarrow.lib.ExtensionType):
...
)My idea was to do the first isinstance check only if pyarrow < 21 |
ianna
left a comment
There was a problem hiding this comment.
@ikrommyd - thanks for looking into it. I think a better way would be to wrap it in a try block.
try:
# If PyExtensionType exists and matches, raise immediately
pyext_type = pyarrow.lib.PyExtensionType # May trigger AttributeError
if isinstance(storage_type, pyext_type):
raise ValueError(
"Arrow arrays containing pickled Python objects can't be converted into Awkward Arrays"
)
except AttributeError:
# PyExtensionType doesn't exist (likely newer PyArrow)
pass
# Handle remaining ExtensionType cases
if isinstance(storage_type, pyarrow.lib.ExtensionType):
assert not isinstance(storage_type, AwkwardArrowType)
# Logic for generic ExtensionType fallback
return
Ah yeah that's possible since an |
Done. |
Fixes #3579