ARROW-5863: [Python] Use atexit module for extension type finalization to avoid segfault #4824
Conversation
|
+1 Travis CI build: https://travis-ci.org/wesm/arrow/builds/555965259 |
|
I'm not sure I understand the underlying cause either. It seems the module-wide |
|
I opened cython/cython#3039 |
|
In order to prevent common cases for crashes or other weird behaviour at shutdown, module-level cleanup is not enabled by default. Does pyarrow use it? The comment in there is not entirely correct for Py3 anymore, where the cleanup is triggered by CPython's module object cleanup. That might not be at system exit but when actually unloading the module for whatever (rare) reason, even before exiting. It's still typically the system exit, though. |
|
Updated the comment here. |
…n to avoid segfault As reported on JIRA, the following script provokes a segfault ``` #! /usr/bin/env python import pyarrow import sys del sys.modules['pyarrow.lib'] ``` For some reason this does not trigger the destruction of the private `_ExtensionTypesInitializer` object. Not sure why (Antoine may know). Using the atexit module instead seems to do the trick Author: Wes McKinney <wesm+git@apache.org> Closes #4824 from wesm/ARROW-5863 and squashes the following commits: ba60578 <Wes McKinney> Use atexit module for extension type finalization
As reported on JIRA, the following script provokes a segfault
For some reason this does not trigger the destruction of the private
_ExtensionTypesInitializerobject. Not sure why (Antoine may know). Using the atexit module instead seems to do the trick