Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Isolate the faulthandler Module Between Interpreters #101509

Open
Tracked by #103092
ericsnowcurrently opened this issue Feb 1, 2023 · 1 comment
Open
Tracked by #103092

Isolate the faulthandler Module Between Interpreters #101509

ericsnowcurrently opened this issue Feb 1, 2023 · 1 comment
Labels
3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters type-feature A feature request or enhancement

Comments

@ericsnowcurrently
Copy link
Member

ericsnowcurrently commented Feb 1, 2023 •
edited

(See gh-100227.)

Currently the faulthandler module has some state in _PyRuntimeState, including objects, which is shared by all interpreters. Interpreters should be isolated from each other, for a variety of reasons (including the possibility of a per-interpreter GIL). Isolating the module will involve moving some of the state to PyInterpreterState (and, under per-interpreter GIL, guarding other state with a global lock).

Code Analysis:

(expand)

Signals

The module installs handlers for various signals.

(expand)
fatal:
  • SIGBUS - bus error
  • SIGILL - illegal instruction
  • SIGFPE - floating point exception
  • SIGABRT - abort
  • SIGSEGV - segfault

"user":

  • the 5 fatal signals
  • others (up to Py_NSIG)

handlers:

  • faulthandler_fatal_error(signum)
  • faulthandler_exc_handler() (on Windows)
  • faulthandler_user()

State

Fields

https://github.com/python/cpython/blob/main/Include/internal/pycore_faulthandler.h#L49-L86
https://github.com/python/cpython/blob/main/Include/internal/pycore_runtime.h#L146

raw

cpython/Include/internal/pycore_faulthandler.h

Lines 49 to 86 in 62251c3

struct _faulthandler_runtime_state {
struct {
int enabled;
PyObject *file;
int fd;
int all_threads;
PyInterpreterState *interp;
#ifdef MS_WINDOWS
void *exc_handler;
#endif
} fatal_error;
struct {
PyObject *file;
int fd;
PY_TIMEOUT_T timeout_us; /* timeout in microseconds */
int repeat;
PyInterpreterState *interp;
int exit;
char *header;
size_t header_len;
/* The main thread always holds this lock. It is only released when
faulthandler_thread() is interrupted before this thread exits, or at
Python exit. */
PyThread_type_lock cancel_event;
/* released by child thread when joined */
PyThread_type_lock running;
} thread;
#ifdef FAULTHANDLER_USER
struct faulthandler_user_signal *user_signals;
#endif
#ifdef FAULTHANDLER_USE_ALT_STACK
stack_t stack;
stack_t old_stack;
#endif
};

cpython/Include/internal/pycore_faulthandler.h

Lines 36 to 46 in 62251c3

#ifdef FAULTHANDLER_USER
struct faulthandler_user_signal {
int enabled;
PyObject *file;
int fd;
int all_threads;
int chain;
_Py_sighandler_t previous;
PyInterpreterState *interp;
};
#endif /* FAULTHANDLER_USER */

tables:

name

type

#ifdef

notes
fatal_error
    . enabled bool "is faulthandler enabled?"
    . file PyObject * only for keeping .fd alive
    . fd int
    . all_threads bool
    . interp PyInterpreterState * show only its threads
the one that enabled faulthandler
    . exc_handler void * MS_WINDOWS

name

type

#ifdef

notes
thread ---
    . file PyObject * only for keeping .fd alive
    . fd int defaults to sys.stderr
    . timeout_us PY_TIMEOUT_T
    . repeat bool
    . interp PyInterpreterState * show only its threads
the one that requested the info
    . exit bool
    . header char *
    . header_len size_t
    . cancel_event PyThread_type_lock
    . running PyThread_type_lock

name

type

#ifdef

notes
user_signals struct faulthandler_user_signal * FAULTHANDLER_USER a dynamically allocated array (Py_NSIG)
user_signals[signum] ---
    . enabled bool
    . file PyObject *
    . fd int
    . all_threads bool
    . chain bool
    . previous _Py_sighandler_t
    . interp PyInterpreterState * show only its threads
the one that added the handler

name

type

#ifdef

notes
old_stack stack_t FAULTHANDLER_USE_ALT_STACK
stack stack_t FAULTHANDLER_USE_ALT_STACK

Usage

simple:

name

context

get

set
fatal_error
    . enabled module faulthandler_is_enabled() faulthandler_disable_py()
signal faulthandler_fatal_error()
internal faulthandler_enable()
faulthandler_disable()		 faulthandler_enable()
faulthandler_disable()
    . file module faulthandler_traverse() faulthandler_py_enable()
faulthandler_traverse()
internal faulthandler_disable() faulthandler_disable()
    . fd module
signal faulthandler_fatal_error()
faulthandler_exc_handler()
    . all_threads module faulthandler_py_enable()
signal faulthandler_fatal_error()
faulthandler_exc_handler()
    . interp module faulthandler_py_enable()
internal faulthandler_fatal_error()
faulthandler_exc_handler()
    . exc_handler internal faulthandler_enable()
faulthandler_disable()		 faulthandler_enable()
faulthandler_disable()

name

context

get

set
thread
    . file module faulthandler_dump_traceback_later()
faulthandler_traverse()		 faulthandler_dump_traceback_later()
faulthandler_traverse()
internal cancel_dump_traceback_later() cancel_dump_traceback_later()
    . fd
module faulthandler_dump_traceback_later()
thread faulthandler_thread()
    . timeout_us module faulthandler_dump_traceback_later()
thread faulthandler_thread()
    . repeat module faulthandler_dump_traceback_later()
thread faulthandler_thread()
    . interp module faulthandler_dump_traceback_later()
thread faulthandler_thread()
    . exit module faulthandler_dump_traceback_later()
thread faulthandler_thread()
    . header module faulthandler_dump_traceback_later()
thread faulthandler_thread()
internal cancel_dump_traceback_later() cancel_dump_traceback_later()
    . header_len module faulthandler_dump_traceback_later()
thread faulthandler_thread()
    . cancel_event module faulthandler_dump_traceback_later() faulthandler_dump_traceback_later()
thread faulthandler_thread()
internal cancel_dump_traceback_later()
    . running module faulthandler_dump_traceback_later() faulthandler_dump_traceback_later()
thread faulthandler_thread()
internal cancel_dump_traceback_later()

name

context

get

set
user_signals module faulthandler_register_py()
faulthandler_unregister_py()		 faulthandler_register_py()
faulthandler_traverse()
C-API _PyFaulthandler_Fini() _PyFaulthandler_Fini()
signal faulthandler_user()
user_signals[signum]
    . enabled module faulthandler_register_py() faulthandler_register_py()
signal faulthandler_user()
internal faulthandler_unregister() faulthandler_unregister()
    . file module faulthandler_register_py()
faulthandler_traverse()		 faulthandler_register_py()
faulthandler_traverse()
internal faulthandler_unregister() faulthandler_unregister()
    . fd module faulthandler_register_py()
signal faulthandler_user()
internal faulthandler_unregister()
    . all_threads module faulthandler_register_py()
signal faulthandler_user()
    . chain module faulthandler_register_py()
signal faulthandler_user()
    . previous module faulthandler_register_py()
signal faulthandler_user()
internal faulthandler_unregister()
    . interp module faulthandler_register_py()
signal faulthandler_user()

name

context

get

set
stack C-API _PyFaulthandler_Init()
_PyFaulthandler_Fini()
internal faulthandler_enable()
faulthandler_register()
faulthandler_allocate_stack()
old_stack C-API _PyFaulthandler_Fini()
modify data in complex fields:

name

context

calls
fatal_error .fd signal faulthandler_fatal_error() -> faulthandler_dump_traceback()
faulthandler_fatal_error() -> _Py_DumpExtensionModules()
faulthandler_exc_handler()
faulthandler_exc_handler() -> _Py_DumpHexadecimal()
faulthandler_exc_handler() -> faulthandler_dump_traceback()
thread.fd thread faulthandler_thread() -> _Py_write_noraise()
faulthandler_thread() -> _Py_DumpTracebackThreads()
thread .cancel_event module faulthandler_dump_traceback_later()
C-API _PyFaulthandler_Fini()
thread faulthandler_thread()
internal cancel_dump_traceback_later()
thread .running module faulthandler_dump_traceback_later()
thread faulthandler_thread()
internal cancel_dump_traceback_later()
user_signals[signum] .fd signal faulthandler_user() -> faulthandler_dump_traceback()
stack C-API _PyFaulthandler_Init()
_PyFaulthandler_Fini()
internal faulthandler_allocate_stack()
old_stack internal faulthandler_allocate_stack() -> sigaltstack()
@ericsnowcurrently ericsnowcurrently added type-feature A feature request or enhancement interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters 3.12 bugs and security fixes labels Feb 1, 2023
@ericsnowcurrently ericsnowcurrently mentioned this issue Feb 1, 2023
Closed
@ericsnowcurrently
Copy link
Member Author

Relevant: https://discuss.python.org/t/pep-684-a-per-interpreter-gil/19583/34

@ericsnowcurrently ericsnowcurrently mentioned this issue Mar 28, 2023
Open
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters type-feature A feature request or enhancement
Projects
Subinterpreters
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ericsnowcurrently