Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segfault when printing MemoryError raised through PyErr_NoMemory() from subinterpreter #109894

Closed
chgnrdv opened this issue Sep 26, 2023 · 3 comments · Fixed by #110911
Closed

Segfault when printing MemoryError raised through PyErr_NoMemory() from subinterpreter #109894

chgnrdv opened this issue Sep 26, 2023 · 3 comments · Fixed by #110911
Labels
3.12 bugs and security fixes 3.13 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@chgnrdv
Copy link
Contributor

chgnrdv commented Sep 26, 2023 •
edited by bedevere-app bot

Crash report

What happened?

Repro:

import _testcapi
_testcapi.run_in_subinterp("[0]*100000000000")

Backtrace:

BaseException_str (self=<optimized out>) at Objects/exceptions.c:120
120	    switch (PyTuple_GET_SIZE(self->args)) {
(gdb) bt
#0  BaseException_str (self=<optimized out>) at Objects/exceptions.c:120
#1  0x00005555556ed00f in PyObject_Str (v=v@entry=0x7ffff75988f0) at Objects/object.c:606
#2  0x0000555555872d45 in print_exception_message (ctx=ctx@entry=0x7fffffffd870, type=type@entry=0x555555b54f20 <_PyExc_MemoryError>, value=0x7ffff75988f0)
    at Python/pythonrun.c:1079
#3  0x0000555555873c59 in print_exception (ctx=ctx@entry=0x7fffffffd870, value=value@entry=0x7ffff75988f0) at Python/pythonrun.c:1226
#4  0x000055555587443a in print_exception_recursive (ctx=ctx@entry=0x7fffffffd870, value=value@entry=0x7ffff75988f0) at Python/pythonrun.c:1507
#5  0x0000555555874af2 in _PyErr_Display (file=file@entry=0x7ffff743fe30, unused=unused@entry=0x0, value=value@entry=0x7ffff75988f0, tb=tb@entry=0x7ffff74f39d0)
    at Python/pythonrun.c:1557
#6  0x0000555555874c91 in PyErr_Display (unused=unused@entry=0x0, value=0x7ffff75988f0, tb=0x7ffff74f39d0) at Python/pythonrun.c:1585
#7  0x000055555588456f in sys_excepthook_impl (module=module@entry=0x7ffff7631370, exctype=<optimized out>, value=<optimized out>, traceback=<optimized out>)
    at ./Python/sysmodule.c:783
#8  0x00005555558845bb in sys_excepthook (module=0x7ffff7631370, args=0x7fffffffd9c0, nargs=<optimized out>) at ./Python/clinic/sysmodule.c.h:101
#9  0x00005555556e691b in cfunction_vectorcall_FASTCALL (func=0x7ffff7631850, args=0x7fffffffd9c0, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/methodobject.c:425
#10 0x00005555556794b8 in _PyObject_VectorcallTstate (tstate=0x7ffff7598938, callable=0x7ffff7631850, args=0x7fffffffd9c0, nargsf=3, kwnames=0x0)
    at ./Include/internal/pycore_call.h:187
#11 0x00005555556795d3 in PyObject_Vectorcall (callable=callable@entry=0x7ffff7631850, args=args@entry=0x7fffffffd9c0, nargsf=nargsf@entry=3, kwnames=kwnames@entry=0x0)
    at Objects/call.c:327
#12 0x0000555555874e20 in _PyErr_PrintEx (tstate=0x7ffff7598938, set_sys_last_vars=set_sys_last_vars@entry=1) at Python/pythonrun.c:840
#13 0x0000555555875199 in PyErr_PrintEx (set_sys_last_vars=set_sys_last_vars@entry=1) at Python/pythonrun.c:878
#14 0x00005555558751a9 in PyErr_Print () at Python/pythonrun.c:884
#15 0x0000555555875b77 in PyRun_SimpleStringFlags (command=<optimized out>, flags=flags@entry=0x7fffffffda70) at Python/pythonrun.c:517
#16 0x00007ffff76ad1a4 in run_in_subinterp (self=<optimized out>, args=args@entry=0x7ffff779a210) at ./Modules/_testcapimodule.c:1405
#17 0x00005555556e65c8 in cfunction_call (func=func@entry=0x7ffff7810f50, args=args@entry=0x7ffff779a210, kwargs=kwargs@entry=0x0) at Objects/methodobject.c:551
#18 0x000055555567916b in _PyObject_MakeTpCall (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, callable=callable@entry=0x7ffff7810f50, 
    args=args@entry=0x7ffff7fc1078, nargs=<optimized out>, keywords=keywords@entry=0x0) at Objects/call.c:242
#19 0x00005555556795ab in _PyObject_VectorcallTstate (tstate=0x555555c02338 <_PyRuntime+508728>, callable=0x7ffff7810f50, args=0x7ffff7fc1078, nargsf=<optimized out>, 
    kwnames=0x0) at ./Include/internal/pycore_call.h:185
#20 0x00005555556795d3 in PyObject_Vectorcall (callable=callable@entry=0x7ffff7810f50, args=args@entry=0x7ffff7fc1078, nargsf=<optimized out>, 
    kwnames=kwnames@entry=0x0) at Objects/call.c:327
#21 0x00005555557e3ad6 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, frame=0x7ffff7fc1020, throwflag=throwflag@entry=0)
    at Python/generated_cases.c.h:3761
#22 0x00005555557ed5e4 in _PyEval_EvalFrame (throwflag=0, frame=<optimized out>, tstate=0x555555c02338 <_PyRuntime+508728>) at ./Include/internal/pycore_ceval.h:107
#23 _PyEval_Vector (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, func=func@entry=0x7ffff777b050, locals=locals@entry=0x7ffff778e690, args=args@entry=0x0, 
    argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:1630
#24 0x00005555557ed68b in PyEval_EvalCode (co=co@entry=0x7ffff7767520, globals=globals@entry=0x7ffff778e690, locals=locals@entry=0x7ffff778e690) at Python/ceval.c:582
#25 0x00005555558718cf in run_eval_code_obj (tstate=tstate@entry=0x555555c02338 <_PyRuntime+508728>, co=co@entry=0x7ffff7767520, globals=globals@entry=0x7ffff778e690, 
--Type <RET> for more, q to quit, c to continue without paging--
    locals=locals@entry=0x7ffff778e690) at Python/pythonrun.c:1720
#26 0x00005555558722b7 in run_mod (mod=mod@entry=0x555555c55608, filename=filename@entry=0x7ffff77b5940, globals=globals@entry=0x7ffff778e690, 
    locals=locals@entry=0x7ffff778e690, flags=flags@entry=0x7fffffffdf98, arena=arena@entry=0x7ffff77de320) at Python/pythonrun.c:1741
#27 0x00005555558723cf in pyrun_file (fp=fp@entry=0x555555c2f3b0, filename=filename@entry=0x7ffff77b5940, start=start@entry=257, globals=globals@entry=0x7ffff778e690, 
    locals=locals@entry=0x7ffff778e690, closeit=closeit@entry=1, flags=0x7fffffffdf98) at Python/pythonrun.c:1641
#28 0x00005555558756bb in _PyRun_SimpleFileObject (fp=fp@entry=0x555555c2f3b0, filename=filename@entry=0x7ffff77b5940, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffdf98) at Python/pythonrun.c:464
#29 0x0000555555875938 in _PyRun_AnyFileObject (fp=fp@entry=0x555555c2f3b0, filename=filename@entry=0x7ffff77b5940, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffdf98) at Python/pythonrun.c:79
#30 0x000055555589dfda in pymain_run_file_obj (program_name=program_name@entry=0x7ffff79790e0, filename=filename@entry=0x7ffff77b5940, skip_source_first_line=0)
    at Modules/main.c:361
#31 0x000055555589e2b7 in pymain_run_file (config=config@entry=0x555555b9cf20 <_PyRuntime+93984>) at Modules/main.c:380
#32 0x000055555589f3fc in pymain_run_python (exitcode=exitcode@entry=0x7fffffffe10c) at Modules/main.c:611
#33 0x000055555589f454 in Py_RunMain () at Modules/main.c:689
#34 0x000055555589f4a8 in pymain_main (args=args@entry=0x7fffffffe150) at Modules/main.c:719
#35 0x000055555589f51d in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:743
#36 0x00005555555cf74e in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15

I can see two simultaneous reasons for this behaviour:

  • unlike for main interpreter that preallocates memory errors, for subinterpreter interp.exc_state.memerrors_freelist remains equal to NULL, which means that PyErr_NoMemory() always returns interp.static_objects.last_resort_memory_error.
  • interp.static_objects.last_resort_memory_error has args field equal to NULL, thus attempt to print it segfaults.

Possible fix:

  • assign interp.static_objects.last_resort_memory_error.args to empty tuple:
diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h
index 2deba02a89..e22ffcd1fb 100644
--- a/Include/internal/pycore_runtime_init.h
+++ b/Include/internal/pycore_runtime_init.h
@@ -177,6 +177,7 @@ extern PyTypeObject _PyExc_MemoryError;
                 }, \
                 .last_resort_memory_error = { \
                     _PyObject_HEAD_INIT(&_PyExc_MemoryError) \
+                    .args = (PyObject*)&_Py_SINGLETON(tuple_empty), \
                 }, \
             }, \
         }, \
  • initialize memory errors on subinterpreter creation in a similar manner as in the main interpreter, through _PyExc_InitGlobalObjects() or whatever

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (heads/main:e81bd3fa16, Sep 26 2023, 12:14:51) [GCC 10.2.1 20210110]

Linked PRs
@chgnrdv chgnrdv added the type-crash A hard crash of the interpreter, possibly with a core dump label Sep 26, 2023
@AA-Turner
Copy link
Member

cc @ericsnowcurrently

@AA-Turner
Copy link
Member

@chgnrdv are you able to reproduce on any earlier versions of Python, or is this just 3.13?

A

@AA-Turner AA-Turner added interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.13 new features, bugs and security fixes labels Sep 26, 2023
@chgnrdv
Copy link
Contributor Author

chgnrdv commented Sep 26, 2023 •
edited

@AA-Turner , I've bisected this crash to feec49c, so 3.12 is affected too.

@AA-Turner AA-Turner added the 3.12 bugs and security fixes label Sep 26, 2023
chgnrdv added a commit to chgnrdv/cpython that referenced this issue Oct 15, 2023
@chgnrdv
pythongh-109894: Fix initialization of static MemoryError in subint…
0d497ed 
…erpreter

Fixes python#109894

* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init
@chgnrdv chgnrdv mentioned this issue Oct 15, 2023
Merged
ericsnowcurrently pushed a commit that referenced this issue Oct 23, 2023
@chgnrdv @blurb-it
gh-109894: Fix initialization of static MemoryError in subinterpret…
47d3e2e 
…er (gh-110911)

Fixes #109894

* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
@miss-islington miss-islington mentioned this issue Oct 23, 2023
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 23, 2023
@chgnrdv @blurb-it @miss-islington
pythongh-109894: Fix initialization of static MemoryError in subint…
8258a62 
…erpreter (pythongh-110911)

Fixes pythonGH-109894

* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init

---------

(cherry picked from commit 47d3e2e)

Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
ericsnowcurrently pushed a commit that referenced this issue Nov 28, 2023
@miss-islington @chgnrdv @blurb-it
[3.12] gh-109894: Fix initialization of static MemoryError in subin…
0aceac5 
…terpreter (gh-110911) (gh-111238)

Fixes GH-109894

* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init

---------

(cherry picked from commit 47d3e2e)

Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@chgnrdv @blurb-it @aisk
pythongh-109894: Fix initialization of static MemoryError in subint…
08c747b 
…erpreter (pythongh-110911)

Fixes python#109894

* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.12 bugs and security fixes 3.13 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters type-crash A hard crash of the interpreter, possibly with a core dump
Projects
Subinterpreters
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

gh-109894: Fix initialization of static MemoryError in subinterpreter chgnrdv/cpython
3 participants
@AA-Turner @chgnrdv @AlexWaygood