Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-42923: _Py_DumpExtensionModules() ignores stdlib ext #24254

Merged
merged 1 commit into from Jan 19, 2021
Merged

bpo-42923: _Py_DumpExtensionModules() ignores stdlib ext #24254

merged 1 commit into from Jan 19, 2021

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Jan 18, 2021 •
edited by bedevere-bot

@vstinner
Copy link
Member Author

vstinner commented Jan 18, 2021 •
edited by bedevere-bot

This PR should be rebased on master once PR #24238 (bpo-42955: Add sys.modules_names) will be merged.

@vstinner
Copy link
Member Author

vstinner commented Jan 18, 2021

Example with Cinder.

./python -m venv env
./env/bin/python -m pip install wheel
./env/bin/python -m pip install cinder

Without the PR, 43 extensions (stdlib + third party) are listed:

$ ./env/bin/python -X dev -c 'import cinder, ctypes, sys; print(f"{len(sys.modules)=}"); ctypes.string_at(0)'
(...)
len(sys.modules)=220
Fatal Python error: Segmentation fault

Current thread 0x00007ff626de1740 (most recent call first):
  File "/home/vstinner/python/master/Lib/ctypes/__init__.py", line 517 in string_at
  File "<string>", line 1 in <module>

Extension modules: sys, builtins, _imp, _warnings, _weakref, _io, marshal, posix, faulthandler, _codecs, _signal, _abc, _stat, itertools, _operator, _collections, _functools, _sre, _locale, _heapq, _opcode, _struct, greenlet._greenlet, _thread, __original_module__thread, errno, math, __original_module_select, time, __original_module_time, _socket, select, array, _datetime, binascii, _hashlib, _blake2, _cffi_backend, _bisect, _sha512, _string, _ssl, _ctypes
Erreur de segmentation (core dumped)

With the PR, 5 extensions (third party only) are listed:

$ ./env/bin/python -X dev -c 'import cinder, ctypes, sys; print(f"{len(sys.modules)=}"); ctypes.string_at(0)'
(...)
len(sys.modules)=220
Fatal Python error: Segmentation fault

Current thread 0x00007f7421252740 (most recent call first):
  File "/home/vstinner/python/master/Lib/ctypes/__init__.py", line 517 in string_at
  File "<string>", line 1 in <module>

Extension modules: greenlet._greenlet, __original_module__thread, __original_module_select, __original_module_time, _cffi_backend (total: 5)
Erreur de segmentation (core dumped)

cc @pablogsal

@vstinner vstinner mentioned this pull request Jan 18, 2021
Merged
@vstinner vstinner force-pushed the dump_3rd_party_ext branch 3 times, most recently from 2b46a07 to 23eb36e Compare Jan 19, 2021
@vstinner vstinner changed the title [WIP] bpo-42923: _Py_DumpExtensionModules() ignores stdlib ext bpo-42923: _Py_DumpExtensionModules() ignores stdlib ext Jan 19, 2021
@vstinner vstinner marked this pull request as ready for review Jan 19, 2021
@vstinner vstinner merged commit 66f77ca into python:master Jan 19, 2021
3 checks passed
@vstinner vstinner deleted the dump_3rd_party_ext branch Jan 19, 2021
@bedevere-bot
Copy link

bedevere-bot commented Jan 19, 2021

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL8 3.x has failed when building commit 66f77ca.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/559/builds/599) and take a look at the build logs.
  4. Check if the failure is related to this commit (66f77ca) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/559/builds/599

Summary of the results of the build (if available):

== Tests result: ENV CHANGED ==

412 tests OK.

10 slowest tests:

  • test_concurrent_futures: 4 min 4 sec
  • test_tokenize: 3 min 17 sec
  • test_unparse: 2 min 51 sec
  • test_peg_generator: 2 min 40 sec
  • test_gdb: 2 min 38 sec
  • test_multiprocessing_spawn: 2 min 23 sec
  • test_lib2to3: 2 min 14 sec
  • test_capi: 1 min 49 sec
  • test_asyncio: 1 min 39 sec
  • test_multiprocessing_forkserver: 1 min 31 sec

1 test altered the execution environment:
test_asyncio

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_ossaudiodev
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

Total duration: 7 min 28 sec

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/sslproto.py", line 321, in __del__
    self.close()
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/sslproto.py", line 316, in close
    self._ssl_protocol._start_shutdown()
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/sslproto.py", line 590, in _start_shutdown
    self._abort()
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/sslproto.py", line 731, in _abort
    self._transport.abort()
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/selector_events.py", line 680, in abort
    self._force_close(None)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/selector_events.py", line 731, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/base_events.py", line 745, in call_soon
    self._check_closed()
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/asyncio/base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
@vstinner @adorilson
bpo-42923: _Py_DumpExtensionModules() ignores stdlib ext (pythonGH-24254
39a8efe 
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@vstinner @bedevere-bot @the-knights-who-say-ni