Skip to content

Replace deprecated imp with importlib#7676

Merged
bwoodsend merged 9 commits intopyinstaller:developfrom
bwoodsend:imp-importlib
Jun 4, 2023
Merged

Replace deprecated imp with importlib#7676
bwoodsend merged 9 commits intopyinstaller:developfrom
bwoodsend:imp-importlib

Conversation

@bwoodsend
Copy link
Copy Markdown
Member

This wasn't as scary as I expected. The most complicated usages of imp without drop-in importlib equivalents were mostly inside unused parts of the codebase.

bwoodsend added 7 commits June 3, 2023 12:25
Whilst the former function does provide additional information, and
whilst the code base does juggle that information around, only the
suffixes themselves ever get used.
This eliminates usages of imp.find_module().
Doing so removes its usages of imp. For future reference, this is done by
running:

    swig -python sample.i

This does highlight though that, since swig's new generated files use
plain imports rather than imp/importlib, swig works fine without all our
special handling so dropping support for old style swig templates may be
the simplest way forward.
hook-pkg_without_hook_for_pkg.sub1.py contains checks that
pkg_without_hook_for_pkg.sub1 has a hook and pkg_without_hook_for_pkg
doesn't. Since PyInstaller lacks a nice standalone
find_hook(module_name) function, it uses imp.find_module() (which is not
how PyInstaller normally finds hooks) with the module search path being
the expected location of the hook plus PyInstaller's own hooks
subpackage.

importlib.util.find_spec() (the closet equivalent to imp.find_module()) is
unable to find any module with a `-` in it so an equivalent check is
impossible.

I consider a dumb `git ls-files | grep hook-pkg_without_hook_for_pkg.py`
to be all I need to convince me that `hook-pkg_without_hook_for_pkg.py`
doesn't exist so lets solve this problem by removing it.
@bwoodsend bwoodsend marked this pull request as ready for review June 3, 2023 15:43
@bwoodsend
Copy link
Copy Markdown
Member Author

This doesn't fix #7524 or modulegraph being recursive so we will still need to replace modulegraph eventually but at least this gets rid of the Python 3.12 deadline.

@rokm
Copy link
Copy Markdown
Member

rokm commented Jun 3, 2023

Okay, let's take this route, then.

I tried to build a hello world program with 3.12b1 on my linux box, and I can get it running with this and #7670 and a couple of small additions that I'll organize into a proper PR. There's also an index-out-of-range error coming from byte-code scanning (which I had to disable), but I think that's standard stuff.

So extending the life of the modulegraph for 3.12 does seem feasible.

rokm added 2 commits June 4, 2023 01:31
pkgutil.ImpImporter is not available in python 3.12 anymore; but
we don't really use it anywhere in that code, and instead end
up using importlib.machinery.FileFinder if python >= 3.3, which
covers all supported python versions.
When retrieving loader from importer/finder, prefer the PEP 451
`find_spec` to retrieve spec and loader from it, instead of using
PEP 302 `find_loader`. The latter has been obsolete for a while,
and became unavailabe on built-in importers/finders with python
3.12.
@rokm
Copy link
Copy Markdown
Member

rokm commented Jun 3, 2023

I've cherry-picked the two commits that are related to modulegraph and use of obsolete import mechanisms, as they thematically fit into this PR.

@bwoodsend bwoodsend merged commit fd0a25a into pyinstaller:develop Jun 4, 2023
@bwoodsend bwoodsend deleted the imp-importlib branch June 4, 2023 10:39
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace deprecated imp module with importlib Don't use imp for Python >= 3.8 in moduelgraph

2 participants