Description of the issue
I am trying to build an application on MacOS that uses pkguilt.iter_modules to dynamically find and load modules however it is not finding any modules.
The modules are there and can be imported with importlib.import_module.
Note that I am using PyInstaller with FBS but I don't believe the latter changes anything that would effect this.
The following is an example project structure
my_package
my_module_1
my_module_2
my_module_3
Here is the code that does not work
import importlib
import pkgutil
m = importlib.import_module("my_package")
print("Found nested modules", list(pkgutil.iter_modules(m.__path__, m.__name__ + ".")))
The last line prints an empty list when built with PyInstaller.
I have done a bit of a dive into the code and I believe this is a rather complex mixup with paths and symbolic links.
https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py
It looks like #6539 had a go at supporting symbolic links but it only resolves symbolic links up to the _MEIPASS directory.
On MacOS _MEIPASS is equal to "/path/to/my_package.app/Contents/MacOS/"
my_package.__path__ is equal to ["/path/to/my_package.app/Contents/MacOS/my_package"]
/path/to/my_package.app/Contents/MacOS/my_package is a symbolic link to /path/to/my_package.app/Contents/Resources/my_package when there are data files present in that package.
The iter_modules hook expands symbolic links in the path variable which converts "/path/to/my_package.app/Contents/MacOS/my_package" to "/path/to/my_package.app/Contents/Resources/my_package"
The following code then skips over the directory because it does not start with SYS_PREFIX
if not pkg_path.startswith(SYS_PREFIX):
# If the path does not start with sys._MEIPASS, it cannot be a bundled package.
continue
I don't know how this should be implemented but we need to resolve the expected path relative to the _MEIPASS directory and then expand symbolic links and only then skip if the package path does not start with the directory.
Context information (for bug reports)
- Output of
pyinstaller --version: 5.13.0
- Version of Python: Python 3.11.4 (main, Aug 22 2023, 11:47:24) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
- Platform: OS X
- How you installed Python: pyenv
- Did you also try this on another platform? Does it work there? This only effects MacOS as far as I am aware
Description of the issue
I am trying to build an application on MacOS that uses pkguilt.iter_modules to dynamically find and load modules however it is not finding any modules.
The modules are there and can be imported with importlib.import_module.
Note that I am using PyInstaller with FBS but I don't believe the latter changes anything that would effect this.
The following is an example project structure
Here is the code that does not work
The last line prints an empty list when built with PyInstaller.
I have done a bit of a dive into the code and I believe this is a rather complex mixup with paths and symbolic links.
https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py
It looks like #6539 had a go at supporting symbolic links but it only resolves symbolic links up to the
_MEIPASSdirectory.On MacOS
_MEIPASSis equal to"/path/to/my_package.app/Contents/MacOS/"my_package.__path__is equal to["/path/to/my_package.app/Contents/MacOS/my_package"]/path/to/my_package.app/Contents/MacOS/my_packageis a symbolic link to/path/to/my_package.app/Contents/Resources/my_packagewhen there are data files present in that package.The
iter_moduleshook expands symbolic links in thepathvariable which converts"/path/to/my_package.app/Contents/MacOS/my_package"to"/path/to/my_package.app/Contents/Resources/my_package"The following code then skips over the directory because it does not start with
SYS_PREFIXI don't know how this should be implemented but we need to resolve the expected path relative to the
_MEIPASSdirectory and then expand symbolic links and only then skip if the package path does not start with the directory.Context information (for bug reports)
pyinstaller --version:5.13.0