Cannot infer top-level imports from packages installed by PDM w/install.cache=true (aka. symlinked packages) #452
Description
PDM has an alternative to install packages (enabled with pdm config install.cache true) where the package is first installed into a local cache (typically under $HOME/.cache/pdm/packages/), and then the package is installed into the target virtualenv, but with symlinks pointing back to the package in the cache. Here is an example:
$ pdm config install.cache true
$ pdm init
[...]
$ pdm add colorama
[...]
$ ls -al .venv/lib/python3.10/site-packages/colorama*
lrwxrwxrwx 1 jherland users 79 Apr 19 03:03 .venv/lib/python3.10/site-packages/colorama -> /home/jherland/.cache/pdm/packages/colorama-0.4.6-py2.py3-none-any/lib/colorama
.venv/lib/python3.10/site-packages/colorama-0.4.6.dist-info:
total 44
drwxr-xr-x 3 jherland users 4096 Apr 19 03:03 .
drwxr-xr-x 4 jherland users 4096 Apr 19 03:03 ..
drwxr-xr-x 2 jherland users 4096 Apr 19 03:03 licenses
-rw-r--r-- 1 jherland users 17158 Apr 19 03:03 METADATA
-rw-r--r-- 1 jherland users 412 Apr 19 03:03 RECORD
-rw-r--r-- 1 jherland users 66 Apr 19 03:03 REFER_TO
-rw-r--r-- 1 jherland users 105 Apr 19 03:03 WHEEL
$ cat .venv/lib/python3.10/site-packages/colorama-0.4.6.dist-info/RECORD
colorama-0.4.6.dist-info/METADATA,sha256=e67SnrUMOym9sz_4TjF3vxvAV4T3aF7NyqRHHH3YEMw,17158
colorama-0.4.6.dist-info/WHEEL,sha256=cdcF4Fbd0FPtw2EMIOwH-3rSOTUdTCeOSXRMD1iLUb8,105
colorama-0.4.6.dist-info/licenses/LICENSE.txt,sha256=ysNcAmhuXQSlpxQL-zs25zrtSWZW6JEQLkKIhteTAxg,1491
colorama-0.4.6.dist-info/REFER_TO,sha256=b2lWg3Osknh6HODITiPmZW55TlJ8okVt-8k6rf3kdpc,66
colorama-0.4.6.dist-info/RECORD,,
colorama,,
$ cat .venv/lib/python3.10/site-packages/colorama-0.4.6.dist-info/REFER_TO
/home/jherland/.cache/pdm/packages/colorama-0.4.6-py2.py3-none-any
Specifically:
- There is no
top-level.txtfile, so top-level imports must be inferred fromRECORD. - The
RECORDfile does not mention any Python files undercolorama/, only thecoloramaentry itself is listed. This is reflected inimportlib_metadata.files("colorama"), and as a result_top_level_inferred(colorama)ends up finding nothing, andpackages_distributions()incorrectly presents zero imports for this packages. - Inside the
site-packagesdirectory,coloramais a symlink to the correspondingcoloramadirectory inside the PDM cache, so Python is indeed able toimport coloramaand find modules within this package.
This was first reported via tweag/FawltyDeps#307, and (as a FawltyDeps maintainer and importlib_metadata user) I'm trying to figure out if this is an issue with importlib_metadata not properly handling PDM's weird (but techinically valid) installation method, or if PDM is doing something strictly incorrect here.