Problem statement

Normally, native linux Python wheels include its dependent external shared libraries. It is done by auditwheel which copies shared libraries into the wheel and repairs the RPATH of elf binaries.

On the other hand, we are serving external shared libraries separately from the Python wheel. This is a bit annoying since it makes wheels not portable themselves. One other partially related problem is that if the wheel contains shared libraries inside its wheel, it is not working well because we have to load external shared libraries globally (unless we manage to make #3054 work) but the libraries inside the wheel are not loaded globally.

Proposal

I think it would be nice if we can embed shared libraries into the wheels so wheels can be distributed and be used without downloading additional shared libraries.

For this, I have been recently working on a tool called auditwheel-emscripten which does jobs similar to auditwheel but for Emscripten / Pyodide wheels: It checks the external libraries that Emscripten WASM module depends and can copy them into the wheel file.

Implementation issues

One issue is that there is no such thing as RPATH in Emscripten WASM modules. They only contain the name of the shared library. So we will need to manually handle how the WASM module should find its dependent libraries. But since we are doing some heuristic path resolution already, I believe it is not a big deal.

One other issue is how we should distinguish Python c-extension modules and shared libraries. We need to load shared libraries globally (for now), but we want to load Python extension modules locally. So we need some ways to distinguish them.
A naive idea that I am thinking of is using the prefix and suffix of the filename: Checking if the filename startswith lib or endswith an extension suffix (cpython-310-wasm32-emscripten). But it would be nice if there is a more proper way.
Note that loading a python c-extension module globally will not cause an error (unless there is a symbol collision), but loading a shared library locally will raise an error.