How do you determine if a .so needs to be loaded globally?

Also global is a very generic name. Maybe global_dll or global_so would be more specific.

Yeah global_dll or global_so both sound better.

How do you determine if a .so needs to be loaded globally?

Well it happens when b.so requires symbols from a.so but isn't linked against a.so. We can find when this happens by trial and error. It's also worth noting that making a library global is synchronous and can be done lazily (after the library is loaded, before any of the missing symbols are used). So in principal the incorrectly linked b.so could declare that it needs a.so to be global in it's meta.yaml file instead of a.so having to know that b.so exists and declare itself as global.

Ideally we would also fix everything to link properly and not need anything to be global. This would be particularly helpful for the command line runner.

Thanks for the work!

Ideally we would also fix everything to link properly and not need anything to be global. This would be particularly helpful for the command line runner.

Yes that would be ideal. It looks like the most problematic library is CLAPACK for now.

Yeah I'm a bit confused about this PR, since I don't reproduce a lot of the CI failures locally. If you feel inclined @ryanking13 you could take it over.

I think this is more or less a prerequisite for #3098 to work well.

Yeah I'm a bit confused about this PR, since I don't reproduce a lot of the CI failures locally. If you feel inclined @ryanking13 you could take it over.

I'll see if I can fix this. I also tried to get rid of global shared lib loading a few months ago but failed back then.
It is interesting that sqlite3 needs to be specified as a global_dso: true. We use a static version of libsqlite3 so something is clearly wrong here...

I have looked into this for a few days, and I am not sure loading a dependent shared library locally (and exposing its symbols to a specific module) is possible without a large modification in emscripten codebase.

Emscripten resolves symbol from:

1. Global symbols
2. Symbols exported from module itself

which means if a module (.so) needs to look for an external symbol, it has to be loaded globally.

if a module (.so) needs to look for an external symbol, it has to be loaded globally.

I don't believe this is correct. See for instance the comment just above that:

We need this fallback because: ... Symbols from side modules are not always added to the global namespace.

I think it is possible for one module to use a symbol from another even if the other module isn't in global scope. But the exact behavior is pretty confusing.