Is there a way to implement this that doesn't require another build flag? Does NixOS require all programs to add in special support for manual shared library loading?

The compiler/stdlib could always check the contents of NIX_LDFLAGS, or not at all and we patch the compiler when building for Nix. I think the build flag is should be a non-issue because for Nix we can just set it permanently in nim.cfg during installation and no one needs to know it exists. Normally stuff just works on NixOS, but if something is using dlopen then it usually needs patched, and we like to upstream patches whenever possible.

Why is NixOS being a special case here? Why does it need this different mechanism, it's a Linux distro after all.

Why is NixOS being a special case here? Why does it need this different mechanism, it's a Linux distro after all.

Because it doesn't follow FHS and each package/lib resides in it's own folder in /nix/store/

I think for Nix you can just use a custom config.nims that import NIX_LDFLAGS, then collect and convert -L into -Wl,-rpath, then add the result to passL. There should be no need for editing the stdlib.

Alternatively, you can use --dynlibOverrideAll then --passL:"$NIX_LDFLAGS" manually (which I presume would be more preferable).

The problem is described in #15194.

Yes, manually passing the correct flags makes the correct thing happen, each package built from Nim sources in Nixpkgs passes special flags to fix this problem, and its the least preferable solution.

@ehmry you can edit global nim.cfg to add these flags for the compiler and all programs (config/nim.cfg)

Maybe we could create a hook in Nixpkgs that would consider NIX_LDFLAGS and create such nim.cfg file. However, I am worried that (nor config.nims) may not be sufficient because

Command line settings have priority over configuration file settings.

For more background, search also for NIX_LDFLAGS in https://nixos.org/nixpkgs/manual/#ssec-setup-hooks.

However, I am worried that (nor config.nims) may not be sufficient because

Command line settings have priority over configuration file settings.

Can you elaborate?

However, I am worried that (nor config.nims) may not be sufficient because

Command line settings have priority over configuration file settings.

Can you elaborate?

Can you point us to the documentation for config.nims.

Can you point us to the documentation for config.nims.

Here you go: https://nim-lang.org/docs/nims.html

config.nims is sourced from any standard location, and for a distribution storing them in /etc/nim/config.nims or $nim/config/config.nims is a good way to ensure that they're always applied.

For packaging in a distribution I'd recommend --dynlibOverrideAll whenever possible, as it would allow you to reuse standard ELF tooling for dependency tracking (and also more consistent loading decisions). The auto -L to -Wl,-rpath will potentially be useful in a nix-env (users might not care too much about figuring all the flags for use with --dynlibOverrideAll), though I've not read up too much about how that environment is setup.

So we could modify config/config.nims or config/nim.cfg to set -L. Checking the default config/nim.cfg I think in the future we may need to clean some things there anyway, for when using nix-build, e.g. use cc %= "$CC" instead of the current default cc = gcc. However, I think also worth considering is what if you want to make a build with Nim for a non-Nix system. Not of interest to me, but worth considering.

It would be much better to modify a .cfg file since these can be interpreted without launching a NimScript interpreter with all of its coincident overhead.

I think in the future we may need to clean some things there anyway, for when using nix-build, e.g. use cc %= "$CC" instead of the current default cc = gcc

cc is actually an object so it should probably be cc.exe. The annoying part is that we need to add these flags regardless of which compiler is used. Consider

However, I think also worth considering is what if you want to make a build with Nim for a non-Nix system. Not of interest to me, but worth considering.

it may be worth creating a cc = nix but that seems ugly.

Edit:

There seems to be an envcc compiler that may be of interest as well instead.

There seems to be an envcc compiler that may be of interest as well instead.

Note that this new cc will only be available for 1.4.0+ (unreleased), so you will need an another solution for older nim compilers (you can probably just backport the commit, but I haven't checked if that's possible).

Personally I would not recommend envcc but instead you should use gcc or clang as those may include special codegen optimized for the target compiler (currently there are none, but there might be some in the future).

For adding flags into builds you can use passC and passL, both of which are applied for all compilers and won't be overridden by the command line (they stack up instead).

For adding flags into builds you can use passC and passL, both of which are applied for all compilers and won't be overridden by the command line (they stack up instead).

Good, that was what I was worried about initially!

In what format are the flags supposed to be passed in? Comma or space separated? I would expect the latter, however

$ tail -n1 config/nim.cfg
--passL="$NIX_LDFLAGS"

One should not use %= here, right?

Hint:  [Link]
gcc: error: /nix/store/xhdxlvh2i2pqm8vnw9f1lwlbk90hdikf-nim-1.2.6/lib64: No such file or directory
gcc: error: /nix/store/xhdxlvh2i2pqm8vnw9f1lwlbk90hdikf-nim-1.2.6/lib: No such file or directory
gcc: error: unrecognized command line option '-rpath'
gcc: error: unrecognized command line option '-rpath'
Error: execution of an external program failed: 'gcc   -o /build/nim-1.2.6/koch  /build/.cache/nim/koch_d/stdlib_assertions.nim.c.o /build/.cache/nim/koch_d/stdlib_dollars.nim.c.o /build/.cache/nim/koch_d/stdlib_io.nim.c.o /build/.cache/nim/koch_d/stdlib_system.nim.c.o /build/.cache/nim/koch_d/stdlib_parseutils.nim.c.o /build/.cache/nim/koch_d/stdlib_math.nim.c.o /build/.cache/nim/koch_d/stdlib_unicode.nim.c.o /build/.cache/nim/koch_d/stdlib_strutils.nim.c.o /build/.cache/nim/koch_d/stdlib_pathnorm.nim.c.o /build/.cache/nim/koch_d/stdlib_posix.nim.c.o /build/.cache/nim/koch_d/stdlib_times.nim.c.o /build/.cache/nim/koch_d/stdlib_os.nim.c.o /build/.cache/nim/koch_d/stdlib_parseopt.nim.c.o /build/.cache/nim/koch_d/stdlib_hashes.nim.c.o /build/.cache/nim/koch_d/stdlib_strtabs.nim.c.o /build/.cache/nim/koch_d/stdlib_streams.nim.c.o /build/.cache/nim/koch_d/stdlib_cpuinfo.nim.c.o /build/.cache/nim/koch_d/stdlib_osproc.nim.c.o /build/.cache/nim/koch_d/stdlib_sets.nim.c.o /build/.cache/nim/koch_d/@mtools@skochdocs.nim.c.o /build/.cache/nim/koch_d/stdlib_base64.nim.c.o /build/.cache/nim/koch_d/stdlib_uri.nim.c.o /build/.cache/nim/koch_d/stdlib_strformat.nim.c.o /build/.cache/nim/koch_d/@mtools@sdeps.nim.c.o /build/.cache/nim/koch_d/@mkoch.nim.c.o  -lm -lrt  $NIX_LDFLAGS  -ldl'

$ echo $NIX_LDFLAGS
-rpath /nix/store/hn1y9dinsrcxfbgjiymv3fhzd10k9m12-nim-1.2.6/lib64 -rpath /nix/store/hn1y9dinsrcxfbgjiymv3fhzd10k9m12-nim-1.2.6/lib -L/nix/store/a0jxdf0wc13l4270fgq9n1f42x45pq69-openssl-1.1.1g/lib -L/nix/store/5gdc4wn4nx74dx5mv2k3gqx8hh4q72mg-pcre-8.44/lib -L/nix/store/s2kyldk2s42n5z2ijjj1v5ns78n9wzr3-ncurses-6.2/lib -L/nix/store/v7b326xn4wq9n4wqjdxar3xadz73mi8f-readline-6.3p08/lib -L/nix/store/19a50vv9bv66rsw91anv8r6avnzrbwxy-boehm-gc-8.0.4/lib -L/nix/store/iwmz9cl5x7xyzdfm3phxn7y4krz27m5j-sfml-2.5.1/lib -L/nix/store/f7lws879jarjs40qznw95pwz4ngc7ib0-sqlite-3.32.3/lib -L/nix/store/a0jxdf0wc13l4270fgq9n1f42x45pq69-openssl-1.1.1g/lib -L/nix/store/5gdc4wn4nx74dx5mv2k3gqx8hh4q72mg-pcre-8.44/lib -L/nix/store/s2kyldk2s42n5z2ijjj1v5ns78n9wzr3-ncurses-6.2/lib -L/nix/store/v7b326xn4wq9n4wqjdxar3xadz73mi8f-readline-6.3p08/lib -L/nix/store/19a50vv9bv66rsw91anv8r6avnzrbwxy-boehm-gc-8.0.4/lib -L/nix/store/iwmz9cl5x7xyzdfm3phxn7y4krz27m5j-sfml-2.5.1/lib -L/nix/store/f7lws879jarjs40qznw95pwz4ngc7ib0-sqlite-3.32.3/lib

In what format are the flags supposed to be passed in?

The same format as what the gcc driver accepts (since we pass the flags directly to gcc during link): -Wl,<flag-here>

Long term we want to get rid of rpath for Nix and I don't see any reason to look up something at runtime when we can know it with certainty at compile-time. I'd rather just patch the compiler downstream than deal with this configuration stuff.

Long term we want to get rid of rpath for Nix and I don't see any reason to look up something at runtime when we can know it with certainty at compile-time.

But you are still doing lookups at runtime? All this change does is to generate library candidates for the compiler with the paths prefixed. Unless I'm reading this wrong, this means you are basically doing dlopen() job for searching within a set of defined path, similar to rpath, while being more inflexible.

Also note that dynlib module is Nim's dlopen and is also used by programs outside of stdlib. With this change of yours every Nim program compiled with -d:nixbuild will look at NIX_LDFLAGS whenever loadLibPattern is used, and that's not something you should blanketly apply. If you would like to localize this in the compiler, then do so by providing a libCandidate within the compiler source (it can just call dynlib.libCandidate then manually append the paths to the generated seq, not super efficient but not super slow either).

I'm still not convinced that this solution is better than rpath.

Ah, shame on me, I am reading this wrong. I didn't notice the existence check.

This does seem to be something you'd want to do in the compiler though. The distros module should have the necessary tooling to detect NixOS so you don't have to use a special define flag.

@ehmry FYI there is https://nim-lang.github.io/Nim/packaging.html - can you please keep it up to date with any relevant information? Thanks

This is something that should be patched downstream.