Skip to content

Shared libraries not found due to rpath changes in v0.56.0 #4451

@scheremisin

Description

@scheremisin

Problem:

After switching to rules_go v0.56.0, we started experiencing the following error:

[2025-09-09T21:15:50.534Z] (21:15:50) [ERROR: [0m/home/ubuntu/.cache/bazel/_bazel_ubuntu/fdc7e464f0a2b4a76bb07016cd725e2e/external/src/infra/service/joom-infra-image-service/BUILD.bazel:75:12: Export of joom-infra-image-service-indices to ../src/infra/service/joom-infra-image-service/joom-infra-image-service-mongo-indices.yaml with mongo-indices kind failed: (Exit 127): joom-infra-image-service failed: error executing ExportMongoIndices command (from target @@src//infra/service/joom-infra-image-service:joom-infra-image-service-indices) 
[2025-09-09T21:15:50.534Z]   (cd /home/ubuntu/.cache/bazel/_bazel_ubuntu/fdc7e464f0a2b4a76bb07016cd725e2e/execroot/_main && \
[2025-09-09T21:15:50.534Z]   exec env - \
[2025-09-09T21:15:50.534Z]   bazel-out/k8-opt-exec-ST-cd39ae64b419/bin/external/src/infra/service/joom-infra-image-service/joom-infra-image-service --env localhost --param 'log.level=fatal' '--printAppInfo=false' --exportManifestKind mongo-indices --exportManifestFile bazel-out/k8-fastbuild-ST-27f7ed591fdb/bin/external/src/infra/service/joom-infra-image-service/joom-infra-image-service-mongo-indices.yaml --param 'countries.disableLoad=true' --param 'log.bufferStderr=false' --param 'payments.paypal.disabled=true' --param 'payments.paypal_usa.disabled=true' --param 'payments.qiwi.disabled=true' --param 'payments.yandex_h2h.disabled=true' '--forceStubRuntimeConfig=true')
[2025-09-09T21:15:50.535Z] # Configuration: 9db84b7f99f055d11de7f063ce43916f499608655caf4f5dc53df99eac1f5242
[2025-09-09T21:15:50.535Z] # Execution platform: //bazel/platforms:force_llvm
[2025-09-09T21:15:50.535Z] bazel-out/k8-opt-exec-ST-cd39ae64b419/bin/external/src/infra/service/joom-infra-image-service/joom-infra-image-service: error while loading shared libraries: libjemalloc.so.2: cannot open shared object file: No such file or directory

Some of our targets stopped resolving shared libraries.

We noticed the issue is related to changes introduced in PR #4390, which modified how rpaths are calculated.

Debug info and observation:

We patched both rules_go v0.51.1 and v0.56.0 to log the actual rpaths being calculated.
For v0.51.1:

rpaths =[
"$ORIGIN/../../../../../_solib_k8/_U_A_Abzlmod~~sysroot_Uldd~sysroot_Uldd_Uamd64_S_Slibjemalloc_Clibjemalloc___Uusr_Slib_Sx86_U64-linux-gnu",
"$ORIGIN/_joom-infra-image-service.runfiles/_main/_solib_k8/_U_A_Abzlmod~~sysroot_Uldd~sysroot_Uldd_Uamd64_S_Slibjemalloc_Clibjemalloc___Uusr_Slib_Sx86_U64-linux-gnu"
]

For v0.56.0:

rpaths = [
"$ORIGIN/../../../../_main/_solib_k8/_U_A_Abzlmod~~sysroot_Uldd~sysroot_Uldd_Uamd64_S_Slibjemalloc_Clibjemalloc___Uusr_Slib_Sx86_U64-linux-gnu", 
"$ORIGIN/_joom-infra-image-service.runfiles/_main/_solib_k8/_U_A_Abzlmod~~sysroot_Uldd~sysroot_Uldd_Uamd64_S_Slibjemalloc_Clibjemalloc___Uusr_Slib_Sx86_U64-linux-gnu"
]

The difference is here:

  • v0.51.1 → $ORIGIN/../../../../../_solib_k8
  • v0.56.0 → $ORIGIN/../../../../_main/_solib_k8

For the problematic target, $ORIGIN is:
/home/ubuntu/.cache/bazel/_bazel_ubuntu/fdc7e464f0a2b4a76bb07016cd725e2e/execroot/_main/bazel-out/k8-opt-exec-ST-cd39ae64b419/bin/external/src/infra/service/joom-infra-image-service

The actual directory with shared libraries is:
bazel-out/k8-opt-exec-ST-cd39ae64b419/bin/_solib_k8/_U_A_Abzlmod~~sysroot_Uldd~sysroot_Uldd_Uamd64_S_Slibjemalloc_Clibjemalloc___Uusr_Slib_Sx86_U64-linux-gnu

  • With v0.51.1, $ORIGIN/../../../../../_solib_k8 resolves correctly to .../bazel-out/k8-opt-exec-ST-cd39ae64b419/bin/_solib_k8

  • With v0.56.0, $ORIGIN/../../../../_main/_solib_k8 resolves to .../bazel-out/k8-opt-exec-ST-cd39ae64b419/bin/external/_main/_solib_k8, which does not contain the shared libraries.

Shared libraries can be put:

  1. Inside .../*.runfiles/_main/_solib_k8/... (when using bazel run/test)
    • This works fine: both versions generate a valid rpath to this directory, we can see it in the rpaths above.
  2. Inside .../bazel-out/k8-opt-exec-ST-cd39ae64b419/bin/_solib_k8/...
    • This is where the problem occurs. We have a custom rule that executes actions via ctx.actions.run, and those binaries expect to load libraries from this directory. In v0.56.0, the rpath is broken.

Conclusion:

It looks like PR #4390 may have missed supporting the second case above.

Could you please confirm whether this is an issue in rules_go, or if we might have a configuration problem on our side?

We’re happy to provide more details if needed.

Additional info:

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions