-
Notifications
You must be signed in to change notification settings - Fork 11.1k
Description
Hi!
I help package grpc / grpcio for conda-forge (mostly centered around the python ecosystem), which has much more powerful infrastructure than what the default pip can do, mostly w.r.t. dealing with non-python dependencies, and tracking which projects are exposed to which projects' ABIs, and rebuilding the respective artefact once new versions of those dependencies get released.
This works very well in practice, but requires some curation, and metadata to distinguish which packages are a host dependence (need to be available at compile time, generally shared libs & their headers), and which need to be available at runtime (e.g. most python stuff, plus the shared libs so the linkage can succeed at runtime).
gRPC is obviously a pretty involved package (due to exposure to various other tricky packages like abseil & protobuf), and so we've done our best to just get & keep things running so far.
But in the process of trying to clean up some metadata for zlib (and to a degree, abseil), we ran into breakage builds consuming abseil, due to the way the link interface of grpc is defined.
For example, the linkage of gprc is defined as:
Lines 2345 to 2368 in d52ed19
| target_link_libraries(grpc | |
| ${_gRPC_BASELIB_LIBRARIES} | |
| ${_gRPC_ZLIB_LIBRARIES} | |
| ${_gRPC_CARES_LIBRARIES} | |
| ${_gRPC_ADDRESS_SORTING_LIBRARIES} | |
| ${_gRPC_RE2_LIBRARIES} | |
| ${_gRPC_UPB_LIBRARIES} | |
| ${_gRPC_ALLTARGETS_LIBRARIES} | |
| absl::flat_hash_map | |
| absl::flat_hash_set | |
| absl::inlined_vector | |
| absl::bind_front | |
| absl::function_ref | |
| absl::hash | |
| absl::type_traits | |
| absl::statusor | |
| absl::span | |
| absl::variant | |
| absl::utility | |
| gpr | |
| ${_gRPC_SSL_LIBRARIES} | |
| address_sorting | |
| upb | |
| ) |
even though many of those libraries are only used internally within grpc AFAICT (in particular zlib, re2, c-ares and abseil; not 100% sure about openssl; address_sorting and upb are less critical because they are vendored and therefore always present).
To avoid that every project using grpc now has to declare a dependence on zlib (to be able to link to grpc, which has it in its public link interface), we tried to patch this for our builds. This is not really "live" at the moment, and also not meant to be a definitive patches, but I've uploaded a branch displaying those changes; the most relevant is h-vetinari@ebd16ea, the other two then deal only with abseil (which more tedious/voluminous but still important because by default, abseil's ABI changes per C++ version, and we do not want to "infect" other packages with grpc's C++ version, especially if abseil is only used internally, and not e.g. as part of the public API).
Would the grpc project be willing to distinguish which libraries are necessary internally / externally?
Thanks for your consideration. :)
PS. Here's a good (but long) overview of public vs. private linkage in CMake.