Ok, I've been iterating on things in a simplified example in https://github.com/wjwwood/weak_linking.git.

I have one pr (wjwwood/weak_linking#3) which is (as near as I can tell) functionally the same as what we have now with the implementation moved into a cpp file from the header. It has Linux passing but OS X and Windows failing.

The crux of the issue is that the cpp files in rclcpp will use symbols declared in the headers in rmw, which are defined in an rmw implementation, but the implementation is neither available when building librclcpp nor could we pick which one to use if they were. On Linux this is ok, because the default behavior when linking is to ignore missing symbols. This means that the librclcpp library is incomplete, because it references symbols which have not been resolved yet and instead will try to at runtime. The default on OS X is to require all symbols be resolved at link time by way of providing the libraries which define them as link arguments. So by default this fails with OS X, however, this can be changed.

The next pr (wjwwood/weak_linking#2) tells the OS X linker to handle undefined symbols by looking them up dynamically at runtime. This allows Linux and OS X to pass, but Windows is still lagging behind.

I've tried a lot of things to get this to work on Windows. I'll try to summarize them:

Use the /FORCE option for link.exe.
- This allows librclcpp to link with "warnings" about the missing symbols and for the user's executables to compile and link.
- When the user executable is run though I get an access violation when trying to call a function from the rmw implementation library.
- Looking at the executable with Dependency Walker it is linked against both librclcpp and librmw_impl{1 or 2}.
- I also tried swapping the library link orders.
- To get this to link against the implementation library I have to also pass /INCLUDE:foo to the link arguments for the user's executable.
- From https://msdn.microsoft.com/en-us/library/2s3hwbhs.aspx: "This feature is useful for including a library object that otherwise would not be linked to the program."
Use the /DELAYLOAD option.
- I couldn't get this to work, but it could possibly work if we produced a generic librmw_impl which was a copy of what ever specific implementation we wanted to use, since we have to have a dll name to delay loading.
I also played with the incremental linking options, but I think that's a dead-end too.
I considered true "weak linking" (not dynamic symbol lookup which is what the other pr's are actually doing).
- Reference: http://stackoverflow.com/questions/2290587/gcc-style-weak-linking-in-visual-studio
- This functionality is provided by gcc/clang with __attribute__(weak), but Windows splits the functionality into to __declspec(selectany), which allows for the definition of a symbol multiple times, allowing the compiler to get any of the available ones. The other option is sort of arcane, see: http://stackoverflow.com/a/11529277/671658
- I was unable to get this working, but maybe I could with some more work.
- I'm still working through http://blogs.msdn.com/b/oldnewthing/archive/2013/01/08/10383017.aspx to understand how ::operator new is implemented in MSVC++.
- This approach could work by providing weak linking in a "default" library which we link against librclcpp when building it, but that does nothing. Then those symbols are overridden by the ones in the rmw implementation in the user's executable.

So, as usual Windows is the stick in the mud.

Unless we figure this problem out on Windows, we'll have to restructure how we build the executables and how/when we select the rmw implementation to be used.

One extreme action is that we could restructure our code so that rclcpp depends on rmw_implemenation, which directly depends on all of the default implementations, but only passes one along. Which one it passes along is controlled by a CMake variable and/or by disabling packages which provide alternative implementations. That way librclcpp will be built after the rmw implementation library and can be linked against it.

Pros:

Works around the need for dynamic symbol lookup, allowing us to make rclcpp a library.
Simpler dependency layout and easier to understand.

Cons:

Moves us farther away from ABI compatible switching of implementations (choosing an implementation at runtime rather than build time).
Removes the ability to have user code generate binaries for each implementation.
Removes the ability to easily automate cross vendor testing.

While we're still testing different DDS and RTPS vendors, the lost features are sort of a deal breaker for right now. So I don't think this is a good solution, but in the long term, when we've settled on one implementation and most people use one at a time, this might be ok.

A slightly different solution might be to make rclcpp depend on rmw_implementations, but instead of picking an implementation before building librclpp, build a version of librclcpp for each implementation and link one of them against the user's executable, similar to how our message package libraries are currently linked.

Pros:

Should let us make rclcpp into a library.

Cons:

Moves us further away from ABI swappable implementations since we'll have to rely on a symbol in the user's code to help the linker pick the right rclcpp library (same as for the message packages).
Generates extra libraries, which increases clutter and build times. (though having rclcpp as a library may decrease the overall build time)

Another idea I have is to use template functions in rclcpp as an indirection to calling the rmw_* functions. This would delay the use of any rmw_* functions until someone uses the rclcpp headers. However, this would not work for intermediate libraries which use rclcpp but are created before an implementation has been chosen. It would also add a lot of clutter to the rclcpp code base with many functions that only call other functions. It might also require us to invent duplicate types to avoid using symbols which are in the rmw implementation, but I'm not sure about that. This is my least baked idea.

refactor into library #48

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions