Skip to content

Library targets are a mess #3909

@ben-clayton

Description

@ben-clayton

Pulling together the following issues:

To summarize the above:

BUILD_SHARED_LIBS is not properly respected, and distro packages are now broken

This is mostly my fault - #3490 fixed Windows builds that had issues building .dlls (#3482), but these changes have caused distro package problems (#3626). Some background:

Prior to #3490, we had "just" the SPIRV_TOOLS targets ${SPIRV_TOOLS} and ${SPIRV_TOOLS}-shared:

  • ${SPIRV_TOOLS} was either static or shared based on BUILD_SHARED_LIBS and had all symbols visible as default (symbols not annotated with SPIRV_TOOLS_EXPORT were also exposed)
  • ${SPIRV_TOOLS}-shared was (and still is) always shared and hides non-pubic-API symbols (only SPIRV_TOOLS_EXPORT symbols are visible).

Both libSPIRV-Tools.so and libSPIRV-Tools-shared.so appear to be included in distro packages.

With #3490:

  • ${SPIRV_TOOLS} was renamed to ${SPIRV_TOOLS}-static and was forced to always build as static.
  • ${SPIRV_TOOLS} is now an alias to ${SPIRV_TOOLS}-static or ${SPIRV_TOOLS}-shared based on BUILD_SHARED_LIBS.
  • Targets that previously depended on ${SPIRV_TOOLS} now depend on ${SPIRV_TOOLS}-static.
  • Other library targets always build as static.

While the changes work transparently when using SPIRV-Tools as a sub-project of another CMake project, it breaks the install package, as these were expecting all libraries to be generated as shared when BUILD_SHARED_LIBS=1.

I can only speculate that the reason we originally had the ${SPIRV_TOOLS} and ${SPIRV_TOOLS}-shared targets was due to symbol visibility - there are a number of targets in this project that depend on ${SPIRV_TOOLS} and use non-public APIs. This is discussed here.

Libraries are not versioned

None of the libraries are versioned, there's no use of versioned SONAMEs. Discussed in detail here.
Consequently, updating to a new version of SPIRV-Tools may break projects in a number of exciting ways.


I propose the following:

  1. We replace the ${SPIRV_TOOLS}, ${SPIRV_TOOLS}-shared and ${SPIRV_TOOLS}-static targets with a single ${SPIRV_TOOLS} target that respects BUILD_SHARED_LIBS

The ${SPIRV_TOOLS} target will hide non-public APIs by default (same for static as shared).

Non public symbols in use by other SPIRV-Tools targets will need to be moved out to other static utility libraries (which will not be included in the install output) that can be privately linked by both ${SPIRV_TOOLS} and other dependee targets.

As the ${SPIRV_TOOLS}-static target has not been hiding private symbols, there's a good chance that external projects are relying on these private APIs. These will have to be fixed, or we need to promote more of the library from private to public.

This will also require fixing up downstream projects that explicitly use ${SPIRV_TOOLS}-static (examples: clspv, dxc) and ${SPIRV_TOOLS}-shared.

Distros will likely have to remove the libSPIRV-Tools-shared.so library from their packages.

  1. We start versioning our shared libraries with VERSION and SOVERSION

The version numbers will be derived from the CHANGES file, and so will always be kept in sync with the source.

  1. We start enforcing semantic versioning rules

We enforce a major version bump whenever there's a breaking API / ABI change to the public API.
We enforce a minor version bump whenever there's backwards-compatible, but new public APIs added.

I have been working on tooling for detecting and enforcing this as part of presubmits.

  1. We classify the public API for SPIRV-Tools[-link,-opt,-reduce]

The ${SPIRV_TOOLS} target attempts to classify public from non-public API using the SPIRV_TOOLS_EXPORT macro.

In order to version the other libraries, we'll need to limit the symbol visibility to well defined public API, otherwise any change to these targets will technically break API / ABI compatibility.

If no public API can be found, we should question the inclusion of these libraries in package distributions.


I'm aware that these proposals will likely cause breakages for downstream projects, but I believe the long-term benefits outweigh the short term pain.

Thoughts and comments welcome.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions