Skip to content

Conversation

@amadio
Copy link
Member

@amadio amadio commented Mar 15, 2025

In preparation to address issues #2415 and #2432, I started to perform a general restructuring of the build system for XRootD 5.8, to use more idiomatic and modern CMake constructs. This pull request is my current draft of it, which I would like you to review.

Before doing anything, I had to set the output directory for binaries and libraries to be <BUILD_DIR>/bin and <BUILD_DIR>/lib, otherwise paths to binaries such as xrdcp, etc, change depending on which CMake directory they were build. Overall, I think this makes it much simpler to manage PATH in tests, etc, since you only need <BUILD_DIR>/bin and not src/ for server binaries and src/XrdCl/ for client binaries, etc. Libraries and plugins are automatically loaded from the build directory when specified only by name, so in configuration files, if you have full paths, you can just use the basename without the directory part in the future.

I am moving most src/Xrd*.cmake scripts to src/Xrd*/CMakeLists.txt and moving build logic as close as possible to where it's needed, such as calls to find_package, etc. I would also like to use this opportunity (since I will need to touch almost all build system files) to update the CMake style we use to be more consistent across the whole repository, maybe something similar to what's shown here and here. Please let me know your opinions, I'd prefer to adjust style before merging this.

I still have to deal with XrdDaemons.cmake, XrdHeaders.cmake, XrdServer.cmake, and XrdUtils.cmake. I'd like to hear your thoughts on how to move their bits down to subdirectories. My plan is to create the libraries and binaries in src/CMakeLists.txt and use target_sources in subdirectories to add the files to the targets as needed.

After this restructuring is over, I will work on improving the exported files (i.e. XRootDConfig.cmake) and consider creating functions to manage libraries and plugins in a more central way in case global changes are needed. For now, I'm just using plain CMake commands everywhere.

Thanks for reviewing!

Update: I've finished most of the remaining work. Now only XrdHeaders.cmake needs to be dealt with. I've dealt with #2415 and #2432 as the last thing in the pull request, but it's still not over yet, I might add a few more changes before adding to master and into 5.8.0.

@amadio amadio added this to the 5.8.0 milestone Mar 15, 2025
@bbockelm
Copy link
Collaborator

@amadio - the improvements look great! Much more idiomatic from the CMake side.

Two thoughts, both for the "next round" I think:

  • It might be worthwhile to reach out to CMake-based dependencies (such as SciTokens) and ask them to modernize and install CMake targets as part of their build process.
  • For hard dependencies, we can use FetchContent_MakeAvailable (here's an example from a different project) to pull things if they're not present. That's allowed me to get rid of in-tree vendor dependencies such as tinyxml. That makes it easier for folks getting started: if done for the entire dependency tree, you can start with nothing but a build chain and have a pretty functional xrootd build. If you take this approach, coordinate with packagers such as @ellert as most distros have a strict no-network-connectivity-during-build and there needs to be a straightforward mechanism to skip this feature.

@amadio
Copy link
Member Author

amadio commented Mar 16, 2025

Thanks for taking a look at the PR!

  • It might be worthwhile to reach out to CMake-based dependencies (such as SciTokens) and ask them to modernize and install CMake targets as part of their build process.

Yes, I'm coming for you, haha. I recently realized that scitokens-cpp does not export a version anywhere, so it's not possible to detect what version is installed. It would be nice to at least export something in the headers or a ScitokensConfigVersion.cmake and ScitokensConfig.cmake, like most packages.

As for bundling dependencies, well, I'm not a big fan of that. When I was working in ROOT I was always fighting against it, but wasn't heard. I think bundled dependencies can easily lead to security vulnerabilities and suffer from code rot, since they are rarely updated. In XRootD, I've removed the bundled isa-l already, and hope to remove the rest instead of adding more. It's not a big deal to install the dependencies on most OSs and Linux distros, since XRootD is already officially packaged in most places. In most distros it's a single command using either the xrootd.spec or debian/control file.

@amadio
Copy link
Member Author

amadio commented Mar 19, 2025

I'm going to split this into two parts, one which I will merge soon to master and which contains part of the changes here, plus a few extras. The other, which I will keep in this pull request, is to do the refactoring for moving Xrd*.cmake to Xrd*/CMakeLists.txt, which I am still working on and will merge into a future release.

@amadio amadio removed this from the 5.8.0 milestone Mar 19, 2025
@amadio amadio moved this to Backlog in Release Planning Apr 10, 2025
@amadio amadio moved this from Backlog to In progress in Release Planning Apr 10, 2025
@amadio amadio force-pushed the build-system branch 2 times, most recently from 9ea6394 to 5496199 Compare April 15, 2025 12:00
amadio added 10 commits April 15, 2025 16:34
This makes the placement of binaries and libraries independent of
the directory structure of the CMake build system, which makes it
easier to set paths to client and server binaries in tests, as well
as ensuring that libraries and plugins from the build itself are
always used when running the test suite. Moreover, this allows to
migrate the build system to use more idiomatic CMake by using, for
example, add_subdirectory(Xrd*) and src/Xrd*/CMakeLists.txt instead
of the current include(Xrd*) and src/Xrd*.cmake, to keep plugins in
the same place as other libraries.

Note: These variables must be set in the CACHE to allow Python builds
using setup.py to override them, so that libraries are placed in the
correct directory for later installation by pip.
amadio added 21 commits April 15, 2025 16:35
This is the first version that provides the multi interface, so it
allows us to remove checks and ifdefs for this from the project.
This is to make the headers, source files, namespaces, etc all the
same name as the plugin itself, for consistency.
…opriate

The intention is to allow XRootD to be embedded with add_subdirectory(xrootd)
within another CMake project.
@amadio amadio changed the title WIP: CMake build system cleanups and restructuring CMake build system cleanups and restructuring Apr 16, 2025
@amadio amadio marked this pull request as ready for review April 16, 2025 11:41
@amadio amadio merged commit a7bdbd8 into xrootd:master Apr 23, 2025
11 checks passed
@github-project-automation github-project-automation bot moved this from In progress to Done in Release Planning Apr 23, 2025
@amadio amadio deleted the build-system branch April 23, 2025 15:05
@amadio amadio added this to the 5.8.2 milestone May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants