Stack from ghstack:

• Add sanity checks for NCCL detection. #22819 Add sanity checks for NCCL detection.
• Let CMake handle NCCL detection instead of our handcrafted Python script. #22818 Let CMake handle NCCL detection instead of our handcrafted Python script.

• USE_NCCL, USE_SYSTEM_NCCL, USE_STATIC_NCCL are made proper CMake build
  options that depend on other options.
• Detections using folders such as lib64, lib64/ are automatically
  handled in find_library in CMake.
• NCCL_ROOT_DIR in findNCCL.cmake is changed to NCCL_ROOT, which is the
  standard CMake variable that specifies the root dir of a library (and
  therefore supports a wide range of automatic detection). E.g., see
  https://cmake.org/cmake/help/latest/envvar/PackageName_ROOT.html

How does the current code subsume all detections in the deleted nccl.py?

• The dependency of USE_NCCL on the OS and USE_CUDA is handled as dependency options in CMakeLists.txt.

• The main NCCL detection happens in FindNCCL.cmake, which is called by nccl.cmake. When USE_SYSTEM_NCCL is false, the previous Python code defer the detection to find_package(NCCL). The change in nccl.cmake retains this.

• USE_STATIC_NCCL in the previous Python code simply changes the name of the detected library. This is done in IF (USE_STATIC_NCCL).

• Now we only need to look at how the lines below line 20 in nccl.cmake are subsumed. These lines list paths to header and library directories that NCCL headers and libraries may reside in and try to search these directories for the key header and library files in turn. These are done by find_path for headers and find_library for the library files in FindNCCL.cmake.

  • The call of find_path (Search for NO_DEFAULT_PATH in the link) by default searches for headers in <prefix>/include for each <prefix> in CMAKE_PREFIX_PATH and CMAKE_SYSTEM_PREFIX_PATH. Like the Python code, this commit sets CMAKE_PREFIX_PATH to search for <prefix> in NCCL_ROOT_DIR and home to CUDA. CMAKE_SYSTEM_PREFIX_PATH includes the standard directories such as /usr/local and /usr. NCCL_INCLUDE_DIR is also specifically handled.

  • Similarly, the call of find_library (Search for NO_DEFAULT_PATH in the link) by default searches for libraries in directories including <prefix>/lib for each <prefix> in CMAKE_PREFIX_PATH and CMAKE_SYSTEM_PREFIX_PATH. But it also handles the edge cases intended to be solved in the Python code more properly:

    • It only searches for <prefix>/lib64 (and <prefix>/lib32) if it is appropriate on the system.
    • It only searches for <prefix>/lib/<arch> for the right <arch>, unlike the Python code searches for lib/<arch> in a generic way (e.g., the Python code searches for /usr/lib/x86_64-linux-gnu but in reality systems have /usr/lib/x86_64-some-customized-name-linux-gnu, see https://unix.stackexchange.com/a/226180/38242 ).

Regarding for relevant issues:

• Add some more locations to search for nccl. #12063 and Fix detection of NCCL_INCLUDE_DIR #2877: These are properly handled, as explained in the updated comment.
• Improve Windows Compatibility(for csrc/scripts) #2941 does not changes NCCL detection specifically for Windows (it changed CUDA detection).
• b7e258f A versioned library detection is added, but the order is reversed: The unversioned library becomes preferred. This is because normally unversioned libraries are linked to versioned libraries and preferred by users, and local installation by users are often unversioned. Like the document of find_library suggests:

When using this to specify names with and without a version suffix, we recommend specifying the unversioned name first so that locally-built packages can be found before those provided by distributions.

Differential Revision: D16281714