Skip to content

[Triton 3.6 pin update] can't build pytorch-triton-rocm package #169313

@atalman

Description

@atalman

🐛 Describe the bug

Related to #168096

Please see: https://github.com/pytorch/pytorch/actions/runs/19831555521/job/56818794230

Can't build triton rocm:

[312/461] Building CXX object third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o
FAILED: [code=1] third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o 
/usr/bin/clang++  -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party/amd -I/tmp/tmpsp56q_zf/triton/third_party/amd -I/tmp/tmpsp56q_zf/triton/include -I/tmp/tmpsp56q_zf/triton/. -I/root/.triton/llvm/llvm-f6ded0be-almalinux-x64/include -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/include -I/tmp/tmpsp56q_zf/triton/third_party -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party -I/tmp/tmpsp56q_zf/triton/python/src -I/tmp/tmpsp56q_zf/triton/third_party/amd/include -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party/amd/include -I/tmp/tmpsp56q_zf/triton/third_party/amd/lib/TritonAMDGPUTransforms/../../include -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party/amd/lib/TritonAMDGPUTransforms/../../include -isystem /opt/python/cp310-cp310/include/python3.10 -isystem /opt/python/cp310-cp310/lib/python3.10/site-packages/pybind11/include -D__STDC_FORMAT_MACROS  -fPIC -std=gnu++17 -Werror -Wno-covered-switch-default -fvisibility=hidden -O2 -g -std=gnu++17 -MD -MT third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o -MF third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o.d -o third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o -c /tmp/tmpsp56q_zf/triton/third_party/amd/python/triton_amd.cc
In file included from /tmp/tmpsp56q_zf/triton/third_party/amd/python/triton_amd.cc:5:
In file included from /tmp/tmpsp56q_zf/triton/third_party/amd/include/hipblas_instance.h:4:
/tmp/tmpsp56q_zf/triton/third_party/amd/include/hipblas_types.h:4:10: fatal error: '../backend/include/hipblas-common/hipblas-common.h' file not found
    4 | #include "../backend/include/hipblas-common/hipblas-common.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

[313/461] Building CXX object third_party/amd/lib/Analysis/CMakeFiles/TritonAMDAnalysis.dir/AMDGPUAllocation.cpp.o
[314/461] Building CXX object third_party/amd/lib/Analysis/CMakeFiles/TritonAMDAnalysis.dir/RangeAnalysis.cpp.o
[315/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/AsyncUtility.cpp.o
[316/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/AtomicRMWOpsEmitter.cpp.o
[317/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/ConvertLayoutOpToLLVM.cpp.o
[318/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/AllocateSharedMemory.cpp.o
[319/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/MemoryOpToLLVM.cpp.o
[320/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/BufferOpsEmitter.cpp.o
[321/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM/FMA.cpp.o
[322/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/TensorPtrOpsToLLVM.cpp.o
[323/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM/MFMA.cpp.o
[324/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/MaskedOpsToLLVM.cpp.o
[325/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM/WMMA.cpp.o
[326/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM.cpp.o
[327/461] Building CXX object third_party/amd/lib/Dialect/TritonAMDGPU/IR/CMakeFiles/TritonAMDGPUIR.dir/Dialect.cpp.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/tmp/tmpsp56q_zf/triton/setup.py", line 824, in <module>
    setup(
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 186, in setup
    return run_commands(dist)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
    dist.run_commands()
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
    self.run_command(cmd)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
    super().run_command(command)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/tmp/tmpsp56q_zf/triton/setup.py", line 717, in run
    super().run()
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
    self.run_command("build")
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
    self.distribution.run_command(command)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
    super().run_command(command)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
    self.distribution.run_command(command)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
    super().run_command(command)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/tmp/tmpsp56q_zf/triton/setup.py", line 383, in run
    self.run_command('build_ext')
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
    self.distribution.run_command(command)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
    super().run_command(command)
  File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/tmp/tmpsp56q_zf/triton/setup.py", line 422, in run
    self.build_extension(ext)
  File "/tmp/tmpsp56q_zf/triton/setup.py", line 532, in build_extension
    subprocess.check_call(["cmake", "--build", "."] + build_args, cwd=cmake_dir)
  File "/opt/python/cp310-cp310/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'TritonRelBuildWithAsserts', '-j16']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/pytorch/.github/scripts/build_triton_wheel.py", line 164, in <module>
    main()
  File "/pytorch/.github/scripts/build_triton_wheel.py", line 151, in main
    build_triton(
  File "/pytorch/.github/scripts/build_triton_wheel.py", line 117, in build_triton
    check_call(
  File "/opt/python/cp310-cp310/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/python/cp310-cp310/bin/python', 'setup.py', 'bdist_wheel']' returned non-zero exit status 

Versions

2.10.0

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @jataylo @hongxiayang @naromero77amd @pragupta @jerrymannil @xinyazhang @iupaikov-amd

Metadata

Metadata

Assignees

Labels

module: rocmAMD GPU support for PytorchtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions