-
Notifications
You must be signed in to change notification settings - Fork 27.1k
Closed
Labels
module: rocmAMD GPU support for PytorchAMD GPU support for PytorchtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Milestone
Description
🐛 Describe the bug
Related to #168096
Please see: https://github.com/pytorch/pytorch/actions/runs/19831555521/job/56818794230
Can't build triton rocm:
[312/461] Building CXX object third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o
FAILED: [code=1] third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o
/usr/bin/clang++ -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party/amd -I/tmp/tmpsp56q_zf/triton/third_party/amd -I/tmp/tmpsp56q_zf/triton/include -I/tmp/tmpsp56q_zf/triton/. -I/root/.triton/llvm/llvm-f6ded0be-almalinux-x64/include -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/include -I/tmp/tmpsp56q_zf/triton/third_party -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party -I/tmp/tmpsp56q_zf/triton/python/src -I/tmp/tmpsp56q_zf/triton/third_party/amd/include -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party/amd/include -I/tmp/tmpsp56q_zf/triton/third_party/amd/lib/TritonAMDGPUTransforms/../../include -I/tmp/tmpsp56q_zf/triton/build/cmake.linux-x86_64-cpython-3.10/third_party/amd/lib/TritonAMDGPUTransforms/../../include -isystem /opt/python/cp310-cp310/include/python3.10 -isystem /opt/python/cp310-cp310/lib/python3.10/site-packages/pybind11/include -D__STDC_FORMAT_MACROS -fPIC -std=gnu++17 -Werror -Wno-covered-switch-default -fvisibility=hidden -O2 -g -std=gnu++17 -MD -MT third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o -MF third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o.d -o third_party/amd/CMakeFiles/TritonAMD.dir/python/triton_amd.cc.o -c /tmp/tmpsp56q_zf/triton/third_party/amd/python/triton_amd.cc
In file included from /tmp/tmpsp56q_zf/triton/third_party/amd/python/triton_amd.cc:5:
In file included from /tmp/tmpsp56q_zf/triton/third_party/amd/include/hipblas_instance.h:4:
/tmp/tmpsp56q_zf/triton/third_party/amd/include/hipblas_types.h:4:10: fatal error: '../backend/include/hipblas-common/hipblas-common.h' file not found
4 | #include "../backend/include/hipblas-common/hipblas-common.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[313/461] Building CXX object third_party/amd/lib/Analysis/CMakeFiles/TritonAMDAnalysis.dir/AMDGPUAllocation.cpp.o
[314/461] Building CXX object third_party/amd/lib/Analysis/CMakeFiles/TritonAMDAnalysis.dir/RangeAnalysis.cpp.o
[315/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/AsyncUtility.cpp.o
[316/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/AtomicRMWOpsEmitter.cpp.o
[317/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/ConvertLayoutOpToLLVM.cpp.o
[318/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/AllocateSharedMemory.cpp.o
[319/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/MemoryOpToLLVM.cpp.o
[320/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/BufferOpsEmitter.cpp.o
[321/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM/FMA.cpp.o
[322/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/TensorPtrOpsToLLVM.cpp.o
[323/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM/MFMA.cpp.o
[324/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/MaskedOpsToLLVM.cpp.o
[325/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM/WMMA.cpp.o
[326/461] Building CXX object third_party/amd/lib/TritonAMDGPUToLLVM/CMakeFiles/TritonAMDGPUToLLVM.dir/DotOpToLLVM.cpp.o
[327/461] Building CXX object third_party/amd/lib/Dialect/TritonAMDGPU/IR/CMakeFiles/TritonAMDGPUIR.dir/Dialect.cpp.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/tmp/tmpsp56q_zf/triton/setup.py", line 824, in <module>
setup(
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
return distutils.core.setup(**attrs)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 186, in setup
return run_commands(dist)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
dist.run_commands()
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
self.run_command(cmd)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
File "/tmp/tmpsp56q_zf/triton/setup.py", line 717, in run
super().run()
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
self.run_command("build")
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
self.distribution.run_command(command)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
self.distribution.run_command(command)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
File "/tmp/tmpsp56q_zf/triton/setup.py", line 383, in run
self.run_command('build_ext')
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
self.distribution.run_command(command)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
super().run_command(command)
File "/opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
cmd_obj.run()
File "/tmp/tmpsp56q_zf/triton/setup.py", line 422, in run
self.build_extension(ext)
File "/tmp/tmpsp56q_zf/triton/setup.py", line 532, in build_extension
subprocess.check_call(["cmake", "--build", "."] + build_args, cwd=cmake_dir)
File "/opt/python/cp310-cp310/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'TritonRelBuildWithAsserts', '-j16']' returned non-zero exit status 1.
Traceback (most recent call last):
File "/pytorch/.github/scripts/build_triton_wheel.py", line 164, in <module>
main()
File "/pytorch/.github/scripts/build_triton_wheel.py", line 151, in main
build_triton(
File "/pytorch/.github/scripts/build_triton_wheel.py", line 117, in build_triton
check_call(
File "/opt/python/cp310-cp310/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/python/cp310-cp310/bin/python', 'setup.py', 'bdist_wheel']' returned non-zero exit status
Versions
2.10.0
cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @jataylo @hongxiayang @naromero77amd @pragupta @jerrymannil @xinyazhang @iupaikov-amd
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
module: rocmAMD GPU support for PytorchAMD GPU support for PytorchtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Type
Projects
Status
Done