CUDA 12 has been released, but we've identified several blocking issues (some code/API compatibility, some API functionality) that need to be addressed before a PyTorch + CUDA 12 build/environment could be considered usable by mainstream users. We're creating this issue to hopefully avoid duplicating work in identifying and resolving issues.

Required code changes for build-breaking changes:

• Drop compute capability < 5.0 (Drop compute capability < 5.0 in CUDA 12 #91213)
• Wrap cudaGraphInstantiate as the API has changed ([CUDA12] Make PyTorch compatible with CUDA 12 #91118)
• Skip building cudaProfilerInitialize ([CUDA12] Make PyTorch compatible with CUDA 12 #91118, Follow up on CUDA 12 support for PyTorch/Caffe2 #95582)
• Support const CUDASparse descriptors (Update cuSPARSE usage for CUDA 12.0 #90765, [CUDA 12] Fix the endif guard position for cusparse const descriptors #90897, [CUDA12] Clean up deprecated APIs #91050)

Known API functionality issues:

• cudaSetDevice is currently eagerly creating a CUDA context when called, which is a significant departure from previous behavior. This potentially creates many unused contexts that waste GPU memory (especially in DDP workloads), and would either require framework or CUDA toolkit code changes to resolve. ([CUDA12] Conditionally set device in autograd engine #91191, [DO NOT MERGE] [CUDA12] Conditionally set device in device guard #91219, [CUDA12] set_device change #94864)

Nightly CUDA 12.1 builds are now available for the adventurous:
@ https://pytorch.org/get-started/locally/
e.g.,

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121

CC @ptrblck @crcrpar @IvanYashchuk @xwang233 @Aidyn-A @Fuzzkatt @syed-ahmed @puririshi98 @ngimel @malfet

Please feel free to update this issue as new items are discovered/resolved.

cc @ngimel