Documentation

Check out https://github.com/pytorch/test-infra/tree/main/aws/ami/windows

I was able to compile, link, install and import the PyTorch including CUDA with VS2022 with the following steps:

Install git, Visual Studio 2022.
Install Nvidia CUDA Toolkit 11.7.1 from https://developer.download.nvidia.com/compute/cuda/11.7.1/local_installers/cuda_11.7.1_516.94_windows.exe
Install Nvidia cuDNN 8.5.0 from https://developer.nvidia.com/compute/cudnn/secure/8.5.0/local_installers/11.7/cudnn-windows-x86_64-8.5.0.96_cuda11-archive.zip
Install 7-Zip from https://www.7-zip.org/a/7z2201-x64.exe
Add C:\Program Files\7-Zip to PATH
Download jq from https://github.com/stedolan/jq/releases/download/jq-1.6/jq-win64.exe , rename it to jq.exe and place it somewhere in the PATH.
git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
git submodule sync
git submodule update --init --recursive --jobs 0
"C:\Program Files\Git\bin\sh.exe" .jenkins\pytorch\win-build.sh
call C:\Projects\pytorch\Miniconda3\Scripts\activate.bat C:\Projects\pytorch\Miniconda3
cd .. (otherwise the import fails with import torch error! #48578)
set PATH=%PATH%;C:\Projects\pytorch\Miniconda3\pkgs\intel-openmp-2021.4.0-haa95532_3556\Library\bin (otherwise the import fails with OSError: [WinError 126] The specified module could not be found. Error loading "C:\Projects\pytorch\torch\lib\backend_with_compiler.dll" or one of its dependencies.)
python
import torch

and with the following changes/workarounds to the build scripts:

diff --git a/.gitignore b/.gitignore
index 5dbad08f4f..7d1f2c798a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -338,3 +338,5 @@ buck-out/
 # Downloaded libraries
 third_party/ruy/
 third_party/glog/
+
+Miniconda3
diff --git a/.jenkins/pytorch/win-build.sh b/.jenkins/pytorch/win-build.sh
index 2de3022d40..6f078cdc49 100755
--- a/.jenkins/pytorch/win-build.sh
+++ b/.jenkins/pytorch/win-build.sh
@@ -40,20 +40,20 @@ fi
 
 export SCRIPT_HELPERS_DIR=$SCRIPT_PARENT_DIR/win-test-helpers
 
-set +ex
-grep -E -R 'PyLong_(From|As)(Unsigned|)Long\(' --exclude=python_numbers.h torch/
-PYLONG_API_CHECK=$?
-if [[ $PYLONG_API_CHECK == 0 ]]; then
-  echo "Usage of PyLong_{From,As}{Unsigned}Long API may lead to overflow errors on Windows"
-  echo "because \`sizeof(long) == 4\` and \`sizeof(unsigned long) == 4\`."
-  echo "Please include \"torch/csrc/python_numbers.h\" and use the correspoding APIs instead."
-  echo "PyLong_FromLong -> THPUtils_packInt32 / THPUtils_packInt64"
-  echo "PyLong_AsLong -> THPUtils_unpackInt (32-bit) / THPUtils_unpackLong (64-bit)"
-  echo "PyLong_FromUnsignedLong -> THPUtils_packUInt32 / THPUtils_packUInt64"
-  echo "PyLong_AsUnsignedLong -> THPUtils_unpackUInt32 / THPUtils_unpackUInt64"
-  exit 1
-fi
-set -ex
+# set +ex
+# grep -E -R 'PyLong_(From|As)(Unsigned|)Long\(' --exclude=python_numbers.h torch/
+# PYLONG_API_CHECK=$?
+# if [[ $PYLONG_API_CHECK == 0 ]]; then
+#   echo "Usage of PyLong_{From,As}{Unsigned}Long API may lead to overflow errors on Windows"
+#   echo "because \`sizeof(long) == 4\` and \`sizeof(unsigned long) == 4\`."
+#   echo "Please include \"torch/csrc/python_numbers.h\" and use the correspoding APIs instead."
+#   echo "PyLong_FromLong -> THPUtils_packInt32 / THPUtils_packInt64"
+#   echo "PyLong_AsLong -> THPUtils_unpackInt (32-bit) / THPUtils_unpackLong (64-bit)"
+#   echo "PyLong_FromUnsignedLong -> THPUtils_packUInt32 / THPUtils_packUInt64"
+#   echo "PyLong_AsUnsignedLong -> THPUtils_unpackUInt32 / THPUtils_unpackUInt64"
+#   exit 1
+# fi
+# set -ex
 
 "$SCRIPT_HELPERS_DIR"/build_pytorch.bat
 
diff --git a/.jenkins/pytorch/win-test-helpers/build_pytorch.bat b/.jenkins/pytorch/win-test-helpers/build_pytorch.bat
index 65ba7ef323..193856e937 100644
--- a/.jenkins/pytorch/win-test-helpers/build_pytorch.bat
+++ b/.jenkins/pytorch/win-test-helpers/build_pytorch.bat
@@ -13,6 +13,9 @@ set PATH=C:\Program Files\CMake\bin;C:\Program Files\7-Zip;C:\ProgramData\chocol
 :: log sizes are too long, but leaving this here incase someone wants to use it locally
 :: set CMAKE_VERBOSE_MAKEFILE=1
 
+set USE_CUDA=1
+set CUDA_VERSION=11.7
+set BUILD_ENVIRONMENT=
 
 set INSTALLER_DIR=%SCRIPT_HELPERS_DIR%\installation-helpers
 
@@ -42,11 +45,7 @@ if not errorlevel 0 exit /b
 
 :: Override VS env here
 pushd .
-if "%VC_VERSION%" == "" (
-    call "C:\Program Files (x86)\Microsoft Visual Studio\%VC_YEAR%\%VC_PRODUCT%\VC\Auxiliary\Build\vcvarsall.bat" x64
-) else (
-    call "C:\Program Files (x86)\Microsoft Visual Studio\%VC_YEAR%\%VC_PRODUCT%\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%VC_VERSION%
-)
+call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
 if errorlevel 1 exit /b
 if not errorlevel 0 exit /b
 @echo on
@@ -124,7 +123,7 @@ for /f "usebackq tokens=*" %%i in (`set`) do echo set "%%i" >> %TMP_DIR_WIN%\ci_
 @echo on
 
 if "%REBUILD%" == "" (
-  if NOT "%BUILD_ENVIRONMENT%" == "" (
+  if not "%BUILD_ENVIRONMENT%" == "" (
     :: Create a shortcut to restore pytorch environment
     echo @echo off >> %TMP_DIR_WIN%/ci_scripts/pytorch_env_restore_helper.bat
     echo call "%TMP_DIR_WIN%/ci_scripts/pytorch_env_restore.bat" >> %TMP_DIR_WIN%/ci_scripts/pytorch_env_restore_helper.bat
@@ -136,14 +135,14 @@ if "%REBUILD%" == "" (
   )
 )
 :: tests if BUILD_ENVIRONMENT contains cuda11 as a substring
-if not x%BUILD_ENVIRONMENT:cuda11=%==x%BUILD_ENVIRONMENT% (
-   set BUILD_SPLIT_CUDA=ON
-)
+::if not x%BUILD_ENVIRONMENT:cuda11=%==x%BUILD_ENVIRONMENT% (
+::  set BUILD_SPLIT_CUDA=ON
+::)
 
 python setup.py install --cmake && sccache --show-stats && (
   if "%BUILD_ENVIRONMENT%"=="" (
     echo NOTE: To run `import torch`, please make sure to activate the conda environment by running `call %CONDA_PARENT_DIR%\Miniconda3\Scripts\activate.bat %CONDA_PARENT_DIR%\Miniconda3` in Command Prompt before running Git Bash.
-  ) else (
+) else (
     7z a %TMP_DIR_WIN%\%IMAGE_COMMIT_TAG%.7z %CONDA_PARENT_DIR%\Miniconda3\Lib\site-packages\torch %CONDA_PARENT_DIR%\Miniconda3\Lib\site-packages\torchgen %CONDA_PARENT_DIR%\Miniconda3\Lib\site-packages\caffe2 %CONDA_PARENT_DIR%\Miniconda3\Lib\site-packages\functorch && copy /Y "%TMP_DIR_WIN%\%IMAGE_COMMIT_TAG%.7z" "%PYTORCH_FINAL_PACKAGE_DIR%\"
     if errorlevel 1 exit /b
     if not errorlevel 0 exit /b
diff --git a/.jenkins/pytorch/win-test-helpers/installation-helpers/activate_miniconda3.bat b/.jenkins/pytorch/win-test-helpers/installation-helpers/activate_miniconda3.bat
index e6660a17b3..b39ede82c2 100644
--- a/.jenkins/pytorch/win-test-helpers/installation-helpers/activate_miniconda3.bat
+++ b/.jenkins/pytorch/win-test-helpers/installation-helpers/activate_miniconda3.bat
@@ -13,7 +13,9 @@ if not exist %CONDA_PARENT_DIR%\Miniconda3 (
 )
 
 if "%INSTALL_FRESH_CONDA%"=="1" (
-  curl --retry 3 -k https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe --output %TMP_DIR_WIN%\Miniconda3-latest-Windows-x86_64.exe
+  if not exist %TMP_DIR_WIN%\Miniconda3-latest-Windows-x86_64.exe (
+    curl --retry 3 -k https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe --output %TMP_DIR_WIN%\Miniconda3-latest-Windows-x86_64.exe
+  )  
   if errorlevel 1 exit /b
   if not errorlevel 0 exit /b
 
diff --git a/.jenkins/pytorch/win-test-helpers/installation-helpers/install_magma.bat b/.jenkins/pytorch/win-test-helpers/installation-helpers/install_magma.bat
index d9f3ab1cf8..9a588b0a30 100644
--- a/.jenkins/pytorch/win-test-helpers/installation-helpers/install_magma.bat
+++ b/.jenkins/pytorch/win-test-helpers/installation-helpers/install_magma.bat
@@ -24,7 +24,9 @@ if "%CUDA_SUFFIX%" == "" (
 
 if "%REBUILD%"=="" (
   if "%BUILD_ENVIRONMENT%"=="" (
-    curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/magma_2.5.4_%CUDA_SUFFIX%_%BUILD_TYPE%.7z --output %TMP_DIR_WIN%\magma_2.5.4_%CUDA_SUFFIX%_%BUILD_TYPE%.7z
+    if not exist %TMP_DIR_WIN%\magma_2.5.4_%CUDA_SUFFIX%_%BUILD_TYPE%.7z (
+      curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/magma_2.5.4_%CUDA_SUFFIX%_%BUILD_TYPE%.7z --output %TMP_DIR_WIN%\magma_2.5.4_%CUDA_SUFFIX%_%BUILD_TYPE%.7z
+    )
   ) else (
     aws s3 cp s3://ossci-windows/magma_2.5.4_%CUDA_SUFFIX%_%BUILD_TYPE%.7z %TMP_DIR_WIN%\magma_2.5.4_%CUDA_SUFFIX%_%BUILD_TYPE%.7z --quiet
   )
diff --git a/.jenkins/pytorch/win-test-helpers/installation-helpers/install_mkl.bat b/.jenkins/pytorch/win-test-helpers/installation-helpers/install_mkl.bat
index c700a04a1e..e97a6be46e 100644
--- a/.jenkins/pytorch/win-test-helpers/installation-helpers/install_mkl.bat
+++ b/.jenkins/pytorch/win-test-helpers/installation-helpers/install_mkl.bat
@@ -1,6 +1,8 @@
 if "%REBUILD%"=="" (
   if "%BUILD_ENVIRONMENT%"=="" (
-    curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/mkl_2020.2.254.7z --output %TMP_DIR_WIN%\mkl.7z
+    if not exist %TMP_DIR_WIN%\mkl.7z (
+      curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/mkl_2020.2.254.7z --output %TMP_DIR_WIN%\mkl.7z
+    )
   ) else (
     aws s3 cp s3://ossci-windows/mkl_2020.2.254.7z %TMP_DIR_WIN%\mkl.7z --quiet
   )
diff --git a/.jenkins/pytorch/win-test-helpers/installation-helpers/install_sccache.bat b/.jenkins/pytorch/win-test-helpers/installation-helpers/install_sccache.bat
index 0165604400..90bbc0e765 100644
--- a/.jenkins/pytorch/win-test-helpers/installation-helpers/install_sccache.bat
+++ b/.jenkins/pytorch/win-test-helpers/installation-helpers/install_sccache.bat
@@ -7,8 +7,12 @@ if "%REBUILD%"=="" (
     del %TMP_DIR_WIN%\bin\sccache.exe || ver > nul
     del %TMP_DIR_WIN%\bin\sccache-cl.exe || ver > nul
     if "%BUILD_ENVIRONMENT%"=="" (
-      curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/sccache.exe --output %TMP_DIR_WIN%\bin\sccache.exe
-      curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/sccache-cl.exe --output %TMP_DIR_WIN%\bin\sccache-cl.exe
+      if not exist %TMP_DIR_WIN%\bin\sccache.exe (
+        curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/sccache.exe --output %TMP_DIR_WIN%\bin\sccache.exe
+      )
+      if not exist %TMP_DIR_WIN%\bin\sccache-cl.exe (
+        curl --retry 3 -k https://s3.amazonaws.com/ossci-windows/sccache-cl.exe --output %TMP_DIR_WIN%\bin\sccache-cl.exe
+      )
     ) else (
       aws s3 cp s3://ossci-windows/sccache.exe %TMP_DIR_WIN%\bin\sccache.exe
       aws s3 cp s3://ossci-windows/sccache-cl.exe %TMP_DIR_WIN%\bin\sccache-cl.exe
diff --git a/.jenkins/pytorch/win-test-helpers/setup_pytorch_env.bat b/.jenkins/pytorch/win-test-helpers/setup_pytorch_env.bat
index 79e8aedfab..5d687afd8f 100644
--- a/.jenkins/pytorch/win-test-helpers/setup_pytorch_env.bat
+++ b/.jenkins/pytorch/win-test-helpers/setup_pytorch_env.bat
@@ -22,11 +22,7 @@ if NOT "%BUILD_ENVIRONMENT%"=="" (
 )
 
 pushd .
-if "%VC_VERSION%" == "" (
-    call "C:\Program Files (x86)\Microsoft Visual Studio\%VC_YEAR%\%VC_PRODUCT%\VC\Auxiliary\Build\vcvarsall.bat" x64
-) else (
-    call "C:\Program Files (x86)\Microsoft Visual Studio\%VC_YEAR%\%VC_PRODUCT%\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%VC_VERSION%
-)
+call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
 if errorlevel 1 exit /b
 if not errorlevel 0 exit /b
 @echo on

This basically mimics how the PyTorch is built on the CI, namely Jenkins, as there are three CI configurations present in the repository sharing some build scripts: CircleCI, Jenkins, and GitLab Actions.

The only outstanding workaround that had to be done is ignoring the "Usage of PyLong_{From,As}{Unsigned}Long API may lead to overflow errors on Windows". I still need to investigate whether this is still an issue and if yes, we'd need to prepare a patch replacing the dozens of occurrences of using the old API with the new one.

Nevertheless, as the requirements are vague, there is not a straightforward way of saying when this issue could be considered as done as there are many possible combinations of enabled/disabled features, dependencies, environments etc. that might be needed to be tried and supported in the future. I'd suggest that this issue could be closed when we will setup at least one working pipeline regularly building the PyTorch with VS 2022 according to expectations on other pipelines. Another approach on considering this issue as complete could be to check and update the following scripts https://github.com/peterjc123/pytorch-scripts with VS 2022. But I am open to any other suggestions how the acceptance criteria of this issue might be defined?

Add support for VS2022 build chain #87695

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions