🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158475

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• linux.aws.h100.8 instance is down, potentially longer queue on linux.aws.h100

✅ You can merge normally! (1 Unrelated Failure)

As of commit 8bd13ea with merge base ddd74d1 ():

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorchbot merge

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchbot revert -m="Diff reverted internally" -c="ghfirst"

This Pull Request has been reverted by a revert inside Meta. To re-land this change, please open another pull request, assign the same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).)

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@albanD your PR has been successfully reverted.

Looking into disabling sync for scripts/proto.ps1 since it looks like it is used.

@pytorchbot merge

Disabling the sync is more painful than planned. So just keeping it in the meantime.

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

• Lint / lintrunner-noclang / linux-job

Dig deeper by viewing the failures on hud

@pytorchbot merge -r

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

Successfully rebased scripts_cleanup onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout scripts_cleanup && git pull --rebase)

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

• Lint / lintrunner-noclang / linux-job

Dig deeper by viewing the failures on hud

Local lint looks good. So we'll see.

@pytorchbot merge

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

❯ cmake -G Ninja -DBUILD_SHARED_LIBS:BOOL=OFF -DBUILD_STATIC_LIBS:BOOL=ON -DCMAKE_OSX_ARCHITECTURES="x86_64" -DCMAKE_OSX_DEPLOYMENT_TARGET="11" -DBUILD_PYTHON=False -DBUILD_TEST=True -DPYTHON_EXECUTABLE:PATH=`which python3` -D CMAKE_POLICY_VERSION_MINIMUM=3.5 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../pytorch-install-macOS-x86_64 -DENABLED_AVX2=OFF -DENABLED_AVX512F=OFF -DUSE_MKLDNN=OFF -DUSE_QNNPACK=OFF -DUSE_PYTORCH_QNNPACK=OFF -DBUILD_TEST=OFF -DUSE_NNPACK=OFF ..
CMake Deprecation Warning at CMakeLists.txt:9 (cmake_policy):
  The OLD behavior for policy CMP0126 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


-- The CXX compiler identification is AppleClang 17.0.0.17000013
-- The C compiler identification is AppleClang 17.0.0.17000013
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Not forcing any particular BLAS to be found
-- CLANG_VERSION_STRING:         17.0
-- sdk version: 15.5, mps supported: ON
-- MPSGraph framework found
./scripts/tmp_protoc_script.sh: line 2: ./scripts/build_host_protoc.sh: No such file or directory
CMake Error at CMakeLists.txt:821 (message):
  Could not compile universal protoc.


-- Configuring incomplete, errors occurred!

How was this submission automatically validated through CI?
On my macOS 26 (I know it's usually unreasonable to give feedback on beta systems, but please ignore this for now), when you submitted and deleted the "scripts/build_host_protoc.sh" file in the pytorch top-level directory, didn't you test the compilation of STATIC_LIBS?
Are you solely considering your own build case?

Of course, I can easily recover these files, but that's not a reason why you can submit a PR without review to delete a large number of scripts. Do you completely trust your grep and not trust the community's accumulation over so many years?

this is my build case:

cd pytorch
mkdir pytorch-build
cd pytorch-build
python3 -m venv .venv
source .venv/bin/activate
pip install numpy ninja pyyaml cffi typing_extensions six requests setuptools
cmake -G Ninja -DBUILD_SHARED_LIBS:BOOL=OFF -DBUILD_STATIC_LIBS:BOOL=ON -DCMAKE_OSX_ARCHITECTURES="x86_64" -DCMAKE_OSX_DEPLOYMENT_TARGET="11" -DBUILD_PYTHON=False -DBUILD_TEST=True -DPYTHON_EXECUTABLE:PATH=`which python3` -D CMAKE_POLICY_VERSION_MINIMUM=3.5 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../pytorch-install-macOS-x86_64 -DENABLED_AVX2=OFF -DENABLED_AVX512F=OFF -DUSE_MKLDNN=OFF -DUSE_QNNPACK=OFF -DUSE_PYTORCH_QNNPACK=OFF -DBUILD_TEST=OFF -DUSE_NNPACK=OFF ..

cmake --build . --target install -j10

about build_host_protoc can see this file : CMakeLists.txt line 795-827

# The below means we are cross compiling for arm64 or x86_64 on MacOSX
if(NOT IOS
   AND CMAKE_SYSTEM_NAME STREQUAL "Darwin"
   AND CMAKE_OSX_ARCHITECTURES MATCHES "^(x86_64|arm64)$")
  set(CROSS_COMPILING_MACOSX TRUE)
  # We need to compile a universal protoc to not fail protobuf build We set
  # CMAKE_TRY_COMPILE_TARGET_TYPE to STATIC_LIBRARY (vs executable) to succeed
  # the cmake compiler check for cross-compiling
  set(protoc_build_command
      "./scripts/build_host_protoc.sh --other-flags -DCMAKE_OSX_ARCHITECTURES=\"x86_64;arm64\" -DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY -DCMAKE_C_COMPILER_WORKS=1 -DCMAKE_CXX_COMPILER_WORKS=1"
  )
  # We write to a temp scriptfile because CMake COMMAND dislikes double quotes
  # in commands
  file(WRITE ${PROJECT_SOURCE_DIR}/tmp_protoc_script.sh
       "#!/bin/bash\n${protoc_build_command}")
  file(
    COPY ${PROJECT_SOURCE_DIR}/tmp_protoc_script.sh
    DESTINATION ${PROJECT_SOURCE_DIR}/scripts/
    FILE_PERMISSIONS OWNER_EXECUTE OWNER_WRITE OWNER_READ)
  execute_process(
    COMMAND ./scripts/tmp_protoc_script.sh
    WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
    RESULT_VARIABLE BUILD_HOST_PROTOC_RESULT)
  file(REMOVE ${PROJECT_SOURCE_DIR}/tmp_protoc_script.sh
       ${PROJECT_SOURCE_DIR}/scripts/tmp_protoc_script.sh)
  if(NOT BUILD_HOST_PROTOC_RESULT EQUAL "0")
    message(FATAL_ERROR "Could not compile universal protoc.")
  endif()
  set(PROTOBUF_PROTOC_EXECUTABLE
      "${PROJECT_SOURCE_DIR}/build_host_protoc/bin/protoc")
  set(CAFFE2_CUSTOM_PROTOC_EXECUTABLE
      "${PROJECT_SOURCE_DIR}/build_host_protoc/bin/protoc")
endif()

Sorry about that @QiuChenly !
Adding it back in #159157
And follow up on protoc state in #159156

If you heavily rely on macos cross compilation, would you be interested in maintaining a CI workflow that runs it to ensure it doesn't get broken? We have not been able to find anyone willing to maintain such a workflow :/
If so, do open an issue and we can discuss there the best way to add such workflow!

Thanks for your help!