Skip to content

reshape in python#84584

Closed
Krovatkin wants to merge 5 commits intopytorch:symbolic-shapesfrom
Krovatkin:krovatkin/symbolic-shapes
Closed

reshape in python#84584
Krovatkin wants to merge 5 commits intopytorch:symbolic-shapesfrom
Krovatkin:krovatkin/symbolic-shapes

Conversation

@Krovatkin
Copy link
Copy Markdown
Contributor

Fixes #ISSUE_NUMBER

@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Sep 6, 2022

🔗 Helpful links

❌ 37 New Failures

As of commit 44221ef (more details on the Dr. CI page):

Expand to see more
  • 37/37 failures introduced in this PR

🕵️ 35 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages

See GitHub Actions build pull / linux-focal-py3.7-gcc7 / test (functorch, 1, 1, linux.2xlarge) (1/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:46:17.3177536Z FAIL [3.111s]: tes...rg_as_static (__main__.TestCompileCacheStaticArgs)
2022-09-06T18:46:17.3174891Z     aot_config,
2022-09-06T18:46:17.3175161Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 452, in create_aot_dispatcher_function
2022-09-06T18:46:17.3175492Z     aot_dispatch_autograd(flat_fn, fake_flat_tensor_args, aot_config)
2022-09-06T18:46:17.3175873Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 328, in aot_dispatch_autograd
2022-09-06T18:46:17.3176189Z     fw_module, bw_module = aot_config.partition_fn(fx_g, joint_inputs)
2022-09-06T18:46:17.3176523Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/partitioners.py", line 157, in default_partition
2022-09-06T18:46:17.3176848Z     assert all(user.target == operator.getitem for user in users)
2022-09-06T18:46:17.3177071Z AssertionError
2022-09-06T18:46:17.3177166Z 
2022-09-06T18:46:17.3177259Z ======================================================================
2022-09-06T18:46:17.3177536Z FAIL [3.111s]: test_tuple_with_first_arg_as_static (__main__.TestCompileCacheStaticArgs)
2022-09-06T18:46:17.3177917Z ----------------------------------------------------------------------
2022-09-06T18:46:17.3178155Z Traceback (most recent call last):
2022-09-06T18:46:17.3178463Z   File "/var/lib/jenkins/workspace/functorch/test/test_compile_cache.py", line 477, in test_tuple_with_first_arg_as_static
2022-09-06T18:46:17.3178754Z     check(a, b, aot_autograd_f, fn)
2022-09-06T18:46:17.3179013Z   File "/var/lib/jenkins/workspace/functorch/test/test_compile_cache.py", line 460, in check
2022-09-06T18:46:17.3179287Z     res = aot_autograd_fn(a, (b0_clone, b1_clone))
2022-09-06T18:46:17.3179588Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 721, in returned_function
2022-09-06T18:46:17.3179846Z     aot_config,
2022-09-06T18:46:17.3180115Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 452, in create_aot_dispatcher_function
2022-09-06T18:46:17.3180447Z     aot_dispatch_autograd(flat_fn, fake_flat_tensor_args, aot_config)

See GitHub Actions build pull / win-vs2019-cpu-py3 / test (default, 1, 2, windows.4xlarge) (2/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:09:12.5883019Z RuntimeError: test_ops failed!
2022-09-06T19:09:11.5658730Z FAILED test_ops.py::TestCommonCPU::test_out_matmul_cpu_float32
2022-09-06T19:09:11.5659027Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!
2022-09-06T19:09:11.5659341Z !!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 1 failures !!!!!!!!!!!!
2022-09-06T19:09:11.5659676Z = 1 failed, 2359 passed, 2461 skipped, 28 xfailed, 86 warnings, 2 rerun in 64.93s (0:01:04) =
2022-09-06T19:09:11.5660029Z Skip info is located in the xml test reports, please either go to s3 or the hud to download them
2022-09-06T19:09:12.5640391Z Traceback (most recent call last):
2022-09-06T19:09:12.5648489Z   File "C:\actions-runner\_work\pytorch\pytorch\test\run_test.py", line 1104, in <module>
2022-09-06T19:09:12.5878523Z     main()
2022-09-06T19:09:12.5878929Z   File "C:\actions-runner\_work\pytorch\pytorch\test\run_test.py", line 1082, in main
2022-09-06T19:09:12.5882751Z     raise RuntimeError(err_message)
2022-09-06T19:09:12.5883019Z RuntimeError: test_ops failed!
2022-09-06T19:09:12.8028615Z 
2022-09-06T19:09:12.8029070Z (base) C:\actions-runner\_work\pytorch\pytorch\test>if ERRORLEVEL 1 goto fail 
2022-09-06T19:09:12.8031458Z 
2022-09-06T19:09:12.8031690Z (base) C:\actions-runner\_work\pytorch\pytorch\test>exit /b 1 
2022-09-06T19:09:12.8095286Z ##[error]Process completed with exit code 1.
2022-09-06T19:09:12.8563668Z Prepare all required actions
2022-09-06T19:09:12.8564229Z Getting action download info
2022-09-06T19:09:12.9927836Z Download action repository 'nick-fields/retry@7d4a37704547a311dbb66ebdf5b23ec19374a767' (SHA:7d4a37704547a311dbb66ebdf5b23ec19374a767)
2022-09-06T19:09:13.6919863Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T19:09:13.6920080Z with:

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) (3/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:39:37.7261278Z AssertionError: Return code should be an integer
2022-09-06T19:39:37.4817340Z AssertionError: 2 unit test(s) failed:
2022-09-06T19:39:37.4817781Z 	TestDistBackendWithSpawn.test_DistributedDataParallel_SyncBatchNorm_Channels_Last
2022-09-06T19:39:37.4818230Z 	TestDistBackendWithSpawn.test_ddp_uneven_inputs
2022-09-06T19:39:37.7255099Z Traceback (most recent call last):
2022-09-06T19:39:37.7255551Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1104, in <module>
2022-09-06T19:39:37.7256631Z     main()
2022-09-06T19:39:37.7256939Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1076, in main
2022-09-06T19:39:37.7259076Z     err_message = run_test_module(test, test_directory, options_clone)
2022-09-06T19:39:37.7259790Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1034, in run_test_module
2022-09-06T19:39:37.7260881Z     assert isinstance(return_code, int) and not isinstance(
2022-09-06T19:39:37.7261278Z AssertionError: Return code should be an integer
2022-09-06T19:39:38.0503775Z 
2022-09-06T19:39:38.0504537Z real	53m55.175s
2022-09-06T19:39:38.0504849Z user	83m21.489s
2022-09-06T19:39:38.0505094Z sys	50m53.985s
2022-09-06T19:39:38.0554638Z ##[error]Process completed with exit code 1.
2022-09-06T19:39:38.0600157Z Prepare all required actions
2022-09-06T19:39:38.0600508Z Getting action download info
2022-09-06T19:39:38.2740680Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T19:39:38.2740983Z with:
2022-09-06T19:39:38.2741442Z   github-token: ***

See GitHub Actions build pull / linux-bionic-py3.7-clang9 / test (dynamo, 1, 2, linux.2xlarge) (4/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:30:50.7708933Z TypeError: len() of a 0-d tensor
2022-09-06T18:30:48.9961724Z + pushd .jenkins/pytorch
2022-09-06T18:30:48.9962053Z ~/workspace/.jenkins/pytorch ~/workspace
2022-09-06T18:30:48.9963254Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())'
2022-09-06T18:30:49.8223340Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:30:49.9670401Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');import torch;print(torch.tensor([torch.tensor(0.), torch.tensor(1.)]))'
2022-09-06T18:30:50.7705539Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:30:50.7705943Z Traceback (most recent call last):
2022-09-06T18:30:50.7706161Z   File "<string>", line 1, in <module>
2022-09-06T18:30:50.7706523Z   File "/opt/conda/lib/python3.7/site-packages/torch/_tensor.py", line 880, in __len__
2022-09-06T18:30:50.7708662Z     raise TypeError("len() of a 0-d tensor")
2022-09-06T18:30:50.7708933Z TypeError: len() of a 0-d tensor
2022-09-06T18:30:50.9175073Z ##[error]Process completed with exit code 1.
2022-09-06T18:30:50.9215352Z Prepare all required actions
2022-09-06T18:30:50.9215659Z Getting action download info
2022-09-06T18:30:51.0856289Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:30:51.0856505Z with:
2022-09-06T18:30:51.0856837Z   github-token: ***
2022-09-06T18:30:51.0856990Z env:
2022-09-06T18:30:51.0857161Z   GIT_DEFAULT_BRANCH: master
2022-09-06T18:30:51.0857344Z ##[endgroup]
2022-09-06T18:30:51.0883145Z ##[group]Run nick-fields/retry@7d4a37704547a311dbb66ebdf5b23ec19374a767

See GitHub Actions build pull / linux-bionic-py3.7-clang9 / test (crossref, 1, 2, linux.2xlarge) (5/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:30:59.2463422Z TypeError: len() of a 0-d tensor
2022-09-06T18:30:57.4547473Z + pushd .jenkins/pytorch
2022-09-06T18:30:57.4547731Z ~/workspace/.jenkins/pytorch ~/workspace
2022-09-06T18:30:57.4548286Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())'
2022-09-06T18:30:58.2862556Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:30:58.4334938Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');import torch;print(torch.tensor([torch.tensor(0.), torch.tensor(1.)]))'
2022-09-06T18:30:59.2460197Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:30:59.2460927Z Traceback (most recent call last):
2022-09-06T18:30:59.2461286Z   File "<string>", line 1, in <module>
2022-09-06T18:30:59.2461837Z   File "/opt/conda/lib/python3.7/site-packages/torch/_tensor.py", line 880, in __len__
2022-09-06T18:30:59.2462937Z     raise TypeError("len() of a 0-d tensor")
2022-09-06T18:30:59.2463422Z TypeError: len() of a 0-d tensor
2022-09-06T18:30:59.4103756Z ##[error]Process completed with exit code 1.
2022-09-06T18:30:59.4142699Z Prepare all required actions
2022-09-06T18:30:59.4143009Z Getting action download info
2022-09-06T18:30:59.5763936Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:30:59.5764158Z with:
2022-09-06T18:30:59.5764478Z   github-token: ***
2022-09-06T18:30:59.5764899Z env:
2022-09-06T18:30:59.5765070Z   GIT_DEFAULT_BRANCH: master
2022-09-06T18:30:59.5765256Z ##[endgroup]
2022-09-06T18:30:59.5791264Z ##[group]Run nick-fields/retry@7d4a37704547a311dbb66ebdf5b23ec19374a767

See GitHub Actions build pull / linux-focal-py3.7-clang7-asan / test (default, 4, 5, linux.2xlarge) (6/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:37:54.8646697Z RuntimeError: test_ops failed!
2022-09-06T18:37:53.3603933Z FAILED test_ops.py::TestCommonCPU::test_out_matmul_cpu_float32
2022-09-06T18:37:53.3607021Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
2022-09-06T18:37:53.3607919Z !!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 1 failures !!!!!!!!!!!!!
2022-09-06T18:37:53.3622776Z = 1 failed, 779 passed, 3658 skipped, 15 xfailed, 63 warnings, 2 rerun in 81.18s (0:01:21) =
2022-09-06T18:37:53.4918262Z Skip info is located in the xml test reports, please either go to s3 or the hud to download them
2022-09-06T18:37:54.8638015Z Traceback (most recent call last):
2022-09-06T18:37:54.8638305Z   File "test/run_test.py", line 1104, in <module>
2022-09-06T18:37:54.8642252Z     main()
2022-09-06T18:37:54.8642509Z   File "test/run_test.py", line 1082, in main
2022-09-06T18:37:54.8646174Z     raise RuntimeError(err_message)
2022-09-06T18:37:54.8646697Z RuntimeError: test_ops failed!
2022-09-06T18:37:55.5027627Z 
2022-09-06T18:37:55.5028009Z real	1m34.475s
2022-09-06T18:37:55.5028363Z user	2m39.040s
2022-09-06T18:37:55.5028573Z sys	0m11.452s
2022-09-06T18:37:55.5066844Z ##[error]Process completed with exit code 1.
2022-09-06T18:37:55.5125802Z Prepare all required actions
2022-09-06T18:37:55.5126127Z Getting action download info
2022-09-06T18:37:55.6971787Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:37:55.6972005Z with:
2022-09-06T18:37:55.6972345Z   github-token: ***

See GitHub Actions build pull / linux-focal-py3.7-gcc7 / test (default, 2, 2, linux.2xlarge) (7/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:47:10.1871163Z FAIL [0.059s]: tes...ag_cpu_float32 (__main__.TestProxyTensorOpInfoCPU)
2022-09-06T18:47:10.1868191Z     batch_dim, channel_dim * kernel_h * kernel_w, num_blocks_row * num_blocks_col
2022-09-06T18:47:10.1868653Z   File "/opt/conda/lib/python3.7/site-packages/torch/utils/_python_dispatch.py", line 74, in wrapped
2022-09-06T18:47:10.1868927Z     return f(self, *args, **kwargs)
2022-09-06T18:47:10.1869296Z   File "/opt/conda/lib/python3.7/site-packages/torch/_subclasses/fake_tensor.py", line 718, in __torch_dispatch__
2022-09-06T18:47:10.1869586Z     r = meta_table[func](*args, **kwargs)
2022-09-06T18:47:10.1869950Z   File "/opt/conda/lib/python3.7/site-packages/torch/_meta_registrations.py", line 98, in reshape
2022-09-06T18:47:10.1870252Z     stride = computeStride(self.sizes(), self.strides(), shape)
2022-09-06T18:47:10.1870584Z AttributeError: 'FakeTensor' object has no attribute 'sizes'
2022-09-06T18:47:10.1870743Z 
2022-09-06T18:47:10.1870841Z ======================================================================
2022-09-06T18:47:10.1871163Z FAIL [0.059s]: test_make_fx_fake_exhaustive_nn_functional_embedding_bag_cpu_float32 (__main__.TestProxyTensorOpInfoCPU)
2022-09-06T18:47:10.1871563Z ----------------------------------------------------------------------
2022-09-06T18:47:10.1871825Z Traceback (most recent call last):
2022-09-06T18:47:10.1872235Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 378, in instantiated_test
2022-09-06T18:47:10.1872526Z     result = test(self, **param_kwargs)
2022-09-06T18:47:10.1872926Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 815, in test_wrapper
2022-09-06T18:47:10.1873211Z     return test(*args, **kwargs)
2022-09-06T18:47:10.1873458Z   File "test_proxy_tensor.py", line 1343, in test_make_fx_fake_exhaustive
2022-09-06T18:47:10.1873717Z     _test_make_fx_helper(self, device, dtype, op, "fake")
2022-09-06T18:47:10.1873978Z   File "test_proxy_tensor.py", line 1332, in _test_make_fx_helper
2022-09-06T18:47:10.1874217Z     self.assertEqual(new_out, old_out)

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) (8/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:00:47.7853014Z AssertionError: 2 unit test(s) failed:
2022-09-06T19:00:47.3570144Z 
2022-09-06T19:00:47.3570246Z OK
2022-09-06T19:00:47.3570452Z 
2022-09-06T19:00:47.3570602Z Generating XML reports...
2022-09-06T19:00:47.3609003Z Generated XML report: test-reports/python-unittest/distributed.test_c10d_nccl/TEST-TimeoutTest-20220906190042.xml
2022-09-06T19:00:47.7847220Z Traceback (most recent call last):
2022-09-06T19:00:47.7847726Z   File "/var/lib/jenkins/workspace/test/distributed/test_c10d_nccl.py", line 2830, in <module>
2022-09-06T19:00:47.7851071Z     run_tests()
2022-09-06T19:00:47.7851770Z   File "/opt/conda/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 708, in run_tests
2022-09-06T19:00:47.7852576Z     assert len(failed_tests) == 0, "{} unit test(s) failed:\n\t{}".format(
2022-09-06T19:00:47.7853014Z AssertionError: 2 unit test(s) failed:
2022-09-06T19:00:47.7853329Z 	CommTest.test_pass_nccl_options_high_priority_stream
2022-09-06T19:00:47.7854017Z 	ProcessGroupNCCLTest.test_reduce_scatter_ops
2022-09-06T19:00:48.0609580Z Traceback (most recent call last):
2022-09-06T19:00:48.0610028Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1104, in <module>
2022-09-06T19:00:48.0610790Z     main()
2022-09-06T19:00:48.0611154Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1082, in main
2022-09-06T19:00:48.0613070Z     raise RuntimeError(err_message)
2022-09-06T19:00:48.0613427Z RuntimeError: distributed/test_c10d_nccl failed!
2022-09-06T19:00:48.3860054Z 
2022-09-06T19:00:48.3861049Z real	15m2.328s

See GitHub Actions build pull / linux-focal-py3.7-clang10-onnx / test (default, 1, 2, linux.2xlarge) (9/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:33:10.1790334Z what():
2022-09-06T18:33:09.9025810Z 
2022-09-06T18:33:09.9026065Z �[0;32m[----------] �[mGlobal test environment tear-down
2022-09-06T18:33:09.9026564Z �[0;32m[==========] �[m2 tests from 1 test suite ran. (1 ms total)
2022-09-06T18:33:09.9026989Z �[0;32m[  PASSED  ] �[m2 tests.
2022-09-06T18:33:09.9555498Z + for test in $(find "$cpp_test_dir" -executable -type f)
2022-09-06T18:33:09.9555792Z + case "$test" in
2022-09-06T18:33:09.9557154Z ++ basename /opt/conda/lib/python3.7/site-packages/torch/test/extension_backend_test
2022-09-06T18:33:09.9563907Z + LD_LIBRARY_PATH=/opt/conda/lib/python3.7/site-packages/torch/lib
2022-09-06T18:33:09.9564771Z + /opt/conda/lib/python3.7/site-packages/torch/test/extension_backend_test --gtest_output=xml:/var/lib/jenkins/workspace/test/test-reports/cpp/extension_backend_test.xml
2022-09-06T18:33:10.1789831Z terminate called after throwing an instance of 'c10::Error'
2022-09-06T18:33:10.1790334Z   what():  
2022-09-06T18:33:10.1790595Z Mismatch in kernel C++ signatures
2022-09-06T18:33:10.1791095Z   operator: aten::empty_strided(int[] size, int[] stride, *, int? dtype=None, int? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
2022-09-06T18:33:10.1791484Z     registered at /var/lib/jenkins/workspace/build/aten/src/ATen/RegisterSchema.cpp:6
2022-09-06T18:33:10.1791911Z   kernel 1: at::Tensor (c10::SymIntArrayRef, c10::SymIntArrayRef, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>)
2022-09-06T18:33:10.1792251Z     dispatch key: BackendSelect
2022-09-06T18:33:10.1792553Z     registered at /var/lib/jenkins/workspace/build/aten/src/ATen/RegisterBackendSelect.cpp:726
2022-09-06T18:33:10.1792959Z   kernel 2: at::Tensor (c10::ArrayRef<long>, c10::ArrayRef<long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>)
2022-09-06T18:33:10.1793277Z     dispatch key: ORT
2022-09-06T18:33:10.1793556Z     registered at /var/lib/jenkins/workspace/aten/src/ATen/test/extension_backend_test.cpp:50
2022-09-06T18:33:10.1793754Z 

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (functorch, 1, 1, linux.4xlarge.nvidia.gpu) (10/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:08:08.6913477Z FAIL [3.992s]: tes...rg_as_static (__main__.TestCompileCacheStaticArgs)
2022-09-06T19:08:08.6909663Z     compiled_fn = create_aot_dispatcher_function(
2022-09-06T19:08:08.6910104Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 452, in create_aot_dispatcher_function
2022-09-06T19:08:08.6910539Z     aot_dispatch_autograd(flat_fn, fake_flat_tensor_args, aot_config)
2022-09-06T19:08:08.6910991Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 328, in aot_dispatch_autograd
2022-09-06T19:08:08.6911437Z     fw_module, bw_module = aot_config.partition_fn(fx_g, joint_inputs)
2022-09-06T19:08:08.6912077Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/partitioners.py", line 157, in default_partition
2022-09-06T19:08:08.6912499Z     assert all(user.target == operator.getitem for user in users)
2022-09-06T19:08:08.6912819Z AssertionError
2022-09-06T19:08:08.6912976Z 
2022-09-06T19:08:08.6913114Z ======================================================================
2022-09-06T19:08:08.6913477Z FAIL [3.992s]: test_tuple_with_first_arg_as_static (__main__.TestCompileCacheStaticArgs)
2022-09-06T19:08:08.6913993Z ----------------------------------------------------------------------
2022-09-06T19:08:08.6914343Z Traceback (most recent call last):
2022-09-06T19:08:08.6914754Z   File "/var/lib/jenkins/workspace/functorch/test/test_compile_cache.py", line 477, in test_tuple_with_first_arg_as_static
2022-09-06T19:08:08.6915152Z     check(a, b, aot_autograd_f, fn)
2022-09-06T19:08:08.6915528Z   File "/var/lib/jenkins/workspace/functorch/test/test_compile_cache.py", line 460, in check
2022-09-06T19:08:08.6915888Z     res = aot_autograd_fn(a, (b0_clone, b1_clone))
2022-09-06T19:08:08.6916299Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 718, in returned_function
2022-09-06T19:08:08.6916713Z     compiled_fn = create_aot_dispatcher_function(
2022-09-06T19:08:08.6917150Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 452, in create_aot_dispatcher_function
2022-09-06T19:08:08.6917584Z     aot_dispatch_autograd(flat_fn, fake_flat_tensor_args, aot_config)

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7-bazel-test / build-and-test (11/35)

Step: "Build" (full log | diagnosis details)

2022-09-06T18:55:33.4847846Z �[0m�[1m�[31mERROR...c1plus: some warnings being treated as errors\n" }
2022-09-06T18:55:33.3745317Z �[32mINFO: �[0mElapsed time: 2089.327s, Critical Path: 291.67s
2022-09-06T18:55:33.3745828Z �[32mINFO: �[0m3397 processes: 416 internal, 1 local, 2980 processwrapper-sandbox.
2022-09-06T18:55:33.3746183Z �[31m�[1mFAILED:�[0m Build did NOT complete successfully
2022-09-06T18:55:33.3787951Z �[31m�[1mFAILED:�[0m Build did NOT complete successfully
2022-09-06T18:55:33.3894414Z �[0m+ sccache_epilogue
2022-09-06T18:55:33.3894997Z + echo '::group::Sccache Compilation Log'
2022-09-06T18:55:33.3895977Z ##[group]Sccache Compilation Log
2022-09-06T18:55:33.3896522Z + echo '=================== sccache compilation log ==================='
2022-09-06T18:55:33.3896943Z =================== sccache compilation log ===================
2022-09-06T18:55:33.3898926Z + python /var/lib/jenkins/workspace/.jenkins/pytorch/print_sccache_log.py /var/lib/jenkins/sccache_error.log
2022-09-06T18:55:33.4847846Z �[0m�[1m�[31mERROR�[0m 2022-09-06T18:55:31Z: sccache::server: Compilation failed: Output { status: ExitStatus(unix_wait_status(256)), stdout: "", stderr: "torch/csrc/lazy/core/tensor_impl.cpp: In member function 'void torch::lazy::LTCTensorImpl::setup_sym_sizes() const':\ntorch/csrc/lazy/core/tensor_impl.cpp:163:35: error: comparison of unsigned expression >= 0 is always true [-Werror=type-limits]\n     for (auto i = last_idx - 1; i >= 0; --i) {\n                                 ~~^~~~\ncc1plus: some warnings being treated as errors\n" }
2022-09-06T18:55:33.4848664Z 
2022-09-06T18:55:33.4849247Z �[0m�[1m�[31mERROR�[0m 2022-09-06T18:55:32Z: sccache::server: Compilation failed: Output { status: ExitStatus(unix_wait_status(256)), stdout: "", stderr: "Assembler messages:\nFatal error: can't create bazel-out/k8-fastbuild/bin/_objs/torch/5/Utils.pic.o: No such file or directory\n" }
2022-09-06T18:55:33.4849616Z 
2022-09-06T18:55:33.4879860Z + echo '=========== If your build fails, please take a look at the log above for possible reasons ==========='
2022-09-06T18:55:33.4880497Z =========== If your build fails, please take a look at the log above for possible reasons ===========
2022-09-06T18:55:33.4880808Z + sccache --show-stats
2022-09-06T18:55:33.4906380Z Compile requests                   3480
2022-09-06T18:55:33.4906733Z Compile requests executed          2758
2022-09-06T18:55:33.4907004Z Cache hits                         2170
2022-09-06T18:55:33.4907227Z Cache hits (C/C++)                 2170

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (distributed, 2, 3, linux.8xlarge.nvidia.gpu) (12/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:31:04.2532125Z RuntimeError: dist...d/algorithms/ddp_comm_hooks/test_ddp_hooks failed!
2022-09-06T19:31:03.8949410Z 
2022-09-06T19:31:03.8949554Z FAILED (errors=1, expected failures=3)
2022-09-06T19:31:03.8949853Z 
2022-09-06T19:31:03.8949971Z Generating XML reports...
2022-09-06T19:31:03.8984844Z Generated XML report: test-reports/python-unittest/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks/TEST-DistributedDataParallelCommHookTest-20220906193019.xml
2022-09-06T19:31:04.2525840Z Traceback (most recent call last):
2022-09-06T19:31:04.2526254Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1104, in <module>
2022-09-06T19:31:04.2528792Z     main()
2022-09-06T19:31:04.2529148Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1082, in main
2022-09-06T19:31:04.2531716Z     raise RuntimeError(err_message)
2022-09-06T19:31:04.2532125Z RuntimeError: distributed/algorithms/ddp_comm_hooks/test_ddp_hooks failed!
2022-09-06T19:31:04.5597016Z 
2022-09-06T19:31:04.5597675Z real	44m42.831s
2022-09-06T19:31:04.5597962Z user	93m34.002s
2022-09-06T19:31:04.5598201Z sys	53m19.022s
2022-09-06T19:31:04.5647791Z ##[error]Process completed with exit code 1.
2022-09-06T19:31:04.5692958Z Prepare all required actions
2022-09-06T19:31:04.5693437Z Getting action download info
2022-09-06T19:31:04.8142640Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T19:31:04.8142943Z with:
2022-09-06T19:31:04.8143404Z   github-token: ***

See GitHub Actions build pull / linux-focal-py3.7-clang7-asan / test (default, 2, 5, linux.2xlarge) (13/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:38:39.4627012Z RuntimeError: test_ops_gradients failed!
2022-09-06T18:38:38.3813653Z FAILED test_ops_gradients.py::TestGradientsCPU::test_fn_fwgrad_bwgrad___getitem___cpu_float64
2022-09-06T18:38:38.3816683Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
2022-09-06T18:38:38.3818569Z !!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 1 failures !!!!!!!!!!!!!
2022-09-06T18:38:38.3824128Z ======= 1 failed, 13 passed, 924 skipped, 33 warnings, 3 rerun in 28.15s =======
2022-09-06T18:38:38.4074680Z Skip info is located in the xml test reports, please either go to s3 or the hud to download them
2022-09-06T18:38:39.4618077Z Traceback (most recent call last):
2022-09-06T18:38:39.4618409Z   File "test/run_test.py", line 1104, in <module>
2022-09-06T18:38:39.4622210Z     main()
2022-09-06T18:38:39.4622457Z   File "test/run_test.py", line 1082, in main
2022-09-06T18:38:39.4626518Z     raise RuntimeError(err_message)
2022-09-06T18:38:39.4627012Z RuntimeError: test_ops_gradients failed!
2022-09-06T18:38:40.1589717Z 
2022-09-06T18:38:40.1590038Z real	0m40.851s
2022-09-06T18:38:40.1590404Z user	1m14.118s
2022-09-06T18:38:40.1590601Z sys	0m13.691s
2022-09-06T18:38:40.1634394Z ##[error]Process completed with exit code 1.
2022-09-06T18:38:40.1691107Z Prepare all required actions
2022-09-06T18:38:40.1691461Z Getting action download info
2022-09-06T18:38:40.3427328Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:38:40.3427559Z with:
2022-09-06T18:38:40.3427888Z   github-token: ***

See GitHub Actions build pull / linux-focal-py3.7-gcc7 / test (distributed, 1, 1, linux.2xlarge) (14/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:35:32.5735889Z AssertionError: 8 unit test(s) failed:
2022-09-06T19:35:32.3029048Z 
2022-09-06T19:35:32.3029108Z OK (skipped=1)
2022-09-06T19:35:32.3029218Z 
2022-09-06T19:35:32.3029303Z Generating XML reports...
2022-09-06T19:35:32.3051849Z Generated XML report: test-reports/python-unittest/distributed.test_c10d_gloo/TEST-TimeoutTest-20220906193532.xml
2022-09-06T19:35:32.5729357Z Traceback (most recent call last):
2022-09-06T19:35:32.5729684Z   File "distributed/test_c10d_gloo.py", line 2370, in <module>
2022-09-06T19:35:32.5733141Z     run_tests()
2022-09-06T19:35:32.5733686Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 709, in run_tests
2022-09-06T19:35:32.5735554Z     len(failed_tests), '\n\t'.join(failed_tests))
2022-09-06T19:35:32.5735889Z AssertionError: 8 unit test(s) failed:
2022-09-06T19:35:32.5736146Z 	ProcessGroupGlooTest.test_allreduce_basics
2022-09-06T19:35:32.5736430Z 	ProcessGroupGlooTest.test_allreduce_basics_using_work_api
2022-09-06T19:35:32.5736697Z 	ProcessGroupGlooTest.test_allreduce_stress
2022-09-06T19:35:32.5736971Z 	ProcessGroupGlooTest.test_barrier_implies_wait
2022-09-06T19:35:32.5737246Z 	ProcessGroupGlooTest.test_round_robin_create_destroy
2022-09-06T19:35:32.5737482Z 	ReducerTest.test_forward_backward
2022-09-06T19:35:32.5737721Z 	ReducerTest.test_forward_backward_optimizer
2022-09-06T19:35:32.5738199Z 	ReducerTest.test_forward_backward_unused_parameters
2022-09-06T19:35:32.8310127Z Traceback (most recent call last):
2022-09-06T19:35:32.8310395Z   File "test/run_test.py", line 1104, in <module>

See GitHub Actions build pull / linux-focal-py3.7-clang7-asan / test (default, 1, 5, linux.2xlarge) (15/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:35:45.9310885Z SUMMARY: Undefined.../jenkins/workspace/aten/src/ATen/Utils.cpp:20:3 in
2022-09-06T18:35:45.9291233Z     #10 0x556acf072c81 in run_mod /home/builder/tkoch/workspace/python_1648536129212/work/Python/pythonrun.c:1037
2022-09-06T18:35:45.9291716Z     #11 0x556acf07dc69 in PyRun_StringFlags /home/builder/tkoch/workspace/python_1648536129212/work/Python/pythonrun.c:961
2022-09-06T18:35:45.9292688Z     #12 0x556acf07dccb in PyRun_SimpleStringFlags /home/builder/tkoch/workspace/python_1648536129212/work/Python/pythonrun.c:455
2022-09-06T18:35:45.9293671Z     #13 0x556acf07ddc8 in pymain_run_command /home/builder/tkoch/workspace/python_1648536129212/work/Modules/main.c:420
2022-09-06T18:35:45.9294040Z     #14 0x556acf07ddc8 in pymain_run_python /home/builder/tkoch/workspace/python_1648536129212/work/Modules/main.c:2907
2022-09-06T18:35:45.9294396Z     #15 0x556acf07ddc8 in pymain_main /home/builder/tkoch/workspace/python_1648536129212/work/Modules/main.c:3460
2022-09-06T18:35:45.9295210Z     #16 0x556acf07e18b in _Py_UnixMain /home/builder/tkoch/workspace/python_1648536129212/work/Modules/main.c:3495
2022-09-06T18:35:45.9309389Z     #17 0x7fa161e27082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082)
2022-09-06T18:35:45.9309703Z     #18 0x556acf023039 in _start (/opt/conda/bin/python3.7+0x1d8039)
2022-09-06T18:35:45.9309863Z 
2022-09-06T18:35:45.9310885Z SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/jenkins/workspace/aten/src/ATen/Utils.cpp:20:3 in 
2022-09-06T18:35:45.9719497Z + retcode=1
2022-09-06T18:35:45.9720080Z + set -e
2022-09-06T18:35:45.9720297Z + return 1
2022-09-06T18:35:45.9721507Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]]
2022-09-06T18:35:45.9721751Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]]
2022-09-06T18:35:45.9726158Z + [[ linux-focal-py3.7-clang7-asan == *tbb* ]]
2022-09-06T18:35:45.9737804Z + [[ linux-focal-py3.7-clang7-asan == *libtorch* ]]
2022-09-06T18:35:45.9738144Z + [[ linux-focal-py3.7-clang7-asan == *-bazel-* ]]
2022-09-06T18:35:45.9740504Z + cd test
2022-09-06T18:35:45.9741260Z + python -c 'import torch; print(torch.__config__.show())'

See GitHub Actions build pull / linux-focal-py3.7-gcc7-no-ops / build (16/35)

Step: "Build" (full log | diagnosis details)

2022-09-06T18:21:52.1817838Z ##[error]Process completed with exit code 1.
2022-09-06T18:21:52.1757563Z Average cache read hit             0.041 s
2022-09-06T18:21:52.1757791Z Failed distributed compilations        0
2022-09-06T18:21:52.1757929Z 
2022-09-06T18:21:52.1758040Z Non-cacheable reasons:
2022-09-06T18:21:52.1758230Z multiple input files                  15
2022-09-06T18:21:52.1758441Z unknown source language                1
2022-09-06T18:21:52.1758567Z 
2022-09-06T18:21:52.1758991Z Cache location                  S3, bucket: Bucket(name=ossci-compiler-cache-circleci-v2, base_url=http://ossci-compiler-cache-circleci-v2.s3.amazonaws.com/)
2022-09-06T18:21:52.1759407Z + echo ::endgroup::
2022-09-06T18:21:52.1759817Z ##[endgroup]
2022-09-06T18:21:52.1817838Z ##[error]Process completed with exit code 1.
2022-09-06T18:21:52.1861630Z Prepare all required actions
2022-09-06T18:21:52.1881703Z ##[group]Run ./.github/actions/teardown-linux
2022-09-06T18:21:52.1881908Z with:
2022-09-06T18:21:52.1882054Z ##[endgroup]
2022-09-06T18:21:52.1896389Z ##[group]Run set -eou pipefail
2022-09-06T18:21:52.1896625Z �[36;1mset -eou pipefail�[0m
2022-09-06T18:21:52.1896809Z �[36;1m�[0m
2022-09-06T18:21:52.1897179Z �[36;1mecho "Holding runner for 2 hours until all ssh sessions have logged out"�[0m
2022-09-06T18:21:52.1897459Z �[36;1mfor _ in $(seq 1440); do�[0m
2022-09-06T18:21:52.1897680Z �[36;1m    # Break if no ssh session exists anymore�[0m

See GitHub Actions build pull / linux-bionic-cuda11_6-py3_10-gcc7-deploy / test (deploy, 1, 1, linux.4xlarge.nvidia.gpu) (17/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:50:33.3352031Z �[0;31m[ FAILED ] �[mTorchpyTest.PrintInstruction
2022-09-06T18:50:33.3347995Z �[0;31m[  FAILED  ] �[mTorchpyTest.AcquireMultipleSessionsInDifferentPackages
2022-09-06T18:50:33.3348501Z �[0;31m[  FAILED  ] �[mTorchpyTest.TensorSharingNotAllowed
2022-09-06T18:50:33.3348894Z �[0;31m[  FAILED  ] �[mTorchpyTest.TaggingRace
2022-09-06T18:50:33.3349261Z �[0;31m[  FAILED  ] �[mTorchpyTest.DisarmHook
2022-09-06T18:50:33.3349653Z �[0;31m[  FAILED  ] �[mTorchpyTest.RegisterModule
2022-09-06T18:50:33.3350062Z �[0;31m[  FAILED  ] �[mTorchpyTest.TensorSerializationSharing
2022-09-06T18:50:33.3350485Z �[0;31m[  FAILED  ] �[mTorchpyTest.SharedLibraryLoad
2022-09-06T18:50:33.3350875Z �[0;31m[  FAILED  ] �[mTorchpyTest.UsesDistributed
2022-09-06T18:50:33.3351243Z �[0;31m[  FAILED  ] �[mTorchpyTest.Autograd
2022-09-06T18:50:33.3351620Z �[0;31m[  FAILED  ] �[mTorchpyTest.ImportlibMetadata
2022-09-06T18:50:33.3352031Z �[0;31m[  FAILED  ] �[mTorchpyTest.PrintInstruction
2022-09-06T18:50:33.3352239Z 
2022-09-06T18:50:33.3352334Z 21 FAILED TESTS
2022-09-06T18:50:33.7601727Z ##[error]Process completed with exit code 1.
2022-09-06T18:50:33.7647532Z Prepare all required actions
2022-09-06T18:50:33.7648004Z Getting action download info
2022-09-06T18:50:34.0075946Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:50:34.0076255Z with:
2022-09-06T18:50:34.0076714Z   github-token: ***
2022-09-06T18:50:34.0076962Z env:
2022-09-06T18:50:34.0077204Z   GIT_DEFAULT_BRANCH: master

See GitHub Actions build pull / linux-focal-py3.7-gcc7 / test (default, 1, 2, linux.2xlarge) (18/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:30:45.6122197Z TypeError: len() of a 0-d tensor
2022-09-06T18:30:43.6461253Z + pushd .jenkins/pytorch
2022-09-06T18:30:43.6461519Z ~/workspace/.jenkins/pytorch ~/workspace
2022-09-06T18:30:43.6462071Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())'
2022-09-06T18:30:44.5351118Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:30:44.7162070Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');import torch;print(torch.tensor([torch.tensor(0.), torch.tensor(1.)]))'
2022-09-06T18:30:45.6119348Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:30:45.6119785Z Traceback (most recent call last):
2022-09-06T18:30:45.6120003Z   File "<string>", line 1, in <module>
2022-09-06T18:30:45.6120417Z   File "/opt/conda/lib/python3.7/site-packages/torch/_tensor.py", line 880, in __len__
2022-09-06T18:30:45.6121931Z     raise TypeError("len() of a 0-d tensor")
2022-09-06T18:30:45.6122197Z TypeError: len() of a 0-d tensor
2022-09-06T18:30:45.8240183Z ##[error]Process completed with exit code 1.
2022-09-06T18:30:45.8275167Z Prepare all required actions
2022-09-06T18:30:45.8275464Z Getting action download info
2022-09-06T18:30:46.0448803Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:30:46.0449031Z with:
2022-09-06T18:30:46.0449376Z   github-token: ***
2022-09-06T18:30:46.0449541Z env:
2022-09-06T18:30:46.0449718Z   GIT_DEFAULT_BRANCH: master
2022-09-06T18:30:46.0449908Z ##[endgroup]
2022-09-06T18:30:46.0476237Z ##[group]Run nick-fields/retry@7d4a37704547a311dbb66ebdf5b23ec19374a767

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (default, 2, 4, linux.4xlarge.nvidia.gpu) (19/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:56:57.5965437Z FAIL [0.004s]: test_zero_dim (__main__.FakeTensorTest)
2022-09-06T18:56:57.5959267Z 
2022-09-06T18:56:57.5959508Z ======================================================================
2022-09-06T18:56:57.5960167Z ERROR [0.001s]: test_no_ref_cycle (__main__.FakeTensorConverterTest)
2022-09-06T18:56:57.5961073Z ----------------------------------------------------------------------
2022-09-06T18:56:57.5961722Z Traceback (most recent call last):
2022-09-06T18:56:57.5962429Z   File "/var/lib/jenkins/workspace/test/test_fake_tensor.py", line 535, in test_no_ref_cycle
2022-09-06T18:56:57.5963155Z     mode = torch._prims.get_prim_fake_mode()
2022-09-06T18:56:57.5964089Z AttributeError: module 'torch._prims' has no attribute 'get_prim_fake_mode'
2022-09-06T18:56:57.5964609Z 
2022-09-06T18:56:57.5964838Z ======================================================================
2022-09-06T18:56:57.5965437Z FAIL [0.004s]: test_zero_dim (__main__.FakeTensorTest)
2022-09-06T18:56:57.5966296Z ----------------------------------------------------------------------
2022-09-06T18:56:57.5966940Z Traceback (most recent call last):
2022-09-06T18:56:57.5967638Z   File "/var/lib/jenkins/workspace/test/test_fake_tensor.py", line 68, in test_zero_dim
2022-09-06T18:56:57.5968377Z     self.assertEqual(out.device, y.device)
2022-09-06T18:56:57.5969475Z   File "/opt/conda/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2401, in assertEqual
2022-09-06T18:56:57.5970192Z     assert_equal(
2022-09-06T18:56:57.5971356Z   File "/opt/conda/lib/python3.10/site-packages/torch/testing/_comparison.py", line 1093, in assert_equal
2022-09-06T18:56:57.5972145Z     raise error_metas[0].to_error(msg)
2022-09-06T18:56:57.5973048Z AssertionError: Object comparison failed: device(type='cpu') != device(type='cuda', index=0)
2022-09-06T18:56:57.5973551Z 

See GitHub Actions build pull / linux-focal-py3.7-gcc7 / test (jit_legacy, 1, 1, linux.2xlarge) (20/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:32:24.8945457Z FAIL [0.022s]: test_trace_slice_with_grad (jit.test_tracer.TestTracer)
2022-09-06T18:32:24.8942947Z     msg=(lambda generated_msg: f"{generated_msg} : {msg}") if isinstance(msg, str) and self.longMessage else msg,
2022-09-06T18:32:24.8943414Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1093, in assert_equal
2022-09-06T18:32:24.8943752Z     raise error_metas[0].to_error(msg)
2022-09-06T18:32:24.8944017Z AssertionError: Tensor-likes are not close!
2022-09-06T18:32:24.8944162Z 
2022-09-06T18:32:24.8944250Z Mismatched elements: 4 / 5 (80.0%)
2022-09-06T18:32:24.8944582Z Greatest absolute difference: 0.7121428073139517 at index (4,) (up to 1e-07 allowed)
2022-09-06T18:32:24.8944954Z Greatest relative difference: 0.6556061642040036 at index (4,) (up to 1e-07 allowed)
2022-09-06T18:32:24.8945106Z 
2022-09-06T18:32:24.8945202Z ======================================================================
2022-09-06T18:32:24.8945457Z FAIL [0.022s]: test_trace_slice_with_grad (jit.test_tracer.TestTracer)
2022-09-06T18:32:24.8945814Z ----------------------------------------------------------------------
2022-09-06T18:32:24.8946062Z Traceback (most recent call last):
2022-09-06T18:32:24.8946352Z   File "/var/lib/jenkins/workspace/test/jit/test_tracer.py", line 482, in test_trace_slice_with_grad
2022-09-06T18:32:24.8946627Z     self.do_trace_slice(True)
2022-09-06T18:32:24.8947011Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 1273, in wrapper
2022-09-06T18:32:24.8947274Z     fn(*args, **kwargs)
2022-09-06T18:32:24.8947529Z   File "/var/lib/jenkins/workspace/test/jit/test_tracer.py", line 469, in do_trace_slice
2022-09-06T18:32:24.8947815Z     self.assertEqual(traced_slice(y), slice(y))
2022-09-06T18:32:24.8948216Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 2437, in assertEqual
2022-09-06T18:32:24.8948600Z     msg=(lambda generated_msg: f"{generated_msg} : {msg}") if isinstance(msg, str) and self.longMessage else msg,

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (default, 3, 4, linux.4xlarge.nvidia.gpu) (21/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:51:25.2286726Z RuntimeError: test_autograd failed!
2022-09-06T18:51:24.6302257Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestAutogradForwardModeBatchedGrad-20220906185056.xml
2022-09-06T18:51:24.6412577Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-autograd.test_functional.TestAutogradFunctional-20220906185056.xml
2022-09-06T18:51:24.6431953Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestAutogradInferenceMode-20220906185056.xml
2022-09-06T18:51:24.6439425Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestAutogradMultipleDispatchCUDA-20220906185056.xml
2022-09-06T18:51:24.6464795Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestMultithreadAutograd-20220906185056.xml
2022-09-06T18:51:25.2281516Z Traceback (most recent call last):
2022-09-06T18:51:25.2281949Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1104, in <module>
2022-09-06T18:51:25.2283689Z     main()
2022-09-06T18:51:25.2284001Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1082, in main
2022-09-06T18:51:25.2286405Z     raise RuntimeError(err_message)
2022-09-06T18:51:25.2286726Z RuntimeError: test_autograd failed!
2022-09-06T18:51:25.4797195Z 
2022-09-06T18:51:25.4797687Z real	1m36.398s
2022-09-06T18:51:25.4797969Z user	1m16.314s
2022-09-06T18:51:25.4798224Z sys	0m8.042s
2022-09-06T18:51:25.4846066Z ##[error]Process completed with exit code 1.
2022-09-06T18:51:25.4890139Z Prepare all required actions
2022-09-06T18:51:25.4890484Z Getting action download info
2022-09-06T18:51:25.7390817Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:51:25.7391116Z with:
2022-09-06T18:51:25.7391744Z   github-token: ***

See GitHub Actions build pull / linux-vulkan-bionic-py3.7-clang9 / test (default, 1, 1, linux.2xlarge) (22/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:32:50.2752086Z what():
2022-09-06T18:32:50.0136014Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2022-09-06T18:32:50.0136559Z + [[ linux-vulkan-bionic-py3.7-clang9 == *asan* ]]
2022-09-06T18:32:50.0136824Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]]
2022-09-06T18:32:50.0137240Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]]
2022-09-06T18:32:50.0138946Z + [[ linux-vulkan-bionic-py3.7-clang9 == *tbb* ]]
2022-09-06T18:32:50.0151136Z + [[ linux-vulkan-bionic-py3.7-clang9 == *libtorch* ]]
2022-09-06T18:32:50.0151708Z + [[ linux-vulkan-bionic-py3.7-clang9 == *-bazel-* ]]
2022-09-06T18:32:50.0153840Z + cd test
2022-09-06T18:32:50.0154140Z + python -c 'import torch; print(torch.__config__.show())'
2022-09-06T18:32:50.2751688Z terminate called after throwing an instance of 'c10::Error'
2022-09-06T18:32:50.2752086Z   what():  
2022-09-06T18:32:50.2752295Z Mismatch in kernel C++ signatures
2022-09-06T18:32:50.2753019Z   operator: aten::empty_strided(int[] size, int[] stride, *, int? dtype=None, int? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
2022-09-06T18:32:50.2753668Z     registered at /var/lib/jenkins/workspace/build/aten/src/ATen/RegisterSchema.cpp:6
2022-09-06T18:32:50.2754249Z   kernel 1: at::Tensor (c10::SymIntArrayRef, c10::SymIntArrayRef, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>)
2022-09-06T18:32:50.2754849Z     dispatch key: BackendSelect
2022-09-06T18:32:50.2755274Z     registered at /var/lib/jenkins/workspace/build/aten/src/ATen/RegisterBackendSelect.cpp:726
2022-09-06T18:32:50.2755692Z   kernel 2: at::Tensor (c10::ArrayRef<long>, c10::ArrayRef<long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>)
2022-09-06T18:32:50.2756004Z     dispatch key: Vulkan
2022-09-06T18:32:50.2756266Z     registered at /var/lib/jenkins/workspace/aten/src/ATen/native/vulkan/ops/Factory.cpp:69
2022-09-06T18:32:50.2756453Z 

See GitHub Actions build pull / linux-bionic-py3.7-clang9 / test (crossref, 2, 2, linux.2xlarge) (23/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:36:06.3956144Z RuntimeError: test_autograd failed!
2022-09-06T18:36:05.9817990Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestAutogradForwardModeBatchedGrad-20220906183543.xml
2022-09-06T18:36:05.9902956Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-autograd.test_functional.TestAutogradFunctional-20220906183543.xml
2022-09-06T18:36:05.9918019Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestAutogradInferenceMode-20220906183543.xml
2022-09-06T18:36:05.9932129Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestAutogradMultipleDispatchCPU-20220906183543.xml
2022-09-06T18:36:05.9939628Z Generated XML report: test-reports/python-unittest/test_autograd/TEST-TestMultithreadAutograd-20220906183543.xml
2022-09-06T18:36:06.3951408Z Traceback (most recent call last):
2022-09-06T18:36:06.3951705Z   File "test/run_test.py", line 1104, in <module>
2022-09-06T18:36:06.3953425Z     main()
2022-09-06T18:36:06.3953644Z   File "test/run_test.py", line 1082, in main
2022-09-06T18:36:06.3955905Z     raise RuntimeError(err_message)
2022-09-06T18:36:06.3956144Z RuntimeError: test_autograd failed!
2022-09-06T18:36:06.6660363Z 
2022-09-06T18:36:06.6660720Z real	3m54.015s
2022-09-06T18:36:06.6661087Z user	0m36.077s
2022-09-06T18:36:06.6661620Z sys	0m7.087s
2022-09-06T18:36:06.6697959Z ##[error]Process completed with exit code 1.
2022-09-06T18:36:06.6737031Z Prepare all required actions
2022-09-06T18:36:06.6737336Z Getting action download info
2022-09-06T18:36:06.8580707Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:36:06.8580929Z with:
2022-09-06T18:36:06.8581264Z   github-token: ***

See GitHub Actions build pull / linux-focal-py3.7-clang7-asan / test (default, 3, 5, linux.2xlarge) (24/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:39:29.0217783Z RuntimeError: test_optim failed!
2022-09-06T18:39:28.3381798Z Generating XML reports...
2022-09-06T18:39:28.3436293Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestDifferentiableOptimizer-20220906183621.xml
2022-09-06T18:39:28.3535817Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestLRScheduler-20220906183621.xml
2022-09-06T18:39:28.3571233Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestOptim-20220906183621.xml
2022-09-06T18:39:28.3581081Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestSWAUtils-20220906183621.xml
2022-09-06T18:39:29.0209929Z Traceback (most recent call last):
2022-09-06T18:39:29.0210259Z   File "test/run_test.py", line 1104, in <module>
2022-09-06T18:39:29.0213593Z     main()
2022-09-06T18:39:29.0213845Z   File "test/run_test.py", line 1082, in main
2022-09-06T18:39:29.0217365Z     raise RuntimeError(err_message)
2022-09-06T18:39:29.0217783Z RuntimeError: test_optim failed!
2022-09-06T18:39:29.6779686Z 
2022-09-06T18:39:29.6780176Z real	3m15.253s
2022-09-06T18:39:29.6780563Z user	3m12.283s
2022-09-06T18:39:29.6780817Z sys	0m3.234s
2022-09-06T18:39:29.6825397Z ##[error]Process completed with exit code 1.
2022-09-06T18:39:29.6864659Z Prepare all required actions
2022-09-06T18:39:29.6864951Z Getting action download info
2022-09-06T18:39:29.8741460Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:39:29.8741686Z with:
2022-09-06T18:39:29.8742055Z   github-token: ***

See GitHub Actions build pull / linux-bionic-py3.7-clang9 / test (dynamo, 2, 2, linux.2xlarge) (25/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:33:46.7433539Z AssertionError: to...n
2022-09-06T18:33:46.5607369Z     ds_x = torch.utils.data.TensorDataset(x)
2022-09-06T18:33:46.5607577Z 
2022-09-06T18:33:46.5607712Z Set torchdynamo.config.verbose=True for more information
2022-09-06T18:33:46.5607927Z ==========
2022-09-06T18:33:46.7358888Z ok (0.184s)
2022-09-06T18:33:46.7429687Z   test_update_bn_dnn (__main__.TestSWAUtils) ... torchdynamo.convert_frame: [ERROR] WON'T CONVERT test_update_bn_dnn test_optim.py line 2721 
2022-09-06T18:33:46.7430261Z due to: 
2022-09-06T18:33:46.7430626Z Traceback (most recent call last):
2022-09-06T18:33:46.7431230Z   File "/var/lib/jenkins/torchdynamo/torchdynamo/variables/tensor.py", line 283, in create
2022-09-06T18:33:46.7432659Z     ), f"torch.* op returned non-Tensor {typestr(example_value)} {proxy.node.op} {proxy.node.target}"
2022-09-06T18:33:46.7433539Z AssertionError: torch.* op returned non-Tensor TensorDataset call_function <class 'torch.utils.data.dataset.TensorDataset'>
2022-09-06T18:33:46.7434059Z 
2022-09-06T18:33:46.7434278Z from user code:
2022-09-06T18:33:46.7434655Z    File "test_optim.py", line 2726, in test_update_bn_dnn
2022-09-06T18:33:46.7435092Z     ds_x = torch.utils.data.TensorDataset(x)
2022-09-06T18:33:46.7435322Z 
2022-09-06T18:33:46.7435554Z Set torchdynamo.config.verbose=True for more information
2022-09-06T18:33:46.7435916Z ==========
2022-09-06T18:33:46.9049212Z ok (0.168s)
2022-09-06T18:33:46.9049406Z 
2022-09-06T18:33:46.9049563Z ======================================================================

See GitHub Actions build pull / linux-bionic-py3_7-clang8-xla / test (xla, 1, 1, linux.2xlarge) (26/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:43:18.6127446Z /var/lib/jenkins/w...f' (aka 'ArrayRef') to 'c10::SymIntArrayRef'
2022-09-06T18:43:18.6060194Z                  ^
2022-09-06T18:43:18.6060701Z /var/lib/jenkins/workspace/c10/core/SymIntArrayRef.h:79:28: note: candidate template ignored: could not match 'const c10::SymInt [N]' against 'at::IntArrayRef' (aka 'ArrayRef<long>')
2022-09-06T18:43:18.6061099Z   /* implicit */ constexpr SymIntArrayRef(const c10::SymInt (&Arr)[N])
2022-09-06T18:43:18.6061333Z                            ^
2022-09-06T18:43:18.6061759Z /opt/conda/lib/python3.7/site-packages/torch/include/ATen/ops/new_empty_strided_native.h:20:85: note: passing argument to parameter 'size' here
2022-09-06T18:43:18.6062347Z TORCH_API at::Tensor new_empty_strided(const at::Tensor & self, c10::SymIntArrayRef size, c10::SymIntArrayRef stride, c10::optional<at::ScalarType> dtype={}, c10::optional<at::Layout> layout={}, c10::optional<at::Device> device={}, c10::optional<bool> pin_memory={});
2022-09-06T18:43:18.6062777Z                                                                                     ^
2022-09-06T18:43:18.6125294Z /var/lib/jenkins/workspace/xla/torch_xla/csrc/aten_xla_type.cpp:3141:32: error: out-of-line definition of 'diagonal_backward' does not match any declaration in 'torch_xla::XLANativeFunctions'
2022-09-06T18:43:18.6126058Z at::Tensor XLANativeFunctions::diagonal_backward(const at::Tensor& grad_output,
2022-09-06T18:43:18.6126505Z                                ^~~~~~~~~~~~~~~~~
2022-09-06T18:43:18.6127446Z /var/lib/jenkins/workspace/xla/torch_xla/csrc/aten_xla_type.cpp:3145:53: error: no viable conversion from 'at::IntArrayRef' (aka 'ArrayRef<long>') to 'c10::SymIntArrayRef'
2022-09-06T18:43:18.6128133Z   return at::native::diagonal_backward(grad_output, input_sizes, offset, dim1,
2022-09-06T18:43:18.6128530Z                                                     ^~~~~~~~~~~
2022-09-06T18:43:18.6129700Z /var/lib/jenkins/workspace/c10/core/SymIntArrayRef.h:38:7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'at::IntArrayRef' (aka 'ArrayRef<long>') to 'const c10::SymIntArrayRef &' for 1st argument
2022-09-06T18:43:18.6130441Z class SymIntArrayRef final {
2022-09-06T18:43:18.6130923Z       ^
2022-09-06T18:43:18.6132040Z /var/lib/jenkins/workspace/c10/core/SymIntArrayRef.h:38:7: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'at::IntArrayRef' (aka 'ArrayRef<long>') to 'c10::SymIntArrayRef &&' for 1st argument
2022-09-06T18:43:18.6133461Z /var/lib/jenkins/workspace/c10/core/SymIntArrayRef.h:57:18: note: candidate constructor not viable: no known conversion from 'at::IntArrayRef' (aka 'ArrayRef<long>') to 'const std::vector<c10::SymInt> &' for 1st argument
2022-09-06T18:43:18.6134228Z   /* implicit */ SymIntArrayRef(const std::vector<c10::SymInt>& Vec)
2022-09-06T18:43:18.6134612Z                  ^
2022-09-06T18:43:18.6135384Z /var/lib/jenkins/workspace/c10/core/SymIntArrayRef.h:67:18: note: candidate template ignored: could not match 'SmallVectorTemplateCommon' against 'ArrayRef'

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (default, 1, 4, linux.4xlarge.nvidia.gpu) (27/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:48:40.6813055Z TypeError: len() of a 0-d tensor
2022-09-06T18:48:37.8428855Z + pushd .jenkins/pytorch
2022-09-06T18:48:37.8429146Z ~/workspace/.jenkins/pytorch ~/workspace
2022-09-06T18:48:37.8429870Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())'
2022-09-06T18:48:39.1678291Z <string>:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:48:39.3545863Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');import torch;print(torch.tensor([torch.tensor(0.), torch.tensor(1.)]))'
2022-09-06T18:48:40.6808490Z <string>:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:48:40.6809144Z Traceback (most recent call last):
2022-09-06T18:48:40.6809599Z   File "<string>", line 1, in <module>
2022-09-06T18:48:40.6810213Z   File "/opt/conda/lib/python3.10/site-packages/torch/_tensor.py", line 880, in __len__
2022-09-06T18:48:40.6812474Z     raise TypeError("len() of a 0-d tensor")
2022-09-06T18:48:40.6813055Z TypeError: len() of a 0-d tensor
2022-09-06T18:48:40.8718350Z ##[error]Process completed with exit code 1.
2022-09-06T18:48:40.8762060Z Prepare all required actions
2022-09-06T18:48:40.8762423Z Getting action download info
2022-09-06T18:48:41.0471097Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:48:41.0471379Z with:
2022-09-06T18:48:41.0471842Z   github-token: ***
2022-09-06T18:48:41.0472242Z env:
2022-09-06T18:48:41.0472489Z   GIT_DEFAULT_BRANCH: master
2022-09-06T18:48:41.0473013Z   GPU_FLAG: --gpus all
2022-09-06T18:48:41.0473266Z ##[endgroup]

See GitHub Actions build pull / linux-bionic-py3.7-clang9 / test (default, 2, 2, linux.2xlarge) (28/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:47:58.9070682Z FAIL [0.067s]: tes...ag_cpu_float32 (__main__.TestProxyTensorOpInfoCPU)
2022-09-06T18:47:58.9067739Z     batch_dim, channel_dim * kernel_h * kernel_w, num_blocks_row * num_blocks_col
2022-09-06T18:47:58.9068135Z   File "/opt/conda/lib/python3.7/site-packages/torch/utils/_python_dispatch.py", line 74, in wrapped
2022-09-06T18:47:58.9068414Z     return f(self, *args, **kwargs)
2022-09-06T18:47:58.9068800Z   File "/opt/conda/lib/python3.7/site-packages/torch/_subclasses/fake_tensor.py", line 718, in __torch_dispatch__
2022-09-06T18:47:58.9069084Z     r = meta_table[func](*args, **kwargs)
2022-09-06T18:47:58.9069462Z   File "/opt/conda/lib/python3.7/site-packages/torch/_meta_registrations.py", line 98, in reshape
2022-09-06T18:47:58.9069780Z     stride = computeStride(self.sizes(), self.strides(), shape)
2022-09-06T18:47:58.9070117Z AttributeError: 'FakeTensor' object has no attribute 'sizes'
2022-09-06T18:47:58.9070266Z 
2022-09-06T18:47:58.9070365Z ======================================================================
2022-09-06T18:47:58.9070682Z FAIL [0.067s]: test_make_fx_fake_exhaustive_nn_functional_embedding_bag_cpu_float32 (__main__.TestProxyTensorOpInfoCPU)
2022-09-06T18:47:58.9071101Z ----------------------------------------------------------------------
2022-09-06T18:47:58.9071351Z Traceback (most recent call last):
2022-09-06T18:47:58.9071764Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 378, in instantiated_test
2022-09-06T18:47:58.9072070Z     result = test(self, **param_kwargs)
2022-09-06T18:47:58.9072470Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 815, in test_wrapper
2022-09-06T18:47:58.9072746Z     return test(*args, **kwargs)
2022-09-06T18:47:58.9072996Z   File "test_proxy_tensor.py", line 1343, in test_make_fx_fake_exhaustive
2022-09-06T18:47:58.9073275Z     _test_make_fx_helper(self, device, dtype, op, "fake")
2022-09-06T18:47:58.9073535Z   File "test_proxy_tensor.py", line 1332, in _test_make_fx_helper
2022-09-06T18:47:58.9073781Z     self.assertEqual(new_out, old_out)

See GitHub Actions build pull / linux-focal-py3.7-clang10-onnx / test (default, 2, 2, linux.2xlarge) (29/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:37:39.5179048Z RuntimeError: iter...peManual.cpp":241, please report a bug to PyTorch.
2022-09-06T18:37:39.5172811Z   File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1178, in _slow_forward
2022-09-06T18:37:39.5173297Z     result = self.forward(*input, **kwargs)
2022-09-06T18:37:39.5173970Z   File "/var/lib/jenkins/.local/lib/python3.7/site-packages/torchvision/models/detection/rpn.py", line 372, in forward
2022-09-06T18:37:39.5174623Z     boxes, scores = self.filter_proposals(proposals, objectness, images.image_sizes, num_anchors_per_level)
2022-09-06T18:37:39.5175458Z   File "/var/lib/jenkins/.local/lib/python3.7/site-packages/torchvision/models/detection/rpn.py", line 267, in filter_proposals
2022-09-06T18:37:39.5175986Z     objectness = objectness[batch_idx, top_n_idx]
2022-09-06T18:37:39.5176618Z   File "/opt/conda/lib/python3.7/site-packages/torch/patch_getitem.py", line 232, in __getitem__
2022-09-06T18:37:39.5177128Z     self = prim_gather(self, gather_dims, gather_tensors)
2022-09-06T18:37:39.5177730Z   File "/opt/conda/lib/python3.7/site-packages/torch/_ops.py", line 60, in __call__
2022-09-06T18:37:39.5178157Z     return self._op(*args, **kwargs or {})
2022-09-06T18:37:39.5179048Z RuntimeError: iter->isIntList() INTERNAL ASSERT FAILED at "/var/lib/jenkins/workspace/torch/csrc/autograd/TraceTypeManual.cpp":241, please report a bug to PyTorch. 
2022-09-06T18:37:39.5179489Z 
2022-09-06T18:37:39.5179836Z ---------- coverage: platform linux, python 3.7.13-final-0 -----------
2022-09-06T18:37:39.5180121Z 
2022-09-06T18:37:39.5180310Z =========================== short test summary info ============================
2022-09-06T18:37:39.5180859Z FAILED test/onnx/test_models_onnxruntime.py::TestModelsONNXRuntime_is_script_False::test_faster_rcnn
2022-09-06T18:37:39.5181503Z FAILED test/onnx/test_models_onnxruntime.py::TestModelsONNXRuntime_is_script_False::test_mask_rcnn
2022-09-06T18:37:39.5182138Z FAILED test/onnx/test_models_onnxruntime.py::TestModelsONNXRuntime_is_script_False::test_roi_heads
2022-09-06T18:37:39.5182944Z �[31m===== �[31m�[1m3 failed�[0m, �[32m3 passed�[0m, �[33m8 skipped�[0m, �[33m110 deselected�[0m, �[33m63 warnings�[0m�[31m in 19.40s�[0m�[31m =====�[0m
2022-09-06T18:37:39.8407283Z ##[error]Process completed with exit code 1.
2022-09-06T18:37:39.8443079Z Prepare all required actions

See GitHub Actions build pull / linux-focal-py3.7-clang7-asan / test (default, 5, 5, linux.2xlarge) (30/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:25:49.5347885Z RuntimeError: test_quantization failed!
2022-09-06T19:25:48.4802839Z Generated XML report: test-reports/python-unittest/test_quantization/TEST-quantization.fx.test_subgraph_rewriter.TestSubgraphRewriter-20220906183637.xml
2022-09-06T19:25:48.4809776Z Generated XML report: test-reports/python-unittest/test_quantization/TEST-quantization.core.test_utils.TestUtils-20220906183637.xml
2022-09-06T19:25:48.4813813Z Generated XML report: test-reports/python-unittest/test_quantization/TEST-quantization.fx.test_numeric_suite_fx.TestFXGraphMatcherModels-20220906183637.xml
2022-09-06T19:25:48.4828529Z Generated XML report: test-reports/python-unittest/test_quantization/TEST-quantization.core.test_quantized_op.TestQNNPackOps-20220906183637.xml
2022-09-06T19:25:48.4833846Z Generated XML report: test-reports/python-unittest/test_quantization/TEST-quantization.eager.test_quantize_eager_ptq.TestQuantizeEagerONNXExport-20220906183637.xml
2022-09-06T19:25:49.5339710Z Traceback (most recent call last):
2022-09-06T19:25:49.5339994Z   File "test/run_test.py", line 1104, in <module>
2022-09-06T19:25:49.5343761Z     main()
2022-09-06T19:25:49.5343985Z   File "test/run_test.py", line 1082, in main
2022-09-06T19:25:49.5347651Z     raise RuntimeError(err_message)
2022-09-06T19:25:49.5347885Z RuntimeError: test_quantization failed!
2022-09-06T19:25:50.1654329Z 
2022-09-06T19:25:50.1654648Z real	49m39.028s
2022-09-06T19:25:50.1654865Z user	55m54.744s
2022-09-06T19:25:50.1655080Z sys	1m50.269s
2022-09-06T19:25:50.1691782Z ##[error]Process completed with exit code 1.
2022-09-06T19:25:50.1739774Z Prepare all required actions
2022-09-06T19:25:50.1740171Z Getting action download info
2022-09-06T19:25:50.3327827Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T19:25:50.3328051Z with:
2022-09-06T19:25:50.3328373Z   github-token: ***

See GitHub Actions build pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (default, 4, 4, linux.4xlarge.nvidia.gpu) (31/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:08:15.9244724Z RuntimeError: test_optim failed!
2022-09-06T19:08:15.3418629Z Generating XML reports...
2022-09-06T19:08:15.3433789Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestDifferentiableOptimizer-20220906190634.xml
2022-09-06T19:08:15.3546230Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestLRScheduler-20220906190634.xml
2022-09-06T19:08:15.3586413Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestOptim-20220906190634.xml
2022-09-06T19:08:15.3595928Z Generated XML report: test-reports/python-unittest/test_optim/TEST-TestSWAUtils-20220906190634.xml
2022-09-06T19:08:15.9240151Z Traceback (most recent call last):
2022-09-06T19:08:15.9240760Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1104, in <module>
2022-09-06T19:08:15.9241393Z     main()
2022-09-06T19:08:15.9241701Z   File "/var/lib/jenkins/workspace/test/run_test.py", line 1082, in main
2022-09-06T19:08:15.9244404Z     raise RuntimeError(err_message)
2022-09-06T19:08:15.9244724Z RuntimeError: test_optim failed!
2022-09-06T19:08:16.1780544Z 
2022-09-06T19:08:16.1781162Z real	18m30.240s
2022-09-06T19:08:16.1781717Z user	18m37.689s
2022-09-06T19:08:16.1782027Z sys	0m12.708s
2022-09-06T19:08:16.1839093Z ##[error]Process completed with exit code 1.
2022-09-06T19:08:16.1886459Z Prepare all required actions
2022-09-06T19:08:16.1886833Z Getting action download info
2022-09-06T19:08:16.3849289Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T19:08:16.3849595Z with:
2022-09-06T19:08:16.3850103Z   github-token: ***

See GitHub Actions build Lint / lintrunner (32/35)

Step: "Run lintrunner on all files" (full log | diagnosis details)

2022-09-06T18:19:38.5926805Z ##[error]Cannot assign to a method
2022-09-06T18:19:38.5916016Z ##[warning]'.decompositions.get_decompositions' imported but unused
See https://www.flake8rules.com/rules/F401.html
2022-09-06T18:19:38.5917356Z ##[error]Cannot determine type of "gather" 
2022-09-06T18:19:38.5918488Z ##[error]Incompatible types in assignment (expression has type "List[int]", variable has type "Tuple[Any, ...]") 
2022-09-06T18:19:38.5919811Z ##[error]Argument 2 to "prim_reorder" has incompatible type "List[int]"; expected "List[Optional[int]]" 
2022-09-06T18:19:38.5921061Z ##[error]Need type annotation for "gather_dims" (hint: "gather_dims: List[<type>] = ...") 
2022-09-06T18:19:38.5922297Z ##[error]Need type annotation for "gather_tensors" (hint: "gather_tensors: List[<type>] = ...") 
2022-09-06T18:19:38.5923594Z ##[error]Need type annotation for "permute" (hint: "permute: List[<type>] = ...") 
2022-09-06T18:19:38.5924871Z ##[error]Argument 3 to "prim_slice" has incompatible type "List[int]"; expected "List[Union[slice, int]]" 
2022-09-06T18:19:38.5925883Z ##[error]Cannot assign to a method 
2022-09-06T18:19:38.5926805Z ##[error]Cannot assign to a method 
2022-09-06T18:19:38.5927894Z ##[warning]Run `lintrunner -a` to apply this patch.
2022-09-06T18:19:38.5929178Z ##[warning]'torch.autograd.grad_mode.F' imported but unused
See https://www.flake8rules.com/rules/F401.html
2022-09-06T18:19:38.5930461Z ##[warning]do not use bare 'except'
See https://www.flake8rules.com/rules/E722.html
2022-09-06T18:19:38.5932305Z ##[warning]Do not use bare `except:`, it also catches unexpected events like memory errors, interrupts, system exit, and so on.  Prefer `except Exception:`.  If you're sure what you're doing, be explicit and write `except BaseException:`.
See https://github.com/PyCQA/flake8-bugbear#list-of-warnings
2022-09-06T18:19:38.5963594Z ##[warning]line too long (144 > 120 characters)
See https://github.com/PyCQA/flake8-bugbear#list-of-warnings
2022-09-06T18:19:38.5965739Z ##[warning]Do not call assert False since python -O removes these calls. Instead callers should raise AssertionError().

See GitHub Actions build pull / linux-bionic-py3.7-clang9 / test (functorch, 1, 1, linux.2xlarge) (33/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:44:58.8055475Z FAIL [3.100s]: tes...rg_as_static (__main__.TestCompileCacheStaticArgs)
2022-09-06T18:44:58.8051099Z     aot_config,
2022-09-06T18:44:58.8051553Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 452, in create_aot_dispatcher_function
2022-09-06T18:44:58.8052072Z     aot_dispatch_autograd(flat_fn, fake_flat_tensor_args, aot_config)
2022-09-06T18:44:58.8052789Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 328, in aot_dispatch_autograd
2022-09-06T18:44:58.8053326Z     fw_module, bw_module = aot_config.partition_fn(fx_g, joint_inputs)
2022-09-06T18:44:58.8053828Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/partitioners.py", line 157, in default_partition
2022-09-06T18:44:58.8054327Z     assert all(user.target == operator.getitem for user in users)
2022-09-06T18:44:58.8054692Z AssertionError
2022-09-06T18:44:58.8054878Z 
2022-09-06T18:44:58.8055012Z ======================================================================
2022-09-06T18:44:58.8055475Z FAIL [3.100s]: test_tuple_with_first_arg_as_static (__main__.TestCompileCacheStaticArgs)
2022-09-06T18:44:58.8056180Z ----------------------------------------------------------------------
2022-09-06T18:44:58.8056639Z Traceback (most recent call last):
2022-09-06T18:44:58.8057169Z   File "/var/lib/jenkins/workspace/functorch/test/test_compile_cache.py", line 477, in test_tuple_with_first_arg_as_static
2022-09-06T18:44:58.8057700Z     check(a, b, aot_autograd_f, fn)
2022-09-06T18:44:58.8058178Z   File "/var/lib/jenkins/workspace/functorch/test/test_compile_cache.py", line 460, in check
2022-09-06T18:44:58.8058659Z     res = aot_autograd_fn(a, (b0_clone, b1_clone))
2022-09-06T18:44:58.8059205Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 721, in returned_function
2022-09-06T18:44:58.8059682Z     aot_config,
2022-09-06T18:44:58.8060174Z   File "/var/lib/jenkins/workspace/functorch/functorch/_src/aot_autograd.py", line 452, in create_aot_dispatcher_function
2022-09-06T18:44:58.8060783Z     aot_dispatch_autograd(flat_fn, fake_flat_tensor_args, aot_config)

See GitHub Actions build pull / win-vs2019-cpu-py3 / test (functorch, 1, 1, windows.4xlarge) (34/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T19:06:40.1279628Z RuntimeError: C:\a...\pytorch\functorch\test\test_compile_cache failed!
2022-09-06T19:06:37.4522925Z FAILED (errors=6, skipped=13, expected failures=18)
2022-09-06T19:06:37.4523082Z 
2022-09-06T19:06:37.4523173Z Generating XML reports...
2022-09-06T19:06:39.1330562Z Generated XML report: test-reports\python-unittest\functorch\test\test_compile_cache\TEST-TestCompileCache-20220906190604.xml
2022-09-06T19:06:39.2397129Z Generated XML report: test-reports\python-unittest\functorch\test\test_compile_cache\TEST-TestCompileCacheStaticArgs-20220906190604.xml
2022-09-06T19:06:40.1271541Z Traceback (most recent call last):
2022-09-06T19:06:40.1271997Z   File "C:\actions-runner\_work\pytorch\pytorch\test\run_test.py", line 1104, in <module>
2022-09-06T19:06:40.1275760Z     main()
2022-09-06T19:06:40.1276096Z   File "C:\actions-runner\_work\pytorch\pytorch\test\run_test.py", line 1082, in main
2022-09-06T19:06:40.1279279Z     raise RuntimeError(err_message)
2022-09-06T19:06:40.1279628Z RuntimeError: C:\actions-runner\_work\pytorch\pytorch\functorch\test\test_compile_cache failed!
2022-09-06T19:06:40.3597582Z 
2022-09-06T19:06:40.3598021Z (base) C:\actions-runner\_work\pytorch\pytorch\test>popd
2022-09-06T19:06:40.3602070Z 
2022-09-06T19:06:40.3602322Z (base) C:\actions-runner\_work\pytorch\pytorch>if ERRORLEVEL 1 goto fail 
2022-09-06T19:06:40.3604513Z 
2022-09-06T19:06:40.3604721Z (base) C:\actions-runner\_work\pytorch\pytorch>exit /b 1 
2022-09-06T19:06:40.3665064Z ##[error]Process completed with exit code 1.
2022-09-06T19:06:40.4128903Z Prepare all required actions
2022-09-06T19:06:40.4129446Z Getting action download info
2022-09-06T19:06:40.5546336Z Download action repository 'nick-fields/retry@7d4a37704547a311dbb66ebdf5b23ec19374a767' (SHA:7d4a37704547a311dbb66ebdf5b23ec19374a767)

See GitHub Actions build pull / linux-bionic-py3.7-clang9 / test (default, 1, 2, linux.2xlarge) (35/35)

Step: "Test" (full log | diagnosis details)

2022-09-06T18:31:05.5162592Z TypeError: len() of a 0-d tensor
2022-09-06T18:31:03.6095658Z + pushd .jenkins/pytorch
2022-09-06T18:31:03.6096021Z ~/workspace/.jenkins/pytorch ~/workspace
2022-09-06T18:31:03.6096674Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())'
2022-09-06T18:31:04.4906585Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:31:04.6489969Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');import torch;print(torch.tensor([torch.tensor(0.), torch.tensor(1.)]))'
2022-09-06T18:31:05.5159732Z -c:1: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:77.)
2022-09-06T18:31:05.5160461Z Traceback (most recent call last):
2022-09-06T18:31:05.5160864Z   File "<string>", line 1, in <module>
2022-09-06T18:31:05.5161523Z   File "/opt/conda/lib/python3.7/site-packages/torch/_tensor.py", line 880, in __len__
2022-09-06T18:31:05.5162091Z     raise TypeError("len() of a 0-d tensor")
2022-09-06T18:31:05.5162592Z TypeError: len() of a 0-d tensor
2022-09-06T18:31:05.6712242Z ##[error]Process completed with exit code 1.
2022-09-06T18:31:05.6760442Z Prepare all required actions
2022-09-06T18:31:05.6760743Z Getting action download info
2022-09-06T18:31:05.8452587Z ##[group]Run ./.github/actions/get-workflow-job-id
2022-09-06T18:31:05.8452818Z with:
2022-09-06T18:31:05.8453162Z   github-token: ***
2022-09-06T18:31:05.8453327Z env:
2022-09-06T18:31:05.8453508Z   GIT_DEFAULT_BRANCH: master
2022-09-06T18:31:05.8453700Z ##[endgroup]
2022-09-06T18:31:05.8480550Z ##[group]Run nick-fields/retry@7d4a37704547a311dbb66ebdf5b23ec19374a767

🕵️‍♀️ 2 failures not recognized by patterns:

The following CI failures may be due to changes from the PR
Job Step
CircleCI Checks build Unknown
GitHub Actions win-vs2019-cpu-py3 / test (default, 2, 2, windows.4xlarge) Unknown

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@Krovatkin Krovatkin force-pushed the krovatkin/symbolic-shapes branch from 9f7bc95 to 44221ef Compare September 6, 2022 18:12
@Krovatkin Krovatkin requested review from Chillee and ezyang September 6, 2022 18:12

# TODO: smarter way to copy? copy.deepcopy?
# can I just use list(shape)?
res = list(shape)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

list is fine

def reshape(self, proposed_shape):
if self.is_sparse:
# TODO: not sure what else to do here?
raise RuntimeError("reshape is not implemented for sparse tensors")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems fine for now

@ezyang
Copy link
Copy Markdown
Contributor

ezyang commented Sep 6, 2022

This seems fine but you will have to monkeypatch it in for it to do anything

@pytorch-bot
Copy link
Copy Markdown

pytorch-bot bot commented Sep 8, 2022

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/84584

Note: Links to docs will display an error until the docs builds have been completed.

❌ 36 Failures

As of commit 3f5840d:

The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@Krovatkin Krovatkin requested a review from ezyang September 9, 2022 17:39
@Krovatkin Krovatkin force-pushed the krovatkin/symbolic-shapes branch from 731fb9e to 3f5840d Compare September 9, 2022 17:42
a = torch.rand(4, 4)
b = a.reshape((16, 1))
#b = torch.reshape(a, (16, 1))
print(b)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test is bad. There should be an OpInfo reshape test you can use

_register_jit_decomposition_for_jvp(torch.ops.aten.log_sigmoid_forward.default)
_register_jit_decomposition_for_jvp(torch.ops.aten.native_layer_norm_backward.default)
_register_jit_decomposition_for_jvp(torch.ops.aten.native_batch_norm_backward.default)
#_register_jit_decomposition_for_jvp(torch.ops.aten.native_batch_norm_backward.default)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

}
return from_complex.get(dtype, dtype)


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't wobble lines unnecessarily; especially in a long lived branch these lead to spurious merge conflicts. Big problem!

return self.view(self.clone(memory_format=torch.contiguous_format), shape)

torch.reshape = reshape
torch.Tensor.reshape = reshape
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should ultimately live in torch._refs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh shoot, there's a reshape implementation in torch._refs.reshape

@facebook-github-bot
Copy link
Copy Markdown
Contributor

/easycla

As part of the transition to the PyTorch Foundation, this project now requires contributions be covered under the new CLA. See #85559 for additional details.

This comment will trigger a new check of this PR. If you are already covered, you will simply see a new "EasyCLA" check that passes. If you are not covered, a bot will leave a new comment with a link to sign.

@linux-foundation-easycla
Copy link
Copy Markdown

CLA Not Signed

@github-actions
Copy link
Copy Markdown
Contributor

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

@github-actions github-actions bot added the Stale label Jan 23, 2023
ezyang added a commit that referenced this pull request Feb 22, 2023
This PR gets `reflect @ R @ reflect` working, where R has unbacked batch size. This pattern occurred in CrystalDPR. The billing of changes:

* torch.broadcast_shapes avoids guarding on unbacked SymInts when testing for broadcastable dims. I extracted this to #95217 for separate review; it's repeated in this PR as it is necessary for the E2E test
* I disable matrix multiply folding when there is an unbacked SymInt on any input. Folding is strictly a performance optimization and can be omitted. Also, I believe export would prefer to get matmul (rather than bmm/etc), so we should eventually actually get #91081 going
* I add a direct Python transcription of the reshape composite adapted from #84584 . I cannot use the PrimTorch composite as it has problems when I register it pre-autograd. It has the same implementation as regular reshape, but at the beginning there is one more test for trivial reshapes, which is sufficient for the matmul example.
* I hand-write a meta function for expand, rather than using the PrimTorch decomposition. I couldn't really figure out how to make the PrimTorch decomposition guard free, but with the hand-written meta it is clear where the divergence lies: we cannot easily choose the correct stride for the unbacked dim, as we need to know whether or not the size is one (in which case we give the predicted stride) versus non-one (in which case we MUST give zero.) In composability sync, we agreed that changes to striding behavior are fair game with unbacked SymInts, so I just unconditionally give these zero stride.

Signed-off-by: Edward Z. Yang <ezyangmeta.com>

[ghstack-poisoned]
@github-actions github-actions bot closed this Feb 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants