WIP: Port `linalg.tensordot` to structured kernels by antocuni · Pull Request #64819 · pytorch/pytorch

antocuni · 2021-09-10T15:28:14Z

PR stack (ghstack-style but done manually)

UPDATE
I think this PR cannot be completed at the moment, because structured kernels are not yet compatible with CompositeImplicitAutograd dispatch. See e.g. the following comment inside codegen/gen.py:

pytorch/tools/codegen/gen.py

Lines 176 to 181 in 4666e3f

    
           if f.structured_delegate is not None: 
        
               # TODO: for ops with structured_delegate it should check the dispatch table of 
        
               # the out variant instead. For now, these structured ops all have CPU/CUDA kernels 
        
               # so we always dispatch to the `backend`, but this could be wrong when we 
        
               # migrate math/default_backend ops to use structured delegate. 
        
               return f'return at::{backend_index.dispatch_key.lower()}::{name}({exprs_str});'

…in common_method_invocations

…nit__.py

…tion

…s. Make sure to preserve compatibility

…a Tensor

…function, but actually it's not

… linalg_tensordot_out, and not vice-versa

facebook-github-bot · 2021-09-10T15:28:20Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/64819
📄 Preview docs built from this PR
📄 Preview C++ docs built from this PR
🔧 Opt-in to CIFlow to control what jobs run on your PRs

💊 CI failures summary and remediations

As of commit 00ad7db (more details on the Dr. CI page):

16/16 failures possibly* introduced in this PR
- 1/16 non-scanned failure(s)

🕵️ 15 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

win-vs2019-cpu-py3 / test (default, 2, 2, windows.4xlarge) (1/15)

Step: "Run test scripts" (full log | diagnosis details | 🔁 rerun)

2021-09-10T17:03:55.9766969Z RuntimeError: test_fx failed!

2021-09-10T17:03:54.7278813Z Generated XML report: test-reports\dist-gloo\test_fx\TEST-TestFXAPIBackwardCompatibility-20210910170344.xml
2021-09-10T17:03:54.7280272Z Generated XML report: test-reports\dist-gloo\test_fx\TEST-TestFunctionalTracing-20210910170344.xml
2021-09-10T17:03:54.7281640Z Generated XML report: test-reports\dist-gloo\test_fx\TEST-TestOperatorSignaturesCPU-20210910170344.xml
2021-09-10T17:03:54.7283129Z Generated XML report: test-reports\dist-gloo\test_fx\TEST-fx.test_subgraph_rewriter.TestSubgraphRewriter-20210910170344.xml
2021-09-10T17:03:54.7284524Z Generated XML report: test-reports\dist-gloo\test_fx\TEST-fx.test_gradual_type.TypeCheckerTest-20210910170344.xml
2021-09-10T17:03:55.9764240Z Traceback (most recent call last):
2021-09-10T17:03:55.9764980Z   File "run_test.py", line 1023, in <module>
2021-09-10T17:03:55.9765379Z     main()
2021-09-10T17:03:55.9766099Z   File "run_test.py", line 1001, in main
2021-09-10T17:03:55.9766550Z     raise RuntimeError(err_message)
2021-09-10T17:03:55.9766969Z RuntimeError: test_fx failed!
2021-09-10T17:03:56.2213755Z 
2021-09-10T17:03:56.2214536Z (base) C:\actions-runner\_work\pytorch\pytorch\pytorch-1221812480\test>popd
2021-09-10T17:03:56.2218637Z 
2021-09-10T17:03:56.2219185Z (base) C:\actions-runner\_work\pytorch\pytorch\pytorch-1221812480>if ERRORLEVEL 1 exit /b 1 
2021-09-10T17:03:56.2242215Z + cleanup
2021-09-10T17:03:56.2242564Z + retcode=1
2021-09-10T17:03:56.2242837Z + set +x
2021-09-10T17:03:56.2275542Z ##[error]Process completed with exit code 1.
2021-09-10T17:03:56.2430887Z ##[group]Run # -ir => recursive include all files in pattern
2021-09-10T17:03:56.2431590Z �[36;1m# -ir => recursive include all files in pattern�[0m

win-vs2019-cpu-py3 / test (default, 1, 2, windows.4xlarge) (2/15)

Step: "Run test scripts" (full log | diagnosis details | 🔁 rerun)

2021-09-10T16:57:24.1355814Z RuntimeError: test_fx_experimental failed!

2021-09-10T16:57:23.0440540Z FAILED (errors=1, skipped=7)
2021-09-10T16:57:23.0440778Z 
2021-09-10T16:57:23.0441157Z Generating XML reports...
2021-09-10T16:57:23.0442072Z Generated XML report: test-reports\python-unittest\test_fx_experimental\TEST-TestFXExperimental-20210910165654.xml
2021-09-10T16:57:23.0443626Z Generated XML report: test-reports\python-unittest\test_fx_experimental\TEST-TestNormalizeOperatorsCPU-20210910165654.xml
2021-09-10T16:57:24.1353298Z Traceback (most recent call last):
2021-09-10T16:57:24.1354082Z   File "run_test.py", line 1023, in <module>
2021-09-10T16:57:24.1354450Z     main()
2021-09-10T16:57:24.1354859Z   File "run_test.py", line 1001, in main
2021-09-10T16:57:24.1355310Z     raise RuntimeError(err_message)
2021-09-10T16:57:24.1355814Z RuntimeError: test_fx_experimental failed!
2021-09-10T16:57:24.3425430Z 
2021-09-10T16:57:24.3426304Z (base) C:\actions-runner\_work\pytorch\pytorch\pytorch-1221812480\test>if ERRORLEVEL 1 exit /b 1 
2021-09-10T16:57:24.3449280Z + cleanup
2021-09-10T16:57:24.3449612Z + retcode=1
2021-09-10T16:57:24.3449894Z + set +x
2021-09-10T16:57:24.3481216Z ##[error]Process completed with exit code 1.
2021-09-10T16:57:24.3639273Z ##[group]Run # -ir => recursive include all files in pattern
2021-09-10T16:57:24.3639916Z �[36;1m# -ir => recursive include all files in pattern�[0m
2021-09-10T16:57:24.3640555Z �[36;1m7z a "test-reports-$Env:COMMIT_SHA1-$Env:WORKFLOW_ID.zip" -ir'!test\*.xml'�[0m
2021-09-10T16:57:24.3665629Z shell: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.EXE -command ". '{0}'"

linux-bionic-py3.8-gcc9-coverage / test (distributed, 1, 1, linux.2xlarge) (3/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T17:22:53.7474802Z AssertionError: Fa...true : Scalars failed to compare as equal! -6 != 0

2021-09-10T17:22:53.7461648Z ----------------------------------------------------------------------
2021-09-10T17:22:53.7462491Z Traceback (most recent call last):
2021-09-10T17:22:53.7463891Z   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_distributed.py", line 418, in wrapper
2021-09-10T17:22:53.7465039Z     self._join_processes(fn)
2021-09-10T17:22:53.7466414Z   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_distributed.py", line 637, in _join_processes
2021-09-10T17:22:53.7467701Z     self._check_return_codes(elapsed_time)
2021-09-10T17:22:53.7469297Z   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_distributed.py", line 692, in _check_return_codes
2021-09-10T17:22:53.7470550Z     self.assertEqual(
2021-09-10T17:22:53.7471942Z   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_utils.py", line 1676, in assertEqual
2021-09-10T17:22:53.7473352Z     super().assertTrue(result, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
2021-09-10T17:22:53.7474802Z AssertionError: False is not true : Scalars failed to compare as equal! -6 != 0
2021-09-10T17:22:53.7476164Z Expect process 2 exit code to match Process 0 exit code of 0, but got -6
2021-09-10T17:22:53.7476810Z 
2021-09-10T17:22:53.7477680Z ----------------------------------------------------------------------
2021-09-10T17:22:53.7478438Z Ran 85 tests in 110.092s
2021-09-10T17:22:53.7478815Z 
2021-09-10T17:22:53.7479365Z FAILED (failures=1, skipped=31)
2021-09-10T17:22:53.7479832Z 
2021-09-10T17:22:53.7480393Z Generating XML reports...
2021-09-10T17:22:53.7496766Z Generated XML report: test-reports/python-unittest/distributed.test_c10d_gloo/TEST-CommTest-20210910172103.xml
2021-09-10T17:22:53.7552631Z Generated XML report: test-reports/python-unittest/distributed.test_c10d_gloo/TEST-DistributedDataParallelTest-20210910172103.xml

linux-bionic-py3.6-clang9 / test (default, 2, 2, linux.2xlarge) (4/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T16:49:47.5487908Z RuntimeError: test_fx failed!

2021-09-10T16:49:46.5272651Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFXAPIBackwardCompatibility-20210910164835.xml
2021-09-10T16:49:46.5368242Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFunctionalTracing-20210910164835.xml
2021-09-10T16:49:46.5657238Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestOperatorSignaturesCPU-20210910164835.xml
2021-09-10T16:49:46.5671148Z Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_subgraph_rewriter.TestSubgraphRewriter-20210910164835.xml
2021-09-10T16:49:46.5715658Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestVisionTracing-20210910164835.xml
2021-09-10T16:49:47.5482282Z Traceback (most recent call last):
2021-09-10T16:49:47.5482795Z   File "test/run_test.py", line 1023, in <module>
2021-09-10T16:49:47.5485110Z     main()
2021-09-10T16:49:47.5485475Z   File "test/run_test.py", line 1001, in main
2021-09-10T16:49:47.5487381Z     raise RuntimeError(err_message)
2021-09-10T16:49:47.5487908Z RuntimeError: test_fx failed!
2021-09-10T16:49:47.8013559Z 
2021-09-10T16:49:47.8014303Z real	4m33.920s
2021-09-10T16:49:47.8014852Z user	10m27.017s
2021-09-10T16:49:47.8015212Z sys	0m17.445s
2021-09-10T16:49:47.8015508Z + cleanup
2021-09-10T16:49:47.8015796Z + retcode=1
2021-09-10T16:49:47.8016098Z + set +x
2021-09-10T16:49:47.8016534Z =================== sccache compilation log ===================
2021-09-10T16:49:47.8222205Z =========== If your build fails, please take a look at the log above for possible reasons ===========
2021-09-10T16:49:47.8245641Z Compile requests                      28

linux-bionic-py3.6-clang9 / test (default, 1, 2, linux.2xlarge) (5/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T16:48:55.4121188Z AssertionError: Items in the second set but not the first:

2021-09-10T16:48:55.4110371Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 766, in test_wrapper
2021-09-10T16:48:55.4111378Z     return test(*args, **kwargs)
2021-09-10T16:48:55.4112588Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 801, in dep_fn
2021-09-10T16:48:55.4113565Z     return fn(slf, *args, **kwargs)
2021-09-10T16:48:55.4114765Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 949, in only_fn
2021-09-10T16:48:55.4115765Z     return fn(self, *args, **kwargs)
2021-09-10T16:48:55.4116407Z   File "test_ops.py", line 173, in test_dtypes
2021-09-10T16:48:55.4117396Z     self.assertEqual(supported_backward_dtypes, claimed_backward_supported, msg=msg)
2021-09-10T16:48:55.4118956Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 1639, in assertEqual
2021-09-10T16:48:55.4120227Z     super().assertEqual(x, y, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
2021-09-10T16:48:55.4121188Z AssertionError: Items in the second set but not the first:
2021-09-10T16:48:55.4121999Z torch.float16
2021-09-10T16:48:55.4122555Z torch.complex64
2021-09-10T16:48:55.4123097Z torch.float32
2021-09-10T16:48:55.4123635Z torch.bfloat16
2021-09-10T16:48:55.4124173Z torch.float64
2021-09-10T16:48:55.4125627Z torch.complex128 : Attempted to compare [set] types: Expected: set(); Actual: {torch.float16, torch.complex64, torch.float32, torch.bfloat16, torch.float64, torch.complex128}.
2021-09-10T16:48:55.4127463Z The supported backward dtypes for linalg.tensordot on cpu according to its OpInfo are
2021-09-10T16:48:55.4129139Z         {torch.float16, torch.complex64, torch.float32, torch.bfloat16, torch.float64, torch.complex128}, but the detected supported backward dtypes are set().
2021-09-10T16:48:55.4131322Z         The following backward dtypes should be removed from the OpInfo: {torch.float16, torch.complex64, torch.float32, torch.bfloat16, torch.float64, torch.complex128}.
2021-09-10T16:48:55.4132505Z

linux-bionic-py3.6-clang9 / test (noarch, 1, 1, linux.2xlarge) (6/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T16:52:08.8624797Z RuntimeError: test_fx failed!

2021-09-10T16:52:07.7994265Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFunctionalTracing-20210910165055.xml
2021-09-10T16:52:07.8275491Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestOperatorSignaturesCPU-20210910165055.xml
2021-09-10T16:52:07.8289510Z Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_subgraph_rewriter.TestSubgraphRewriter-20210910165055.xml
2021-09-10T16:52:07.8332992Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestVisionTracing-20210910165055.xml
2021-09-10T16:52:07.8698470Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestOperatorSignaturesMETA-20210910165055.xml
2021-09-10T16:52:08.8619553Z Traceback (most recent call last):
2021-09-10T16:52:08.8620073Z   File "test/run_test.py", line 1023, in <module>
2021-09-10T16:52:08.8622227Z     main()
2021-09-10T16:52:08.8622639Z   File "test/run_test.py", line 1001, in main
2021-09-10T16:52:08.8624321Z     raise RuntimeError(err_message)
2021-09-10T16:52:08.8624797Z RuntimeError: test_fx failed!
2021-09-10T16:52:09.1439399Z 
2021-09-10T16:52:09.1440040Z real	6m53.850s
2021-09-10T16:52:09.1440555Z user	11m9.579s
2021-09-10T16:52:09.1441011Z sys	0m55.523s
2021-09-10T16:52:09.1441397Z + cleanup
2021-09-10T16:52:09.1441901Z + retcode=1
2021-09-10T16:52:09.1443149Z + set +x
2021-09-10T16:52:09.1443807Z =================== sccache compilation log ===================
2021-09-10T16:52:09.1649257Z =========== If your build fails, please take a look at the log above for possible reasons ===========
2021-09-10T16:52:09.1672422Z Compile requests                      34

linux-bionic-py3.8-gcc9-coverage / test (default, 1, 2, linux.2xlarge) (7/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T17:06:11.6104668Z RuntimeError: test_linalg failed!

2021-09-10T17:06:10.5087620Z 
2021-09-10T17:06:10.5087956Z FAILED (errors=2, skipped=55)
2021-09-10T17:06:10.5088205Z 
2021-09-10T17:06:10.5088549Z Generating XML reports...
2021-09-10T17:06:11.0086414Z Generated XML report: test-reports/python-unittest/test_linalg/TEST-TestLinalgCPU-20210910170342.xml
2021-09-10T17:06:11.6101216Z Traceback (most recent call last):
2021-09-10T17:06:11.6101898Z   File "test/run_test.py", line 1023, in <module>
2021-09-10T17:06:11.6102854Z     main()
2021-09-10T17:06:11.6103496Z   File "test/run_test.py", line 1001, in main
2021-09-10T17:06:11.6104189Z     raise RuntimeError(err_message)
2021-09-10T17:06:11.6104668Z RuntimeError: test_linalg failed!
2021-09-10T17:06:12.0414057Z 
2021-09-10T17:06:12.0415150Z real	7m48.854s
2021-09-10T17:06:12.0415702Z user	16m28.954s
2021-09-10T17:06:12.0416178Z sys	0m11.340s
2021-09-10T17:06:12.0416612Z + cleanup
2021-09-10T17:06:12.0417349Z + retcode=1
2021-09-10T17:06:12.0417809Z + set +x
2021-09-10T17:06:12.0418404Z =================== sccache compilation log ===================
2021-09-10T17:06:12.0604645Z =========== If your build fails, please take a look at the log above for possible reasons ===========
2021-09-10T17:06:12.0626711Z Compile requests                      30

linux-xenial-py3.6-gcc5.4 / test (default, 1, 2, linux.2xlarge) (8/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T16:45:11.3166040Z RuntimeError: test_fx failed!

2021-09-10T16:45:10.2875318Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFXAPIBackwardCompatibility-20210910164357.xml
2021-09-10T16:45:10.2969412Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFunctionalTracing-20210910164357.xml
2021-09-10T16:45:10.3250533Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestOperatorSignaturesCPU-20210910164357.xml
2021-09-10T16:45:10.3265556Z Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_subgraph_rewriter.TestSubgraphRewriter-20210910164357.xml
2021-09-10T16:45:10.3310195Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestVisionTracing-20210910164357.xml
2021-09-10T16:45:11.3160801Z Traceback (most recent call last):
2021-09-10T16:45:11.3161349Z   File "test/run_test.py", line 1023, in <module>
2021-09-10T16:45:11.3163237Z     main()
2021-09-10T16:45:11.3163630Z   File "test/run_test.py", line 1001, in main
2021-09-10T16:45:11.3165518Z     raise RuntimeError(err_message)
2021-09-10T16:45:11.3166040Z RuntimeError: test_fx failed!
2021-09-10T16:45:11.5479730Z + cleanup
2021-09-10T16:45:11.5480619Z + retcode=1
2021-09-10T16:45:11.5480887Z + set +x
2021-09-10T16:45:11.5481233Z =================== sccache compilation log ===================
2021-09-10T16:45:11.5674226Z =========== If your build fails, please take a look at the log above for possible reasons ===========
2021-09-10T16:45:11.5694199Z Compile requests                     28
2021-09-10T16:45:11.5694834Z Compile requests executed            26
2021-09-10T16:45:11.5695398Z Cache hits                            1
2021-09-10T16:45:11.5695928Z Cache hits (C/C++)                    1
2021-09-10T16:45:11.5696443Z Cache misses                         25

linux-bionic-py3.8-gcc9-coverage / test (default, 2, 2, linux.2xlarge) (9/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T17:04:23.6244974Z RuntimeError: test_fx failed!

2021-09-10T17:04:22.1389254Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFunctionalTracing-20210910165826.xml
2021-09-10T17:04:22.2703632Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestOperatorSignaturesCPU-20210910165826.xml
2021-09-10T17:04:22.2732296Z Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_subgraph_rewriter.TestSubgraphRewriter-20210910165826.xml
2021-09-10T17:04:22.2833990Z Generated XML report: test-reports/python-unittest/test_fx/TEST-TestVisionTracing-20210910165826.xml
2021-09-10T17:04:22.2898341Z Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_gradual_type.TypeCheckerTest-20210910165826.xml
2021-09-10T17:04:23.6241163Z Traceback (most recent call last):
2021-09-10T17:04:23.6242099Z   File "test/run_test.py", line 1023, in <module>
2021-09-10T17:04:23.6242709Z     main()
2021-09-10T17:04:23.6243281Z   File "test/run_test.py", line 1001, in main
2021-09-10T17:04:23.6244455Z     raise RuntimeError(err_message)
2021-09-10T17:04:23.6244974Z RuntimeError: test_fx failed!
2021-09-10T17:04:24.0815111Z 
2021-09-10T17:04:24.0815863Z real	6m2.620s
2021-09-10T17:04:24.0816386Z user	8m25.201s
2021-09-10T17:04:24.0816802Z sys	0m6.421s
2021-09-10T17:04:24.0817074Z + cleanup
2021-09-10T17:04:24.0817357Z + retcode=1
2021-09-10T17:04:24.0817618Z + set +x
2021-09-10T17:04:24.0817988Z =================== sccache compilation log ===================
2021-09-10T17:04:24.1011269Z =========== If your build fails, please take a look at the log above for possible reasons ===========
2021-09-10T17:04:24.1034853Z Compile requests                      28

linux-xenial-py3.6-gcc5.4 / test (default, 2, 2, linux.2xlarge) (10/15)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T16:47:02.1398557Z AssertionError: Items in the second set but not the first:

2021-09-10T16:47:02.1391692Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 766, in test_wrapper
2021-09-10T16:47:02.1392328Z     return test(*args, **kwargs)
2021-09-10T16:47:02.1393237Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 801, in dep_fn
2021-09-10T16:47:02.1393865Z     return fn(slf, *args, **kwargs)
2021-09-10T16:47:02.1394632Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 949, in only_fn
2021-09-10T16:47:02.1395233Z     return fn(self, *args, **kwargs)
2021-09-10T16:47:02.1395654Z   File "test_ops.py", line 173, in test_dtypes
2021-09-10T16:47:02.1396257Z     self.assertEqual(supported_backward_dtypes, claimed_backward_supported, msg=msg)
2021-09-10T16:47:02.1397225Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 1639, in assertEqual
2021-09-10T16:47:02.1397965Z     super().assertEqual(x, y, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
2021-09-10T16:47:02.1398557Z AssertionError: Items in the second set but not the first:
2021-09-10T16:47:02.1399003Z torch.float16
2021-09-10T16:47:02.1399319Z torch.float32
2021-09-10T16:47:02.1399654Z torch.bfloat16
2021-09-10T16:47:02.1399987Z torch.complex64
2021-09-10T16:47:02.1400329Z torch.float64
2021-09-10T16:47:02.1401224Z torch.complex128 : Attempted to compare [set] types: Expected: set(); Actual: {torch.float16, torch.float32, torch.bfloat16, torch.complex64, torch.float64, torch.complex128}.
2021-09-10T16:47:02.1402307Z The supported backward dtypes for linalg.tensordot on cpu according to its OpInfo are
2021-09-10T16:47:02.1403378Z         {torch.float16, torch.float32, torch.bfloat16, torch.complex64, torch.float64, torch.complex128}, but the detected supported backward dtypes are set().
2021-09-10T16:47:02.1404681Z         The following backward dtypes should be removed from the OpInfo: {torch.float16, torch.float32, torch.bfloat16, torch.complex64, torch.float64, torch.complex128}.
2021-09-10T16:47:02.1405397Z

pytorch_linux_xenial_py3_clang7_asan_test2 (11/15)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Sep 10 18:46:34 AssertionError: Items in the second set but not the first:

Sep 10 18:46:34   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 765, in test_wrapper
Sep 10 18:46:34     return test(*args, **kwargs)
Sep 10 18:46:34   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 800, in dep_fn
Sep 10 18:46:34     return fn(slf, *args, **kwargs)
Sep 10 18:46:34   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 948, in only_fn
Sep 10 18:46:34     return fn(self, *args, **kwargs)
Sep 10 18:46:34   File "test_ops.py", line 173, in test_dtypes
Sep 10 18:46:34     self.assertEqual(supported_backward_dtypes, claimed_backward_supported, msg=msg)
Sep 10 18:46:34   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 1607, in assertEqual
Sep 10 18:46:34     super().assertEqual(x, y, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
Sep 10 18:46:34 AssertionError: Items in the second set but not the first:
Sep 10 18:46:34 torch.float16
Sep 10 18:46:34 torch.float32
Sep 10 18:46:34 torch.bfloat16
Sep 10 18:46:34 torch.complex128
Sep 10 18:46:34 torch.float64
Sep 10 18:46:34 torch.complex64 : Attempted to compare [set] types: Expected: set(); Actual: {torch.float16, torch.float32, torch.bfloat16, torch.complex128, torch.float64, torch.complex64}.
Sep 10 18:46:34 The supported backward dtypes for linalg.tensordot on cpu according to its OpInfo are
Sep 10 18:46:34         {torch.float16, torch.float32, torch.bfloat16, torch.complex128, torch.float64, torch.complex64}, but the detected supported backward dtypes are set().
Sep 10 18:46:34         The following backward dtypes should be removed from the OpInfo: {torch.float16, torch.float32, torch.bfloat16, torch.complex128, torch.float64, torch.complex64}.
Sep 10 18:46:34

pytorch_macos_10_13_py3_test (12/15)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Sep 10 17:10:36 RuntimeError: test_fx failed!

Sep 10 17:10:36 Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFXAPIBackwardCompatibility-20210910171031.xml
Sep 10 17:10:36 Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFunctionalTracing-20210910171031.xml
Sep 10 17:10:36 Generated XML report: test-reports/python-unittest/test_fx/TEST-TestOperatorSignaturesCPU-20210910171031.xml
Sep 10 17:10:36 Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_subgraph_rewriter.TestSubgraphRewriter-20210910171031.xml
Sep 10 17:10:36 Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_gradual_type.TypeCheckerTest-20210910171031.xml
Sep 10 17:10:36 Traceback (most recent call last):
Sep 10 17:10:36   File "test/run_test.py", line 1023, in <module>
Sep 10 17:10:36     main()
Sep 10 17:10:36   File "test/run_test.py", line 1001, in main
Sep 10 17:10:36     raise RuntimeError(err_message)
Sep 10 17:10:36 RuntimeError: test_fx failed!
Sep 10 17:10:37 + cleanup
Sep 10 17:10:37 + retcode=1
Sep 10 17:10:37 + set +x


Exited with code exit status 1

pytorch_linux_xenial_py3_6_gcc5_4_test (13/15)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Sep 10 17:30:40 RuntimeError: test_linalg failed!

Sep 10 17:30:40 
Sep 10 17:30:40 FAILED (errors=2, skipped=55)
Sep 10 17:30:40 
Sep 10 17:30:40 Generating XML reports...
Sep 10 17:30:40 Generated XML report: test-reports/python-unittest/test_linalg/TEST-TestLinalgCPU-20210910172843.xml
Sep 10 17:30:40 Traceback (most recent call last):
Sep 10 17:30:40   File "test/run_test.py", line 1023, in <module>
Sep 10 17:30:40     main()
Sep 10 17:30:40   File "test/run_test.py", line 1001, in main
Sep 10 17:30:40     raise RuntimeError(err_message)
Sep 10 17:30:40 RuntimeError: test_linalg failed!
Sep 10 17:30:41 + cleanup
Sep 10 17:30:41 + retcode=1
Sep 10 17:30:41 + set +x
Sep 10 17:30:41 =================== sccache compilation log ===================
Sep 10 17:30:41 =========== If your build fails, please take a look at the log above for possible reasons ===========
Sep 10 17:30:41 Compile requests                      30
Sep 10 17:30:41 Compile requests executed             26
Sep 10 17:30:41 Cache hits                             2
Sep 10 17:30:41 Cache hits (C/C++)                     2
Sep 10 17:30:41 Cache misses                          24

pytorch_xla_linux_bionic_py3_6_clang9_test (14/15)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Sep 10 18:09:47 ERROR [0.011s]: test_reference_eager_inner_xla_float32 (__main__.TestOpInfoXLA)

Sep 10 18:09:22 X = torch.solve(B, A).solution
Sep 10 18:09:22 should be replaced with
Sep 10 18:09:22 X = torch.linalg.solve(A, B) (Triggered internally at  /var/lib/jenkins/workspace/aten/src/ATen/native/BatchLinearAlgebra.cpp:766.)
Sep 10 18:09:22   return self.op(*args, **kwargs)
Sep 10 18:09:40 ................................................./opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_methods_invocations.py:8750: UserWarning: 0The operator aten::unfold appears to be a view operator, but it has no implementation for the backend "xla:0". View operators don't support falling back to run on the CPU, since the tensor's storage cannot be shared across devices. (Triggered internally at  /var/lib/jenkins/workspace/aten/src/ATen/native/CPUFallback.cpp:173.)
Sep 10 18:09:40   op=lambda x, *args: x.unfold(*args),
Sep 10 18:09:44 ....../opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_methods_invocations.py:648: UserWarning: 0The operator aten::view_as_complex appears to be a view operator, but it has no implementation for the backend "xla:0". View operators don't support falling back to run on the CPU, since the tensor's storage cannot be shared across devices. (Triggered internally at  /var/lib/jenkins/workspace/aten/src/ATen/native/CPUFallback.cpp:173.)
Sep 10 18:09:44   return self.op(*args, **kwargs)
Sep 10 18:09:47 ...............
Sep 10 18:09:47 ======================================================================
Sep 10 18:09:47 ERROR [0.011s]: test_reference_eager_inner_xla_float32 (__main__.TestOpInfoXLA)
Sep 10 18:09:47 ----------------------------------------------------------------------
Sep 10 18:09:47 Traceback (most recent call last):
Sep 10 18:09:47   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 378, in instantiated_test
Sep 10 18:09:47     raise rte
Sep 10 18:09:47   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 373, in instantiated_test
Sep 10 18:09:47     result = test(self, **param_kwargs)
Sep 10 18:09:47   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 766, in test_wrapper
Sep 10 18:09:47     return test(*args, **kwargs)
Sep 10 18:09:47   File "/var/lib/jenkins/workspace/xla/test/test_ops.py", line 411, in test_reference_eager
Sep 10 18:09:47     self.compare_with_eager_reference(op, sample_input)

pytorch_linux_xenial_py3_clang7_asan_test1 (15/15)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Sep 10 18:47:04 RuntimeError: test_fx failed!

Sep 10 18:47:02 Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFXAPIBackwardCompatibility-20210910184022.xml
Sep 10 18:47:02 Generated XML report: test-reports/python-unittest/test_fx/TEST-TestFunctionalTracing-20210910184022.xml
Sep 10 18:47:02 Generated XML report: test-reports/python-unittest/test_fx/TEST-TestOperatorSignaturesCPU-20210910184022.xml
Sep 10 18:47:02 Generated XML report: test-reports/python-unittest/test_fx/TEST-fx.test_subgraph_rewriter.TestSubgraphRewriter-20210910184022.xml
Sep 10 18:47:02 Generated XML report: test-reports/python-unittest/test_fx/TEST-TestVisionTracing-20210910184022.xml
Sep 10 18:47:04 Traceback (most recent call last):
Sep 10 18:47:04   File "test/run_test.py", line 1023, in <module>
Sep 10 18:47:04     main()
Sep 10 18:47:04   File "test/run_test.py", line 1001, in main
Sep 10 18:47:04     raise RuntimeError(err_message)
Sep 10 18:47:04 RuntimeError: test_fx failed!
Sep 10 18:47:05 + cleanup
Sep 10 18:47:05 + retcode=1
Sep 10 18:47:05 + set +x
Sep 10 18:47:05 =================== sccache compilation log ===================
Sep 10 18:47:05 =========== If your build fails, please take a look at the log above for possible reasons ===========
Sep 10 18:47:05 Compile requests                      0
Sep 10 18:47:05 Compile requests executed             0
Sep 10 18:47:05 Cache hits                            0
Sep 10 18:47:05 Cache misses                          0
Sep 10 18:47:05 Cache timeouts                        0

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm4.2-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

github-actions · 2022-04-13T17:38:43Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

antocuni added 30 commits August 18, 2021 12:33

introduce linalg.tensordot and make torch.tensordot an alias to it

7fd3e17

replace torch.tensordot with torch.linalg.tensordot everywhere

1278440

make linalg.tensordot the main op and torch.tensordot an alias to it …

ffbeff1

…in common_method_invocations

fix mypy

5b1902f

intern this string

d48e254

fix docs, and use torch.linalg.tensordot everywhere

0d17d66

fix ONNX

335bbcd

fix tensordot autocast

544efae

add the new alias to the alias map

7e7cb83

move the implementation of tensordot from functional.py to linalg/__i…

3c11c7d

…nit__.py

tentatively try to teach the JIT about the new linalg.tensordot opera…

a1516d9

…tion

torch.functional.tensordot is explictly listed in the torchscript doc…

2380bfa

…s. Make sure to preserve compatibility

Merge branch 'master' into antocuni/linalg-tensordot

07350b5

reduce the diff w.r.t. master

96780a1

put tensordot first

af85b06

kill leftover

e873e14

rename test_functional_tensordot_* into test_linalg_tensordot_*

2598273

improve test_tensordot by testing also the variants in which dims is …

053beac

…a Tensor

improve the test to use multiple dtypes

e7fc1ea

fix test_fx

90e37ce

fix flake8

362e380

fix mypy

ecedda4

torch.linalg imports _overload as overload so it looks like a public …

ecb7e90

…function, but actually it's not

Merge branch 'master' into antocuni/linalg-tensordot

0f3c83d

fix

6338349

fix flake8

4ef63cd

try to fix flake8 again

32a1b0a

WIP

ece88bd

simplify

d9c604d

don't compute rsize twice

26b1e9f

antocuni added 3 commits September 10, 2021 14:41

one more small step towards implementing linalg_tensordot in terms of…

5ae2e26

… linalg_tensordot_out, and not vice-versa

implement linalg_tensordot_out in terms of helper

91a8ce7

finally implement linalg_tensordot as a structured kernel

00ad7db

antocuni requested review from BowenBao, IvanYashchuk, ezyang, lezcano, neginraoof, nikitaved and shubhambhokare1 as code owners September 10, 2021 15:28

facebook-github-bot added cla signed oncall: jit Add this issue/PR to JIT oncall triage queue labels Sep 10, 2021

facebook-github-bot added the module: fx label Sep 10, 2021

antocuni marked this pull request as draft September 10, 2021 15:30

antocuni removed request for BowenBao, IvanYashchuk, ezyang, lezcano, neginraoof, nikitaved and shubhambhokare1 September 10, 2021 15:30

antocuni mentioned this pull request Sep 10, 2021

Create linalg.tensordot #63478

Closed

pytorchbot added the open source label Sep 10, 2021

github-actions Bot added the Stale label Apr 13, 2022

github-actions Bot closed this May 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Port `linalg.tensordot` to structured kernels#64819

WIP: Port `linalg.tensordot` to structured kernels#64819
antocuni wants to merge 33 commits intopytorch:masterfrom
antocuni:antocuni/linalg-tensordot-structured

antocuni commented Sep 10, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Sep 10, 2021 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	if f.structured_delegate is not None:
	# TODO: for ops with structured_delegate it should check the dispatch table of
	# the out variant instead. For now, these structured ops all have CPU/CUDA kernels
	# so we always dispatch to the `backend`, but this could be wrong when we
	# migrate math/default_backend ops to use structured delegate.
	return f'return at::{backend_index.dispatch_key.lower()}::{name}({exprs_str});'

Conversation

antocuni commented Sep 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Sep 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

🕵️ 15 new failures recognized by patterns

win-vs2019-cpu-py3 / test (default, 2, 2, windows.4xlarge) (1/15)

win-vs2019-cpu-py3 / test (default, 1, 2, windows.4xlarge) (2/15)

linux-bionic-py3.8-gcc9-coverage / test (distributed, 1, 1, linux.2xlarge) (3/15)

linux-bionic-py3.6-clang9 / test (default, 2, 2, linux.2xlarge) (4/15)

linux-bionic-py3.6-clang9 / test (default, 1, 2, linux.2xlarge) (5/15)

linux-bionic-py3.6-clang9 / test (noarch, 1, 1, linux.2xlarge) (6/15)

linux-bionic-py3.8-gcc9-coverage / test (default, 1, 2, linux.2xlarge) (7/15)

linux-xenial-py3.6-gcc5.4 / test (default, 1, 2, linux.2xlarge) (8/15)

linux-bionic-py3.8-gcc9-coverage / test (default, 2, 2, linux.2xlarge) (9/15)

linux-xenial-py3.6-gcc5.4 / test (default, 2, 2, linux.2xlarge) (10/15)

pytorch_linux_xenial_py3_clang7_asan_test2 (11/15)

pytorch_macos_10_13_py3_test (12/15)

pytorch_linux_xenial_py3_6_gcc5_4_test (13/15)

pytorch_xla_linux_bionic_py3_6_clang9_test (14/15)

pytorch_linux_xenial_py3_clang7_asan_test1 (15/15)

ci.pytorch.org: 1 failed

Uh oh!

github-actions Bot commented Apr 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

antocuni commented Sep 10, 2021 •

edited

Loading

facebook-github-bot commented Sep 10, 2021 •

edited

Loading