Add errors if we try to use PyTorch with Python 2 by dreiss · Pull Request #35611 · pytorch/pytorch

dreiss · 2020-03-28T20:45:55Z

Stack from ghstack:

Remove structseq_slice #35625 Remove structseq_slice
Remove (most) Python 2 support from C++ code #35614 Remove (most) Python 2 support from C++ code
Add errors if we try to use PyTorch with Python 2 #35611 Add errors if we try to use PyTorch with Python 2
Remove __future__ imports from many files #35627 Remove future imports from many files
Remove a use of exec #35624 Remove a use of exec
Remove a use of checkScript(str) #35623 Remove a use of checkScript(str)
Remove caffe2/python/compatibility #35622 Remove caffe2/python/compatibility
Remove _get_wrapped_func #35621 Remove _get_wrapped_func
Remove _compatible_subtest #35620 Remove _compatible_subtest
Remove _reset_warning_registry #35619 Remove _reset_warning_registry
Remove _uses_true_division #35618 Remove _uses_true_division
Remove (most) Python 2 support from setup.py #35617 Remove (most) Python 2 support from setup.py
Remove (most) Python 2 support from CI scripts #35616 Remove (most) Python 2 support from CI scripts
Remove (most) Python 2 support from Python code #35615 Remove (most) Python 2 support from Python code

Summary:
Python 2 has reached end-of-life and is no longer supported by PyTorch.
To avoid confusing behavior when trying to use PyTorch with Python 2,
detect this case early and fail with a clear message. This commit
covers "import torch" and attempting to build torch/csrc.

Test Plan:

Attempted to "import torch" with Python 2 and saw a clear error.
Attempted to build PyTorch with Python 2 and saw a clear error.

Differential Revision: D20842878

dr-ci · 2020-03-28T20:55:37Z

💊 Build failures summary and remediations

As of commit 7f7bed7 (more details on the Dr. CI page):

6/6 failures possibly* introduced in this PR
- 1/6 non-CircleCI failure(s)

🕵️ 5 new failures recognized by patterns

The following build failures do not appear to be due to upstream breakages:

pytorch_linux_xenial_py3_6_gcc5_4_test (1/5)

Step: "Test" (full log | pattern match details | 🔁 rerun)

Apr 22 06:24:40 RuntimeError: test_overrides failed!

Apr 22 06:24:40   File "/opt/conda/lib/python3.6/site-packages/torch/_overrides.py", line 810, in get_overridable_functions 
Apr 22 06:24:40     if func in get_ignored_functions(): 
Apr 22 06:24:40   File "/opt/conda/lib/python3.6/site-packages/torch/_overrides.py", line 125, in get_ignored_functions 
Apr 22 06:24:40     torch.nn.functional.division, 
Apr 22 06:24:40 AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Apr 22 06:24:40 Traceback (most recent call last): 
Apr 22 06:24:40   File "test/run_test.py", line 692, in <module> 
Apr 22 06:24:40     main() 
Apr 22 06:24:40   File "test/run_test.py", line 685, in main 
Apr 22 06:24:40     raise RuntimeError(message) 
Apr 22 06:24:40 RuntimeError: test_overrides failed! 
Apr 22 06:24:41 =================== sccache compilation log =================== 
Apr 22 06:24:41 + cleanup 
Apr 22 06:24:41 + retcode=1 
Apr 22 06:24:41 + set +x 
Apr 22 06:24:41 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/tmp/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/tmp/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
Apr 22 06:24:41  
Apr 22 06:24:41 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Apr 22 06:24:41 Compile requests                 79 
Apr 22 06:24:41 Compile requests executed        34 
Apr 22 06:24:41 Cache hits                       26

pytorch_macos_10_13_py3_test (2/5)

Step: "Test" (full log | pattern match details | 🔁 rerun)

Apr 21 23:32:52 RuntimeError: test_overrides failed!

Apr 21 23:32:52   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/_overrides.py", line 810, in get_overridable_functions 
Apr 21 23:32:52     if func in get_ignored_functions(): 
Apr 21 23:32:52   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/_overrides.py", line 125, in get_ignored_functions 
Apr 21 23:32:52     torch.nn.functional.division, 
Apr 21 23:32:52 AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Apr 21 23:32:52 Traceback (most recent call last): 
Apr 21 23:32:52   File "test/run_test.py", line 692, in <module> 
Apr 21 23:32:52     main() 
Apr 21 23:32:52   File "test/run_test.py", line 685, in main 
Apr 21 23:32:52     raise RuntimeError(message) 
Apr 21 23:32:52 RuntimeError: test_overrides failed! 
Apr 21 23:32:52 + cleanup 
Apr 21 23:32:52 + retcode=1 
Apr 21 23:32:52 + set +x

pytorch_windows_vs2019_py36_cuda10.1_test2 (3/5)

Step: "Test" (full log | pattern match details | 🔁 rerun)

RuntimeError: test_overrides failed!

  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 810, in get_overridable_functions 
    if func in get_ignored_functions(): 
  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 125, in get_ignored_functions 
    torch.nn.functional.division, 
AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Traceback (most recent call last): 
  File "run_test.py", line 692, in <module> 
    main() 
  File "run_test.py", line 685, in main 
    raise RuntimeError(message) 
RuntimeError: test_overrides failed! 
 
(base) circleci@PACKER-5E9E037A C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1  
+ cleanup
+ retcode=1
+ set +x

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (4/5)

Step: "Test" (full log | pattern match details | 🔁 rerun)

Apr 22 07:10:04 RuntimeError: test_overrides failed!

Apr 22 07:10:04   File "/opt/conda/lib/python3.6/site-packages/torch/_overrides.py", line 810, in get_overridable_functions 
Apr 22 07:10:04     if func in get_ignored_functions(): 
Apr 22 07:10:04   File "/opt/conda/lib/python3.6/site-packages/torch/_overrides.py", line 125, in get_ignored_functions 
Apr 22 07:10:04     torch.nn.functional.division, 
Apr 22 07:10:04 AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Apr 22 07:10:04 Traceback (most recent call last): 
Apr 22 07:10:04   File "test/run_test.py", line 692, in <module> 
Apr 22 07:10:04     main() 
Apr 22 07:10:04   File "test/run_test.py", line 685, in main 
Apr 22 07:10:04     raise RuntimeError(message) 
Apr 22 07:10:04 RuntimeError: test_overrides failed! 
Apr 22 07:10:05 + cleanup 
Apr 22 07:10:05 + retcode=1 
Apr 22 07:10:05 + set +x 
Apr 22 07:10:05 =================== sccache compilation log =================== 
Apr 22 07:10:05 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Apr 22 07:10:05 Compile requests                15 
Apr 22 07:10:05 Compile requests executed        0 
Apr 22 07:10:05 Cache hits                       0 
Apr 22 07:10:05 Cache misses                     0 
Apr 22 07:10:05 Cache timeouts                   0

pytorch_linux_xenial_py3_clang5_asan_test (5/5)

Step: "Test" (full log | pattern match details | 🔁 rerun)

Apr 22 07:33:03 RuntimeError: test_overrides failed!

Apr 22 07:33:03   File "/opt/conda/lib/python3.6/site-packages/torch/_overrides.py", line 810, in get_overridable_functions 
Apr 22 07:33:03     if func in get_ignored_functions(): 
Apr 22 07:33:03   File "/opt/conda/lib/python3.6/site-packages/torch/_overrides.py", line 125, in get_ignored_functions 
Apr 22 07:33:03     torch.nn.functional.division, 
Apr 22 07:33:03 AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Apr 22 07:33:03 Traceback (most recent call last): 
Apr 22 07:33:03   File "test/run_test.py", line 692, in <module> 
Apr 22 07:33:03     main() 
Apr 22 07:33:03   File "test/run_test.py", line 685, in main 
Apr 22 07:33:03     raise RuntimeError(message) 
Apr 22 07:33:03 RuntimeError: test_overrides failed! 
Apr 22 07:33:03 + cleanup 
Apr 22 07:33:03 + retcode=1 
Apr 22 07:33:03 + set +x 
Apr 22 07:33:03 =================== sccache compilation log =================== 
Apr 22 07:33:03 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Apr 22 07:33:03 Compile requests                13 
Apr 22 07:33:03 Compile requests executed        0 
Apr 22 07:33:03 Cache hits                       0 
Apr 22 07:33:03 Cache misses                     0 
Apr 22 07:33:03 Cache timeouts                   0

Extra GitHub checks: 1 failed

Failed: GitHub Actions - flake8-py3

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 40 times.

dreiss · 2020-03-31T19:11:40Z

@soumith , I'm confused by this failure. Do you know if binary_linux_libtorch_2_7m_cpu_gcc5_4_cxx11-abi_shared-with-deps_build is supposed to be a Pythonless libtorch build? If so, do you know why it is building a file that includes torch/csrc/python_headers.h?

ezyang · 2020-03-31T21:31:09Z

I'm not convinced.

Re (1), the user is more likely to run into a syntax error first when they try to import PyTorch. I'm not sure the runtime exception is buying you all that much. And we can prevent people from trying to install PyTorch on Python 2 by advertising appropriately in setup.py that it's python 3 only.

Re (2), shouldn't cmake check Python version and error out earlier than when you try to compile?

dreiss · 2020-03-31T22:50:11Z

Later diffs in the stack update cmake and setup.py. This is mostly useful for internal builds that don't use those.

Summary: Python 2 has reached end-of-life and is no longer supported by PyTorch. To avoid confusing behavior when trying to use PyTorch with Python 2, detect this case early and fail with a clear message. This commit covers "import torch" and attempting to build torch/csrc. Test Plan: - Attempted to "import torch" with Python 2 and saw a clear error. - Attempted to build PyTorch with Python 2 and saw a clear error. Differential Revision: [D20842878](https://our.internmc.facebook.com/intern/diff/D20842878)

Summary: Python 2 has reached end-of-life and is no longer supported by PyTorch. To avoid confusing behavior when trying to use PyTorch with Python 2, detect this case early and fail with a clear message. This commit covers "import torch" and attempting to build torch/csrc. Test Plan: - Attempted to "import torch" with Python 2 and saw a clear error. - Attempted to build PyTorch with Python 2 and saw a clear error. Differential Revision: [D20842878](https://our.internmc.facebook.com/intern/diff/D20842878) [ghstack-poisoned]

github-actions · 2022-04-12T19:52:30Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

dreiss requested review from ezyang and soumith March 31, 2020 15:32

ezyang approved these changes Apr 1, 2020

View reviewed changes

soumith approved these changes Apr 2, 2020

View reviewed changes

dreiss added 5 commits April 15, 2020 10:27

facebook-github-bot added the cla signed label Oct 30, 2020

github-actions Bot added the Stale label Apr 12, 2022

github-actions Bot closed this May 12, 2022

facebook-github-bot deleted the gh/dreiss/28/head branch June 12, 2022 14:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add errors if we try to use PyTorch with Python 2#35611

Add errors if we try to use PyTorch with Python 2#35611
dreiss wants to merge 5 commits intogh/dreiss/28/basefrom
gh/dreiss/28/head

dreiss commented Mar 28, 2020 •

edited

Loading

Uh oh!

dr-ci Bot commented Mar 28, 2020 •

edited

Loading

Uh oh!

dreiss commented Mar 31, 2020

Uh oh!

ezyang commented Mar 31, 2020

Uh oh!

dreiss commented Mar 31, 2020

Uh oh!

github-actions Bot commented Apr 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

dreiss commented Mar 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci Bot commented Mar 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 Build failures summary and remediations

🕵️ 5 new failures recognized by patterns

pytorch_linux_xenial_py3_6_gcc5_4_test (1/5)

pytorch_macos_10_13_py3_test (2/5)

pytorch_windows_vs2019_py36_cuda10.1_test2 (3/5)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (4/5)

pytorch_linux_xenial_py3_clang5_asan_test (5/5)

Extra GitHub checks: 1 failed

Uh oh!

dreiss commented Mar 31, 2020

Uh oh!

ezyang commented Mar 31, 2020

Uh oh!

dreiss commented Mar 31, 2020

Uh oh!

github-actions Bot commented Apr 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dreiss commented Mar 28, 2020 •

edited

Loading

dr-ci Bot commented Mar 28, 2020 •

edited

Loading