Skip to content

properly update _flat_weights in RNN modules#32939

Closed
ngimel wants to merge 2 commits intopytorch:masterfrom
ngimel:fix_lstm_flat_weight
Closed

properly update _flat_weights in RNN modules#32939
ngimel wants to merge 2 commits intopytorch:masterfrom
ngimel:fix_lstm_flat_weight

Conversation

@ngimel
Copy link
Collaborator

@ngimel ngimel commented Feb 3, 2020

Should fix #32346 hopefully. Now when _flat_weights list is updated, None elements are appended to it if some weights are missing, subsequent setattr calls for the missing weights should repair _flat_weights and make it suitable to use in the backend.

@ngimel ngimel requested a review from mruberry February 3, 2020 20:48
@ngimel ngimel requested a review from apaszke as a code owner February 3, 2020 20:48
# does not throw an error
m.cuda()
# recompute the weight and make sure that module can be used
setattr(m, "weight_hh_l0", weight_orig.cuda())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.weight_hh_l0 = weight_orig.cuda()

Copy link
Collaborator

@mruberry mruberry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks better than before!

@kostmo
Copy link
Member

kostmo commented Feb 3, 2020

💊 CircleCI build failures summary and remediations

As of commit 9a18f36:

None of the build failures appear to be your fault.

  • 1/1 recognized as flaky ❄️
    • Re-run these jobs?

Detailed failure analysis

One may explore the probable reasons each build failed interactively on the Dr. CI website.

❄️ 1 failure recognized as flaky

The following build failures have been detected as flaky and may not be your fault:

See CircleCI build caffe2_onnx_py3_6_clang7_ubuntu16_04_test (1/1)

Step: "Test" (full log | pattern match details) ❄️

Feb 03 22:38:00 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_slice_index_exceeds_dim Fatal Python error: Segmentation fault
Feb 03 22:37:59 test/onnx/test_utility_funs.py::TestUtilityFuns_opset11::test_error_on_data_parallel PASSED [ 99%] 
Feb 03 22:37:59 test/onnx/test_utility_funs.py::TestUtilityFuns_opset11::test_is_in_onnx_export PASSED [ 99%] 
Feb 03 22:37:59 test/onnx/test_utility_funs.py::TestUtilityFuns_opset11::test_strip_doc_string PASSED [ 99%] 
Feb 03 22:37:59 test/onnx/test_utility_funs.py::TestUtilityFuns_opset11::test_validate_dynamic_axes_invalid_input_output_name PASSED [ 99%] 
Feb 03 22:37:59 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_concat PASSED [ 99%] 
Feb 03 22:38:00 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_div PASSED [ 99%] 
Feb 03 22:38:00 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_lstm PASSED [ 99%] 
Feb 03 22:38:00 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_mul PASSED [ 99%] 
Feb 03 22:38:00 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_reshape PASSED [ 99%] 
Feb 03 22:38:00 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_slice PASSED [ 99%] 
Feb 03 22:38:00 test/onnx/test_utility_funs.py::TestUtilityFuns::test_constant_fold_slice_index_exceeds_dim Fatal Python error: Segmentation fault 
Feb 03 22:38:00  
Feb 03 22:38:00 Current thread 0x00007fba1402c700 (most recent call first): 
Feb 03 22:38:00   File "/usr/local/lib/python3.6/dist-packages/_pytest/_io/saferepr.py", line 43 in repr_instance 
Feb 03 22:38:00   File "/usr/lib/python3.6/reprlib.py", line 65 in repr1 
Feb 03 22:38:00   File "/usr/lib/python3.6/reprlib.py", line 55 in repr 
Feb 03 22:38:00   File "/usr/local/lib/python3.6/dist-packages/_pytest/_io/saferepr.py", line 36 in repr 
Feb 03 22:38:00   File "/usr/local/lib/python3.6/dist-packages/_pytest/_io/saferepr.py", line 67 in saferepr 
Feb 03 22:38:00   File "/usr/local/lib/python3.6/dist-packages/_pytest/_code/code.py", line 655 in repr_args 
Feb 03 22:38:00   File "/usr/local/lib/python3.6/dist-packages/_pytest/_code/code.py", line 736 in repr_traceback_entry 
Feb 03 22:38:00   File "/usr/local/lib/python3.6/dist-packages/_pytest/_code/code.py", line 777 in repr_traceback 

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 4 times.

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@ngimel merged this pull request in df71b3e.

ngimel pushed a commit to ngimel/pytorch that referenced this pull request Feb 4, 2020
Summary:
Should fix pytorch#32346 hopefully. Now when _flat_weights list is updated, `None` elements are appended to it if some weights are missing, subsequent `setattr` calls for the missing weights should repair _flat_weights and make it suitable to use in the backend.
Pull Request resolved: pytorch#32939

Differential Revision: D19710990

Pulled By: ngimel

fbshipit-source-id: c978c7519464e94beeffa9bc33b9172854a2f298
facebook-github-bot pushed a commit that referenced this pull request Feb 5, 2020
Summary:
Resubmitting #32939
Should fix #32346 hopefully. Now when _flat_weights list is updated, None elements are appended to it if some weights are missing, subsequent setattr calls for the missing weights should repair _flat_weights and make it suitable to use in the backend.
Pull Request resolved: #32989

Differential Revision: D19731952

Pulled By: ngimel

fbshipit-source-id: 2118a19840491e7ab0fef15185fad982f42795a6
BowenBao pushed a commit to BowenBao/pytorch that referenced this pull request Feb 12, 2020
Summary:
Should fix pytorch#32346 hopefully. Now when _flat_weights list is updated, `None` elements are appended to it if some weights are missing, subsequent `setattr` calls for the missing weights should repair _flat_weights and make it suitable to use in the backend.
Pull Request resolved: pytorch#32939

Differential Revision: D19710990

Pulled By: ngimel

fbshipit-source-id: c978c7519464e94beeffa9bc33b9172854a2f298
BowenBao pushed a commit to BowenBao/pytorch that referenced this pull request Feb 12, 2020
Summary:
Resubmitting pytorch#32939
Should fix pytorch#32346 hopefully. Now when _flat_weights list is updated, None elements are appended to it if some weights are missing, subsequent setattr calls for the missing weights should repair _flat_weights and make it suitable to use in the backend.
Pull Request resolved: pytorch#32989

Differential Revision: D19731952

Pulled By: ngimel

fbshipit-source-id: 2118a19840491e7ab0fef15185fad982f42795a6
ttumiel pushed a commit to ttumiel/pytorch that referenced this pull request Mar 4, 2020
Summary:
Should fix pytorch#32346 hopefully. Now when _flat_weights list is updated, `None` elements are appended to it if some weights are missing, subsequent `setattr` calls for the missing weights should repair _flat_weights and make it suitable to use in the backend.
Pull Request resolved: pytorch#32939

Differential Revision: D19710990

Pulled By: ngimel

fbshipit-source-id: c978c7519464e94beeffa9bc33b9172854a2f298
ttumiel pushed a commit to ttumiel/pytorch that referenced this pull request Mar 4, 2020
Summary:
Resubmitting pytorch#32939
Should fix pytorch#32346 hopefully. Now when _flat_weights list is updated, None elements are appended to it if some weights are missing, subsequent setattr calls for the missing weights should repair _flat_weights and make it suitable to use in the backend.
Pull Request resolved: pytorch#32989

Differential Revision: D19731952

Pulled By: ngimel

fbshipit-source-id: 2118a19840491e7ab0fef15185fad982f42795a6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pytorch 1.4.0 weight drop - 'LSTM' object has no attribute 'weight_hh_l0'

4 participants