fix unflatten_dense_tensor when there is empty tensor inside#50321
fix unflatten_dense_tensor when there is empty tensor inside#50321zhaojuanmao wants to merge 1 commit intopytorch:masterfrom
Conversation
💊 CI failures summary and remediationsAs of commit 98573e6 (more details on the Dr. CI page):
1 failure not recognized by patterns:
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions to the (internal) Dr. CI Users group. |
|
This pull request was exported from Phabricator. Differential Revision: D25859503 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D25859503 |
eb0f8e1 to
db797f8
Compare
|
This pull request was exported from Phabricator. Differential Revision: D25859503 |
db797f8 to
c2ee44d
Compare
|
Thanks for fixing this! Test plan LG, code lg as well but I'd hope to wait from someone who is more familiar with the code to accept. |
|
@mrshenli @pritamdamania87 would you please help reviewing this diff? thanks! |
There was a problem hiding this comment.
nit: We don't need flat_tensor_view here right? We can just use flat.options()? Then we only need to create flat_tensor_view in the else block below?
There was a problem hiding this comment.
Should we assert here that each one's data_ptr is 0 instead of asserting both data_ptrs are equal? This assertion seems to suggest both should share the same storage (which they shouldn't). They're equal just from the fact that they are empty for now.
|
This pull request was exported from Phabricator. Differential Revision: D25859503 |
c2ee44d to
20a3a20
Compare
…#50321) Summary: Pull Request resolved: pytorch#50321 Quantization team reported that when there are two empty tensors are replicated among ranks, the two empty tensors start to share storage after resizing. The root cause is unflatten_dense_tensor unflattened the empty tensor as view of flat tensor and thus share storage with other tensors. This PR is trying to avoid unflatten the empty tensor as view of flat tensor so that empty tensor will not share storage with other tensors. Test Plan: unit test Reviewed By: pritamdamania87 Differential Revision: D25859503 fbshipit-source-id: 4cb9756911a3bcaf46daf66b5a66b08fe9f1bda5
20a3a20 to
98573e6
Compare
|
This pull request was exported from Phabricator. Differential Revision: D25859503 |
|
This pull request has been merged in c9cae14. |
…#50321) Summary: Pull Request resolved: pytorch#50321 Quantization team reported that when there are two empty tensors are replicated among ranks, the two empty tensors start to share storage after resizing. The root cause is unflatten_dense_tensor unflattened the empty tensor as view of flat tensor and thus share storage with other tensors. This PR is trying to avoid unflatten the empty tensor as view of flat tensor so that empty tensor will not share storage with other tensors. Test Plan: unit test Reviewed By: pritamdamania87 Differential Revision: D25859503 fbshipit-source-id: 5b760b31af6ed2b66bb22954cba8d1514f389cca
Summary:
Quantization team reported that when there are two empty tensors are replicated among ranks, the two empty tensors start to share storage after resizing.
The root cause is unflatten_dense_tensor unflattened the empty tensor as view of flat tensor and thus share storage with other tensors.
This PR is trying to avoid unflatten the empty tensor as view of flat tensor so that empty tensor will not share storage with other tensors.
Test Plan: unit test
Differential Revision: D25859503