-
Notifications
You must be signed in to change notification settings - Fork 28k
<< and >> operators seem silently broken for DTensor operand 1 and scalar operand 2 #156533
Copy link
Copy link
Open
Labels
bot-triagedThis is a label only to be used by the auto triage botThis is a label only to be used by the auto triage botmodule: dtensordistributed tensor tagdistributed tensor tagoncall: distributedAdd this issue/PR to distributed oncall triage queueAdd this issue/PR to distributed oncall triage queueoncall: distributed parallelismsAdd this issue/PR to the distributed parallelisms oncall triage queueAdd this issue/PR to the distributed parallelisms oncall triage queueptd-bot-triagedtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
Metadata
Metadata
Assignees
Labels
bot-triagedThis is a label only to be used by the auto triage botThis is a label only to be used by the auto triage botmodule: dtensordistributed tensor tagdistributed tensor tagoncall: distributedAdd this issue/PR to distributed oncall triage queueAdd this issue/PR to distributed oncall triage queueoncall: distributed parallelismsAdd this issue/PR to the distributed parallelisms oncall triage queueAdd this issue/PR to the distributed parallelisms oncall triage queueptd-bot-triagedtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Type
Fields
Give feedbackNo fields configured for issues without a type.
🐛 Describe the bug
In pytorch/ao#2420, I found that
<<and>>were silently wrong for DTensor operands. Specifically:user code:
if
max_abs_int32is not a DTensor, this gave correct results. Ifmax_abs_int32is a DTensor, the>>had no effect, and the local value oftmpwas equal to the local value ofmax_abs_int32. I fixed it by replacing>>withtorch.bitwise_right_shift, filing an issue if someone can make the>>operator work as expected here.In the same PR, a similar issue happened with
<<, where it had no effect for DTensor operands, but worked properly when I replaced it withtorch.bitwise_left_shift.I'm on an NVIDIA B200 machine, in the off chance that this is hardware specific.
Versions
https://www.internalfb.com/phabricator/paste/view/P1847216180
cc @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @msaroufim @dcci @aditvenk @weifengpy @tianyu-l @XilunWu @SherlockNoMad @ppwwyyxx @H-Huang