🐛 Bug
The following usage of as_strided errors when lowering.
x = torch.randn(20, device=xm.xla_device())
y = x[10:]
z = y.as_strided((20,), (1,), 0)
print(z)
Traceback (most recent call last):
File "bug-as-strided.py", line 10, in <module>
print(z)
File "torch/_tensor.py", line 442, in __repr__
return torch._tensor_str._str(self, tensor_contents=tensor_contents)
File "torch/_tensor_str.py", line 664, in _str
return _str_intern(self, tensor_contents=tensor_contents)
File "torch/_tensor_str.py", line 430, in _str_intern
self = self.to("cpu")
RuntimeError: Error while lowering: [] aten::as_strided, xla_shape=f32[20]{0}, size=(20), stride=(1), storage_offset=0
Error: torch_xla/csrc/ops/as_strided.cpp:33 : Check failed: storage_offset + slice_size <= input_element_count (20 vs. 10)
*** Begin stack trace ***
tsl::CurrentStackTrace[abi:cxx11]()
torch_xla::AsStrided::Lower(torch_xla::LoweringContext*) const
torch_xla::LoweringContext::LowerNode(torch::lazy::Node const*)
torch_xla::LoweringContext::LoweringContext(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, torch::lazy::BackendDevice, c10::ArrayRef<torch::lazy::Node const*>, std::unordered_map<torch::lazy::Node const*, torch::lazy::Util::EmitStatus, std::hash<torch::lazy::Node const*>, std::equal_to<torch::lazy::Node const*>, std::allocator<std::pair<torch::lazy::Node const* const, torch::lazy::Util::EmitStatus> > >)This error shows up when trying to execute hf_Reformer from Torchbench, using openxla as backend. As far as I understand, the problem is that AOTAutograd is calling as_strided -- not entirely sure why. This problem seems to be related to the limitations of reshape functions in XLA, as suggested in #2964.
Expected behavior
I would expect it to break earlier, say, when XLANativeFunctions::as_strided is being executed, instead of when it gets to the lowering part. Or, maybe better than that, we could fallback to CPU while issueing a warning that "as_strided is creating a copy, which may not be optimal...".
Environment
PyTorch/XLA: c9a1324 (Oct 3)
🐛 Bug
The following usage of
as_stridederrors when lowering.This error shows up when trying to execute
hf_Reformerfrom Torchbench, usingopenxlaas backend. As far as I understand, the problem is that AOTAutograd is callingas_strided-- not entirely sure why. This problem seems to be related to the limitations of reshape functions in XLA, as suggested in #2964.Expected behavior
I would expect it to break earlier, say, when
XLANativeFunctions::as_stridedis being executed, instead of when it gets to the lowering part. Or, maybe better than that, we could fallback to CPU while issueing a warning that "as_stridedis creating a copy, which may not be optimal...".Environment
PyTorch/XLA: c9a1324 (Oct 3)