I wasn't 100% sure on whether or not I needed to involve as_strided_backward() somewhere in the derivative formula here. I see get some interesting op info failures... but for the most part they're the same failures that as_strided already has (plus two new ones for conjugate views that I didn't try too hard to debug)

The problem is that you assume that doing "as_strided" on the grad with the parameters from the forward will do the right thing (it won't if any of these have different strides/storage_offset to begin with).
Hence the madness of as_strided_backward() :D

I think the simplest way to do this is to create a new function in functions manual that call the backward of each elementary function in order (copy_backward, as_strided_backward) as if you were doing AD on the CompositeExplicit implementation that you have. (nit: it's ok to replicate the copy_backward impl from tensor.cpp instead of making it a helper function I think).

Not sure what that means?

I have a similar comment for all of the other "slicing" ops like slice/select/diagonal.

For most operators, the "inverse" can be a view (e.g. the inverse of permute is another permute call).

functionalize() will also by default re-apply views instead of using view_copy operators, so in most of the functions in this file you'll see code like "if (re-apply views) return view() else view_copy()". But for the ops listed above we can't do that - running the "inverse" forces us to allocate a new tensor.

The problem is that you assume that doing "as_strided" on the grad with the parameters from the forward will do the right thing (it won't if any of these have different strides/storage_offset to begin with).
Hence the madness of as_strided_backward() :D

I think the simplest way to do this is to create a new function in functions manual that call the backward of each elementary function in order (copy_backward, as_strided_backward) as if you were doing AD on the CompositeExplicit implementation that you have. (nit: it's ok to replicate the copy_backward impl from tensor.cpp instead of making it a helper function I think).

There are quite a few skips here. I updated the derivative formula to handle the contiguous case properly, and locally was able to run this:

  a = torch.tensor([[3, 3], [3, 3]], requires_grad=True, dtype=torch.float32)
  b = torch.ones(2, requires_grad=True, dtype=torch.float32)

  c = torch.as_strided_scatter(a, b, (2,), (2,))
  c.sum().backward()
  print(a.grad)

as_strided_scatter doesn't work in more complicated cases, but this PR adds enough support to pass the basic LazyTensor test case, so I'd rather land it sooner to get everything working E2E, and handle the harder cases later.