@zdevito @ezyang @driazati I've made the improvement according to review.

Could you also include an example of how to use this?

@sidazhang Do you mean just this JIT support or the returned named tuple in general? Does test_torch.py#L6913 help?

@ezyang Maybe we need to change the docs of, for example max, from

>>> a = torch.randn(4, 4)
>>> a
tensor([[-1.2360, -0.2942, -0.1222,  0.8475],
        [ 1.1949, -1.1127, -2.2379, -0.6702],
        [ 1.5717, -0.9207,  0.1297, -1.8768],
        [-0.6172,  1.0036, -0.6060, -0.2432]])
>>> torch.max(a, 1)
torch.return_types.max(values=tensor([0.8475, 1.1949, 1.5717, 1.0036]), indices=tensor([3, 0, 0, 1]))

to something like

>>> a = torch.randn(4, 4)
>>> a
tensor([[-1.2360, -0.2942, -0.1222,  0.8475],
        [ 1.1949, -1.1127, -2.2379, -0.6702],
        [ 1.5717, -0.9207,  0.1297, -1.8768],
        [-0.6172,  1.0036, -0.6060, -0.2432]])
>>> torch.max(a, 1)
torch.return_types.max(values=tensor([0.8475, 1.1949, 1.5717, 1.0036]), indices=tensor([3, 0, 0, 1]))
>>> torch.max(a, 1).values
tensor([0.8475, 1.1949, 1.5717, 1.0036]
>>> torch.max(a, 1).indices
tensor([3, 0, 0, 1])
>>> values, indices = torch.max(a, 1)

?? What do you think? Namedtuple is a rather advanced topic of Python, and I believe that there are lots of deep learning users do not know what is a namedtuple, how to use it, and how it prints.

@sidazhang Do you mean just this JIT support or the returned named tuple in general? Does test_torch.py#L6913 help?

The test unfortunately is not super helpful in showing how I can use namedtuples in my own code. Like how to write a function that returns a namedtuple.

@sidazhang Do you mean, you want to create your own function that returns namedtuple, and use it on JIT? This is not the purpose of this PR.

The title of this PR might unfortunately not explain things well, but what is happening is, we are changing the return type of some builtin operators from tuple to namedtuple. For example, in the past, we have to write torch.max(a, dim=1)[1] but now users could also write torch.max(a, dim=1).indices, which is more readable. This PR add support of this new way of returning to JIT. But this support is currently only limited to builtin operators, not operators defined by users.

@sidazhang Do you mean, you want to create your own function that returns namedtuple, and use it on JIT? This is not the purpose of this PR.

The title of this PR might unfortunately not explain things well, but what is happening is, we are changing the return type of some builtin operators from tuple to namedtuple. For example, in the past, we have to write torch.max(a, dim=1)[1] but now users could also write torch.max(a, dim=1).indices, which is more readable. This PR add support of this new way of returning to JIT. But this support is currently only limited to builtin operators, not operators defined by users.

Got it!

namedtuple would be nice to have for user defined functions. but probably just a nice to have. Not a high priority feature for me :)

Since custom user functions are done using pybind11, I suppose you could roll it by hand. It wouldn't be as nice as having it done automatically, but it seems feasible.

@zdevito I actually did a bit more than you suggest:

• isSubtypeOf also had the same issue of casting multiple times, it's fixed now.
• the compare_names was actually an reimplementation of optional equality, so it is also removed now