This contains very basic functionality of adding 'send' autograd
function to our autograd graph. The purpose of this change is to validate the
basic structure proposed here makes sense. Once this makes sense, we can build
upon this to address more complicated scenarios. At a high level we've added
the following functionality:

1) Define a very simple 'SendRpcBackwards' autograd function.
2) Attach this function to appropriate tensors when we call an RPC.
3) Store the send function in our distributed autograd context.

Differential Revision: [D16903255](https://our.internmc.facebook.com/intern/diff/D16903255/)

ghstack-source-id: 88585551
Pull Request resolved: #24876

Pull Request resolved: #24876

This contains very basic functionality of adding 'send' autograd
function to our autograd graph. The purpose of this change is to validate the
basic structure proposed here makes sense. Once this makes sense, we can build
upon this to address more complicated scenarios. At a high level we've added
the following functionality:

1) Define a very simple 'SendRpcBackwards' autograd function.
2) Attach this function to appropriate tensors when we call an RPC.
3) Store the send function in our distributed autograd context.
ghstack-source-id: 88696420

Differential Revision: [D16903255](https://our.internmc.facebook.com/intern/diff/D16903255/)

Pull Request resolved: #24876

This contains very basic functionality of adding 'send' autograd
function to our autograd graph. The purpose of this change is to validate the
basic structure proposed here makes sense. Once this makes sense, we can build
upon this to address more complicated scenarios. At a high level we've added
the following functionality:

1) Define a very simple 'SendRpcBackwards' autograd function.
2) Attach this function to appropriate tensors when we call an RPC.
3) Store the send function in our distributed autograd context.
ghstack-source-id: 88916065

Differential Revision: [D16903255](https://our.internmc.facebook.com/intern/diff/D16903255/)

…uiltin operators RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

…uiltin operators RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

…uiltin operators RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

Pull Request resolved: #25527

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
ghstack-source-id: 90403466

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

…uiltin operators RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

Pull Request resolved: #25527

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
ghstack-source-id: 90478263

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

…uiltin operators RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interfaces to ensure we have a RpcBase class which 
all RPC messages inherit from. This is useful since we create a RpcWithAutograd 
message which can wrap regular RPC messages and augment them with additional 
autograd information.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interfaces to ensure we have a RpcBase class which 
all RPC messages inherit from. This is useful since we create a RpcWithAutograd 
message which can wrap regular RPC messages and augment them with additional 
autograd information.

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

Pull Request resolved: #25527

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
ghstack-source-id: 90545822

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

…uiltin operators RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

Pull Request resolved: #25527

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
ghstack-source-id: 90623247

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

…uiltin operators RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

Pull Request resolved: #25527

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
ghstack-source-id: 90877831

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

…uiltin operators RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

Pull Request resolved: #25527

Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
ghstack-source-id: 90989582

Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

… RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

…uiltin operators RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

…uiltin operators RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]

… RPC."


Master GH issue: #23110.

This change builds upon #24876 and
provides all the autograd hooks needed for a forward pass with distributed rpc
for builtin operators. This change does not address distributed rpc for python
UDFs and that will be addressed in follow up PRs.

Summary of changes:
1. Attach send autograd functions when a request is sent from the client and
response is sent from the server.
2. Attach receive autograd functions when a request is received on the server
and a response is received on the client.
3. Generate a globally unique autograd_message_id for each send/recv autograd
function pair to uniquely identify them.
4. Major re-work of the RPC interface to support autograd seamlessly:
    * Ensure all RPC messages inherit from a base class `RpcCommandBase`. 
    * `RpcCommandBase` allows us to build a wrapper class `RpcWithAutograd` which basically holds a `RpcCommandBase` object and some additional autograd information.
    * Added a `RequestCallback` interface which we need to implement to handle RPCs. The abstract class `RequestCallback` performs operations that would be common across all implementations.
    * Moved PythonUDF handling to an appropriate class which can plug into the `RpcCommandBase` interface. This would allows us to easily attach autograd information to PythonUDF calls. 
Differential Revision: [D17148077](https://our.internmc.facebook.com/intern/diff/D17148077/)

[ghstack-poisoned]