Conversation
3ec9f92 to
4244dcb
Compare
|
How could SPMD possibly work for eager mode? |
consider eager mode as calling |
Then how sharding propogation and auto partition work? I assume they don't carry states from last graph? |
The we will compile a graph for |
|
Okay, that's fair. |
| XLATensor::ShardingSpecPtr sharding = input_tensor->sharding_spec(); | ||
| if (sharding && sharding->sharding.type() != xla::OpSharding::UNKNOWN) { | ||
| // don't propagate sharding in eager mode. | ||
| if (!XLAGraphExecutor::Get()->UseEagerMode() && sharding && |
There was a problem hiding this comment.
It complained about the output tensor already has a sharding and we can't propagate to it. This happens in the backward. I didn't spend enough time to debug it but I don't expect user to actually run eager mode with step fn(forward and backward), I only expect them to run it with some data preprocessing on device so I just quickly unblock myself.
alanwaketan
left a comment
There was a problem hiding this comment.
LGTM. Thanks, Jack!
No description provided.