Amend nondeterminism notes#12217
Conversation
include indexAdd commentary as this is less well known
| include :meth:`torch.Tensor.index_add_`, :meth:`torch.Tensor.scatter_add_`, | ||
| :meth:`torch.bincount`. | ||
| A number of operations have backwards that use :attr:`atomicAdd`, in particular | ||
| :meth:`torch.nn.functional.embedding`, :meth:`torch.nn.functional.embedding_bag`, |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
Thank you, @colesbury for the correction!
|
If the other attributions seem OK, I would propose to add this. |
|
|
||
| There are some PyTorch functions that use CUDA functions that can be a source | ||
| of non-determinism. One class of such CUDA functions are atomic operations, | ||
| in particular :attr:`atomicAdd`, where the order of parallel additions to the |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
ssnl
left a comment
There was a problem hiding this comment.
LGTM assuming that the doc links work
facebook-github-bot
left a comment
There was a problem hiding this comment.
SsnL is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
These changes do not show up in the master docs, is it expected? @t-vi |
|
Insofar as the master docs are at a previous git revision and not instantly updated, yes. |
Summary: include atomicAdd commentary as this is less well known There is some discussion in pytorch#12207 Unfortunately, I cannot seem to get the ..include working in `_tensor_docs.py` and `_torch_docs.py`. I could use a hint for that. Pull Request resolved: pytorch#12217 Differential Revision: D10419739 Pulled By: SsnL fbshipit-source-id: eecd04fb7486bd9c6ee64cd34859d61a0a97ec4e
include atomicAdd commentary as this is less well known
There is some discussion in #12207
Unfortunately, I cannot seem to get the ..include working in
_tensor_docs.pyand_torch_docs.py. I could use a hint for that.