Move normal() to DistributionTemplates by pbelevich · Pull Request #35167 · pytorch/pytorch

pbelevich · 2020-03-21T20:29:32Z

The purpose of this PR is to move normal/normal_/normal_out to native/DistributionTemplates.h, native/cpu/DistributionTemplates.h and native/cuda/DistributionTemplates.h to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG.

Stack from ghstack:

Move normal() to DistributionTemplates #35167 Move normal() to DistributionTemplates

Differential Revision: D20588248

[ghstack-poisoned]

dr-ci · 2020-03-21T20:52:37Z

💊 CircleCI build failures summary and remediations

As of commit 74050a3 (more details on the Dr. CI page):

2/2 failures introduced in this PR

1 job timed out:

pytorch_linux_xenial_py3_clang5_asan_test

Job pytorch_xla_linux_xenial_py3_6_clang7_test is failing. Please create an issue with title prefixed by [PT_BREAK] in pytorch/xla and link to to this PR. If you have questions, please reach out to @ailzhang/@dlibenzi/@JackCaoG.

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 89 times.

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. [ghstack-poisoned]

ghstack-source-id: b030e01 Pull Request resolved: #35167

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. [ghstack-poisoned]

ghstack-source-id: 740730f Pull Request resolved: #35167

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. [ghstack-poisoned]

ghstack-source-id: 06a8c6c Pull Request resolved: #35167

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. Differential Revision: [D20588248](https://our.internmc.facebook.com/intern/diff/D20588248) [ghstack-poisoned]

ghstack-source-id: f18ae30 Pull Request resolved: #35167

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. Differential Revision: [D20588248](https://our.internmc.facebook.com/intern/diff/D20588248) [ghstack-poisoned]

ghstack-source-id: 6e3b4b8 Pull Request resolved: #35167

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. Differential Revision: [D20588248](https://our.internmc.facebook.com/intern/diff/D20588248) [ghstack-poisoned]

ghstack-source-id: 838d00b Pull Request resolved: #35167

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. Differential Revision: [D20588248](https://our.internmc.facebook.com/intern/diff/D20588248) [ghstack-poisoned]

ghstack-source-id: eede7b3 Pull Request resolved: #35167

pbelevich · 2020-03-25T01:54:06Z

@ailzhang what can be the cause of XLA failures?
test_normal_xla_float64
test_randn_xla_float32
test_randn_xla_float64

ailzhang · 2020-03-25T02:03:05Z

cc: @JackCaoG who lowered normal :D. I assume we will not match randn result given we now have XLA-specific normal?
Also we should likely fix test_normal_xla_float64.

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. Differential Revision: [D20588248](https://our.internmc.facebook.com/intern/diff/D20588248) [ghstack-poisoned]

ghstack-source-id: cd78c03 Pull Request resolved: #35167

anjali411 · 2020-03-25T17:19:00Z

aten/src/ATen/native/cuda/Copy.cu

    }
  } else {
-    AT_DISPATCH_ALL_TYPES_AND3(kHalf, kBool, kBFloat16, iter.dtype(0), "copy_", [&] {
+    AT_DISPATCH_ALL_TYPES_AND_COMPLEX_AND3(kHalf, kBool, kBFloat16, iter.dtype(0), "copy_", [&] {


this is fixed in this PR #35344

anjali411

lgtm after fixing the copy_() change :) thanks for the awesome work!

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. Differential Revision: [D20588248](https://our.internmc.facebook.com/intern/diff/D20588248) [ghstack-poisoned]

pbelevich · 2020-03-25T18:38:06Z

@ailzhang @JackCaoG should I wait for XLA change or land this PR?

ailzhang · 2020-03-25T19:04:32Z

test/test_torch.py

-        self.assertEqual(r.mean(), 2, 0.2)
-        self.assertEqual(r.std(), 3, 0.2)
+    @dtypes(torch.float, torch.double, torch.complex64, torch.complex128)
+    @dtypesIfCUDA(torch.float, torch.double, torch.complex64, torch.complex128)


Curious: these two list are identical, is this intended?

fixed, thanks!

The purpose of this PR is to move `normal`/`normal_`/`normal_out` to `native/DistributionTemplates.h`, `native/cpu/DistributionTemplates.h` and `native/cuda/DistributionTemplates.h` to make it reusable for custom RNG, see cpu_rng_test.cpp as an example of custom RNG. Differential Revision: [D20588248](https://our.internmc.facebook.com/intern/diff/D20588248) [ghstack-poisoned]

ghstack-source-id: 56d5787 Pull Request resolved: #35167

JackCaoG · 2020-03-25T23:29:11Z

Sorry, missed the email earlier. I think you can land this pr, xla will skip new tests for now. I will investigate it a bit more, most likely test issue.

ailzhang

Thanks for waiting! This PR is good to go when you're ready.
[note to self]:
It's fine to remove device dispatch for normal here since we have derivatives defined for

normal(Tensor, double)
normal(double, Tensor)
normal(Tensor, Tensor)

So that they're still overridable by XLA.
normal(double, double) is a composite op defined in TensorFactories.cpp which points to normal_ as leaf node.

facebook-github-bot · 2020-03-26T06:35:11Z

@pbelevich merged this pull request in 2dd867f.

Move normal() to DistributionTemplates

5ad2bb0

[ghstack-poisoned]

This was referenced Mar 21, 2020

Replace Generator* with Generator that holds std::shared_ptr<GeneratorImpl> #34468

Closed

pybind11 type_caster for at::Generator and custom RNG python test #34774

Closed

pbelevich mentioned this pull request Mar 21, 2020

Move normal() to DistributionTemplates #34050

Closed

pbelevich added a commit that referenced this pull request Mar 22, 2020

Move normal() to DistributionTemplates

13afad4

ghstack-source-id: b030e01 Pull Request resolved: #35167

pbelevich added a commit that referenced this pull request Mar 22, 2020

Move normal() to DistributionTemplates

5c1a191

ghstack-source-id: 740730f Pull Request resolved: #35167

pbelevich added a commit that referenced this pull request Mar 22, 2020

Move normal() to DistributionTemplates

bbad39c

ghstack-source-id: 06a8c6c Pull Request resolved: #35167

pbelevich added a commit that referenced this pull request Mar 22, 2020

Move normal() to DistributionTemplates

fe1ed12

ghstack-source-id: f18ae30 Pull Request resolved: #35167

pbelevich mentioned this pull request Mar 22, 2020

Add THP_API to THPGenerator_Wrap #35194

Closed

pbelevich added a commit that referenced this pull request Mar 23, 2020

Move normal() to DistributionTemplates

fa9aa1a

ghstack-source-id: 6e3b4b8 Pull Request resolved: #35167

pbelevich added a commit that referenced this pull request Mar 24, 2020

Move normal() to DistributionTemplates

97c347d

ghstack-source-id: 838d00b Pull Request resolved: #35167

pbelevich added a commit that referenced this pull request Mar 24, 2020

Move normal() to DistributionTemplates

cf1a251

ghstack-source-id: eede7b3 Pull Request resolved: #35167

pbelevich requested review from ailzhang and anjali411 March 25, 2020 01:59

pbelevich added a commit that referenced this pull request Mar 25, 2020

Move normal() to DistributionTemplates

695f7f9

ghstack-source-id: cd78c03 Pull Request resolved: #35167

anjali411 reviewed Mar 25, 2020

View reviewed changes

anjali411 approved these changes Mar 25, 2020

View reviewed changes

ailzhang reviewed Mar 25, 2020

View reviewed changes

pbelevich added a commit that referenced this pull request Mar 25, 2020

Move normal() to DistributionTemplates

e5a7436

ghstack-source-id: 56d5787 Pull Request resolved: #35167

ailzhang approved these changes Mar 26, 2020

View reviewed changes

facebook-github-bot closed this in 2dd867f Mar 26, 2020

facebook-github-bot added the merged label Mar 26, 2020

facebook-github-bot deleted the gh/pbelevich/90/head branch March 29, 2020 14:18

pbelevich mentioned this pull request May 21, 2020

Move bernoulli_() to DistributionTemplates #38558

Closed

mruberry added the Merged label Oct 28, 2020

nkaretnikov mentioned this pull request Dec 8, 2021

Add meta variants to float variants of normal #69632

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move normal() to DistributionTemplates#35167

Move normal() to DistributionTemplates#35167
pbelevich wants to merge 22 commits intogh/pbelevich/90/basefrom
gh/pbelevich/90/head

pbelevich commented Mar 21, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Mar 21, 2020 •

edited

Loading

Uh oh!

pbelevich commented Mar 25, 2020 •

edited

Loading

Uh oh!

ailzhang commented Mar 25, 2020

Uh oh!

anjali411 Mar 25, 2020

Uh oh!

pbelevich Mar 25, 2020

Uh oh!

anjali411 left a comment

Uh oh!

pbelevich commented Mar 25, 2020

Uh oh!

ailzhang Mar 25, 2020

Uh oh!

pbelevich Mar 25, 2020

Uh oh!

JackCaoG commented Mar 25, 2020

Uh oh!

ailzhang left a comment

Uh oh!

facebook-github-bot commented Mar 26, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

pbelevich commented Mar 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Mar 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CircleCI build failures summary and remediations

Uh oh!

pbelevich commented Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ailzhang commented Mar 25, 2020

Uh oh!

anjali411 Mar 25, 2020

Choose a reason for hiding this comment

Uh oh!

pbelevich Mar 25, 2020

Choose a reason for hiding this comment

Uh oh!

anjali411 left a comment

Choose a reason for hiding this comment

Uh oh!

pbelevich commented Mar 25, 2020

Uh oh!

ailzhang Mar 25, 2020

Choose a reason for hiding this comment

Uh oh!

pbelevich Mar 25, 2020

Choose a reason for hiding this comment

Uh oh!

JackCaoG commented Mar 25, 2020

Uh oh!

ailzhang left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Mar 26, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pbelevich commented Mar 21, 2020 •

edited

Loading

dr-ci bot commented Mar 21, 2020 •

edited

Loading

pbelevich commented Mar 25, 2020 •

edited

Loading