Move std and var tests to OpInfos by peterbell10 · Pull Request #50901 · pytorch/pytorch

peterbell10 · 2021-01-21T20:34:13Z

Stack from ghstack:

Add correction parameter to std/var #50903 Add correction parameter to std/var
Make at::numeric_limits non CUDA-specific #50902 Make at::numeric_limits non CUDA-specific
Move std and var tests to OpInfos #50901 Move std and var tests to OpInfos

Differential Revision: D26083289

[ghstack-poisoned]

facebook-github-bot · 2021-01-21T20:34:25Z

💊 CI failures summary and remediations

As of commit 3f36143 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

[ghstack-poisoned]

peterbell10 · 2021-01-21T21:30:07Z

+        test_backward = (
+            (dtype.is_complex and op.test_complex_grad) or
+            (dtype.is_floating_point and (not op.skip_bfloat16_grad or dtype != torch.bfloat16)))
+        samples = op.sample_inputs(device, dtype, requires_grad=test_backward)


std and var aren't whitelisted for complex autograd, so test_complex_grad=False needs to never call backward() at all. Why wasn't this here already? Well it looks like sigmoid is the only other user of test_complex_grad and it explicitly skips these two jit tests:

pytorch/torch/testing/_internal/common_methods_invocations.py

Lines 1608 to 1612 in 7fdc6a2

# RuntimeError: sigmoid does not support automatic differentiation for outputs with complex dtype.

SkipInfo('TestCommon', 'test_variant_consistency_jit',

dtypes=[torch.complex64, torch.complex128]),

SkipInfo('TestCommon', 'test_variant_consistency_eager',

dtypes=[torch.complex64, torch.complex128]),),

These changes to the JIT testing look good to me, it was a bad oversight. Thanks for catching it

[ghstack-poisoned]

codecov · 2021-01-22T04:32:32Z

Codecov Report

Merging #50901 (ac0a3cc) into gh/peterbell10/45/base (186c3da) will increase coverage by 0.00%.
The diff coverage is 97.32%.

@@                   Coverage Diff                   @@
##           gh/peterbell10/45/base   #50901   +/-   ##
=======================================================
  Coverage                   80.91%   80.92%           
=======================================================
  Files                        1924     1926    +2     
  Lines                      210008   210019   +11     
=======================================================
+ Hits                       169923   169950   +27     
+ Misses                      40085    40069   -16

[ghstack-poisoned]

mruberry · 2021-01-25T12:41:57Z

-        samples = op.sample_inputs(device, dtype, requires_grad=True)
+        test_backward = (
+            (dtype.is_complex and op.test_complex_grad) or
+            (dtype.is_floating_point and (not op.skip_bfloat16_grad or dtype != torch.bfloat16)))


cc @Lilyjjo

mruberry · 2021-01-25T12:43:50Z

+           supports_tensor_out=False,
+           test_complex_grad=False,
+           test_inplace_grad=False,
+           skips=(SkipInfo('TestGradients', 'test_fn_gradgrad'),


Would you elaborate on these two skips?

I removed the first skip, after reviewing it I realised the tests passed as-is. (probably fixed by the test_complex_grad changes here). As for the second skip, the CPU version of std and var have only partial support for half and complex, because they sometimes call into the old TH version for all-reductions:

pytorch/aten/src/ATen/native/ReduceOps.cpp

Lines 1151 to 1156 in ac0a3cc

// NOTE: CPU performance significantly regressed when attempting to port to ATen,

// so this dispatches differently based on device type.

// See https://github.com/pytorch/pytorch/pull/43858.

if (self.device().type() == kCPU) {

return at::_var(self, unbiased);

}

So for the OpInfo: Half, ComplexFloat and ComplexDouble are declared as unsupported on cpu but not all sample inputs will raise.

That makes sense. Would you file an issue and add a comment citing it?

Opened #51127

mruberry · 2021-01-25T12:44:42Z



+def sample_inputs_std_var(op_info, device, dtype, requires_grad):
+    tensor_nd = torch.rand(S, S, S, device=device, dtype=dtype, requires_grad=requires_grad)


Why rand and not make_tensor?

Changed to make_tensor now.

mruberry

Hey @peterbell10!

The cleanup in test_ops.py looks great. I have a suggestion and a couple questions about the OpInfos, though. Should be simple to resolve.

[ghstack-poisoned]

peterbell10 · 2021-01-26T17:45:50Z

@mruberry PTAL.

mruberry

Cool!

Would you just add a comment with an issue for the skips and also remove the std and var entries here:

pytorch/test/test_torch.py

Line 6726 in 6dda036

    
           ('std', '', _small_3d, lambda t, d: [], 1e-3, 1e-5, 1e-5, _float_types, _cpu_types, False),

Then ping me and I'll merge this.

[ghstack-poisoned]

peterbell10 · 2021-01-26T19:04:54Z

@mruberry

facebook-github-bot · 2021-01-27T18:54:11Z

@mruberry merged this pull request in 9b6d463.

ghstack-source-id: 8a65fb5 Pull Request resolved: pytorch#50901

Summary: Pull Request resolved: pytorch#50901 Test Plan: Imported from OSS Reviewed By: ngimel Differential Revision: D26083289 Pulled By: mruberry fbshipit-source-id: 7e14ff37bba46dd456e0bc0aa9c4e0a632d0734c

Move std and var tests to OpInfos

16c357f

[ghstack-poisoned]

facebook-github-bot added the cla signed label Jan 21, 2021

This was referenced Jan 21, 2021

Make at::numeric_limits non CUDA-specific #50902

Closed

Add correction parameter to std/var #50903

Closed

pytorchbot added the open source label Jan 21, 2021

peterbell10 added 2 commits January 21, 2021 20:57

Update on "Move std and var tests to OpInfos"

1f4d1ed

[ghstack-poisoned]

Update on "Move std and var tests to OpInfos"

68ebadb

[ghstack-poisoned]

peterbell10 requested a review from mruberry January 21, 2021 21:03

peterbell10 commented Jan 21, 2021

View reviewed changes

Update on "Move std and var tests to OpInfos"

8f0246a

[ghstack-poisoned]

Update on "Move std and var tests to OpInfos"

a2c58f5

[ghstack-poisoned]

mruberry reviewed Jan 25, 2021

View reviewed changes

peterbell10 added 3 commits January 25, 2021 16:58

Update on "Move std and var tests to OpInfos"

66139e2

[ghstack-poisoned]

Update on "Move std and var tests to OpInfos"

98cfb33

[ghstack-poisoned]

Update on "Move std and var tests to OpInfos"

724b22a

[ghstack-poisoned]

mruberry self-requested a review January 26, 2021 18:14

mruberry approved these changes Jan 26, 2021

View reviewed changes

Update on "Move std and var tests to OpInfos"

3f36143

[ghstack-poisoned]

facebook-github-bot closed this in 9b6d463 Jan 27, 2021

facebook-github-bot added the Merged label Jan 27, 2021

peterbell10 added a commit to peterbell10/pytorch that referenced this pull request Jan 29, 2021

Move std and var tests to OpInfos

b422ae5

ghstack-source-id: 8a65fb5 Pull Request resolved: pytorch#50901

peterbell10 mentioned this pull request Jan 29, 2021

De-prioritise Dimname and DimnameList in python overload resolution #51350

Closed

facebook-github-bot deleted the gh/peterbell10/45/head branch January 31, 2021 15:18

	# RuntimeError: sigmoid does not support automatic differentiation for outputs with complex dtype.
	SkipInfo('TestCommon', 'test_variant_consistency_jit',
	dtypes=[torch.complex64, torch.complex128]),
	SkipInfo('TestCommon', 'test_variant_consistency_eager',
	dtypes=[torch.complex64, torch.complex128]),),

	// NOTE: CPU performance significantly regressed when attempting to port to ATen,
	// so this dispatches differently based on device type.
	// See https://github.com/pytorch/pytorch/pull/43858.
	if (self.device().type() == kCPU) {
	return at::_var(self, unbiased);
	}



		def sample_inputs_std_var(op_info, device, dtype, requires_grad):
		tensor_nd = torch.rand(S, S, S, device=device, dtype=dtype, requires_grad=requires_grad)

Conversation

peterbell10 commented Jan 21, 2021 • edited by mruberry Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jan 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

Uh oh!

peterbell10 Jan 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jan 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mruberry left a comment

Choose a reason for hiding this comment

Uh oh!

peterbell10 commented Jan 26, 2021

Uh oh!

mruberry left a comment

Choose a reason for hiding this comment

Uh oh!

peterbell10 commented Jan 26, 2021

Uh oh!

facebook-github-bot commented Jan 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

peterbell10 commented Jan 21, 2021 •

edited by mruberry

Loading

facebook-github-bot commented Jan 21, 2021 •

edited

Loading

peterbell10 Jan 21, 2021 •

edited

Loading

codecov Bot commented Jan 22, 2021 •

edited

Loading