Reduce sample inputs for prototype transform kernels by pmeier · Pull Request #6714 · pytorch/vision

pmeier · 2022-10-06T12:49:02Z

Addresses 3. from #6622.

This PR achieves a significant reduction of the number of sample inputs. Only for the bounding box kernels, the number goes up slightly. After manual inspection, it became clear that our previous sample inputs were not sufficient.

kernel	# sample inputs `main`	# sample inputs PR
`pad_image_tensor`	2448	108
`resize_image_tensor`	960	73
`pad_mask`	864	12
`center_crop_image_tensor`	864	48
`resize_mask`	720	12
`center_crop_mask`	648	12
`affine_image_tensor`	624	180
`ten_crop_image_tensor`	480	60
`rotate_image_tensor`	432	84
`crop_image_tensor`	384	30
`affine_mask`	288	12
`convert_color_space_image_tensor`	288	168
`crop_mask`	288	12
`gaussian_blur_image_tensor`	288	36
`affine_bounding_box`	252	264
`five_crop_image_tensor`	240	30
`pad_bounding_box`	108	252
`rotate_mask`	72	12
`resize_bounding_box`	60	180

In total we cut the number of tests (and approximately the runtime) roughly to a third of what is was before. Plus, we even increase the coverage ever so slightly:

main:

---------- coverage: platform linux, python 3.7.12-final-0 -----------
Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
torchvision/prototype/transforms/functional/__init__.py               8      0   100%
torchvision/prototype/transforms/functional/_augment.py              18      1    94%
torchvision/prototype/transforms/functional/_color.py                91      0   100%
torchvision/prototype/transforms/functional/_deprecated.py           29     17    41%
torchvision/prototype/transforms/functional/_geometry.py            539     32    94%
torchvision/prototype/transforms/functional/_meta.py                137     28    80%
torchvision/prototype/transforms/functional/_misc.py                 48      7    85%
torchvision/prototype/transforms/functional/_type_conversion.py      31      7    77%
-------------------------------------------------------------------------------------
TOTAL                                                               901     92    90%

===== 66202 passed, 87703 skipped, 1792 xfailed, 1 warning in 173.32s (0:02:53) =====

PR:

---------- coverage: platform linux, python 3.7.12-final-0 -----------
Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
torchvision/prototype/transforms/functional/__init__.py               8      0   100%
torchvision/prototype/transforms/functional/_augment.py              18      1    94%
torchvision/prototype/transforms/functional/_color.py                91      0   100%
torchvision/prototype/transforms/functional/_deprecated.py           29     17    41%
torchvision/prototype/transforms/functional/_geometry.py            539     31    94%
torchvision/prototype/transforms/functional/_meta.py                137     25    82%
torchvision/prototype/transforms/functional/_misc.py                 48      7    85%
torchvision/prototype/transforms/functional/_type_conversion.py      31      7    77%
-------------------------------------------------------------------------------------
TOTAL                                                               901     88    90%

====== 21785 passed, 21954 skipped, 544 xfailed, 1 warning in 60.59s (0:01:00) ======

pmeier · 2022-10-06T12:50:00Z

test/prototype_transforms_dispatcher_infos.py



-def xfail_python_scalar_arg_jit(name, *, reason=None):
+def xfail_jit_python_scalar_arg(name, *, reason=None):


Just a rename that moves the jit term to the front of the name to make it clear this is only a JIT issue.

pmeier · 2022-10-06T12:51:39Z

test/prototype_transforms_dispatcher_infos.py

+            # TODO: check if this is a regression since it seems that should be supported if `int` is ok
+            xfail_jit_list_of_ints("fill"),


We have a few of these. I'm not sure if this is a regression of #6636. Will check and send a follow-up PR since this is one is only test changes.

pmeier · 2022-10-06T12:53:05Z

torchvision/transforms/functional_tensor.py

    kernel = _get_gaussian_kernel2d(kernel_size, sigma, dtype=dtype, device=img.device)
    kernel = kernel.expand(img.shape[-3], 1, kernel.shape[0], kernel.shape[1])

-    img, need_cast, need_squeeze, out_dtype = _cast_squeeze_in(


Just a formatting change like #5880 that I saw while going through the kernels.

vfdev-5 · 2022-10-06T13:11:51Z

test/prototype_transforms_dispatcher_infos.py

                and args_kwargs.kwargs.get("padding_mode", "constant") == "constant",
-            )
+            ),
+            xfail_jit_python_scalar_arg("padding"),


I wonder why all this xfail appeared in this PR and not before ?

Because we didn't test scalar padding before

vision/test/prototype_transforms_kernel_infos.py

Line 826 in e3941af

padding=[[1], [1, 1], [1, 1, 2, 2]],

Thus, while reducing the number of sample inputs now, the tests are actually more comprehensive.

vfdev-5

Thanks @pmeier !

Summary: * pad_image_tensor * pad_mask and pad_bounding_box * resize_{image_tensor, mask, bounding_box} * center_crop_{image_tensor, mask} * {five, ten}_crop_image_tensor * crop_{image_tensor, mask} * convert_color_space_image_tensor * affine_{image_tensor, mask, bounding_box} * rotate_{image_tensor, mask} * gaussian_blur_image_tensor * cleanup Reviewed By: NicolasHug Differential Revision: D40427478 fbshipit-source-id: 4954886d7d0b8212ea92847b07092a8dfad809cb

pmeier added 11 commits October 6, 2022 10:24

pad_image_tensor

1b9c76d

pad_mask and pad_bounding_box

a43100b

resize_{image_tensor, mask, bounding_box}

2b36b59

center_crop_{image_tensor, mask}

96587b2

{five, ten}_crop_image_tensor

ed2f037

crop_{image_tensor, mask}

ffed381

convert_color_space_image_tensor

7e7bc09

affine_{image_tensor, mask, bounding_box}

a405772

rotate_{image_tensor, mask}

136fcbc

gaussian_blur_image_tensor

885ed0f

cleanup

4b4a26d

pmeier added module: transforms module: tests prototype labels Oct 6, 2022

pmeier requested a review from vfdev-5 October 6, 2022 12:49

facebook-github-bot added the cla signed label Oct 6, 2022

Merge branch 'main' into reduce-sample-inputs

eb95f17

pmeier commented Oct 6, 2022

View reviewed changes

vfdev-5 reviewed Oct 6, 2022

View reviewed changes

vfdev-5 approved these changes Oct 6, 2022

View reviewed changes

pmeier merged commit 026991b into pytorch:main Oct 6, 2022

pmeier deleted the reduce-sample-inputs branch October 6, 2022 13:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce sample inputs for prototype transform kernels#6714

Reduce sample inputs for prototype transform kernels#6714
pmeier merged 12 commits intopytorch:mainfrom
pmeier:reduce-sample-inputs

pmeier commented Oct 6, 2022

Uh oh!

pmeier Oct 6, 2022

Uh oh!

pmeier Oct 6, 2022

Uh oh!

pmeier Oct 6, 2022

Uh oh!

vfdev-5 Oct 6, 2022

Uh oh!

pmeier Oct 6, 2022

Uh oh!

vfdev-5 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants



		def xfail_python_scalar_arg_jit(name, *, reason=None):
		def xfail_jit_python_scalar_arg(name, *, reason=None):

		# TODO: check if this is a regression since it seems that should be supported if `int` is ok
		xfail_jit_list_of_ints("fill"),

Conversation

pmeier commented Oct 6, 2022

Uh oh!

pmeier Oct 6, 2022

Choose a reason for hiding this comment

Uh oh!

pmeier Oct 6, 2022

Choose a reason for hiding this comment

Uh oh!

pmeier Oct 6, 2022

Choose a reason for hiding this comment

Uh oh!

vfdev-5 Oct 6, 2022

Choose a reason for hiding this comment

Uh oh!

pmeier Oct 6, 2022

Choose a reason for hiding this comment

Uh oh!

vfdev-5 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants