-
Notifications
You must be signed in to change notification settings - Fork 27.4k
Make interpolation output size compatible with opencv, scikit-image and scipy for floating scale factor #62396
Description
🚀 Feature
Interpolation output size is incorrectly computed with floor vs opencv/skimage/scipy are using round:
# pytorch
output_size = floor(input_size * scale)
vs
# opencv
output_size = round(input_size * scale)
Details
Currently, interpolation output size if using float scale factor and recompute_scale_factor=False is computed as
pytorch/aten/src/ATen/native/UpSample.cpp
Line 22 in c06b6e4
| c10::SmallVector<int64_t, 3> ret; |
pytorch/aten/src/ATen/native/UpSample.cpp
Line 24 in c06b6e4
| ret.push_back(static_cast<double>(input_size[i+2]) * scale_factors.value()[i]); |
if
recompute_scale_factor=True:pytorch/torch/nn/functional.py
Lines 3687 to 3695 in 9fdf7ec
| if not torch.jit.is_scripting() and torch._C._get_tracing_state(): | |
| # make scale_factor a tensor in tracing so constant doesn't get baked in | |
| output_size = [ | |
| (torch.floor((input.size(i + 2).float() * torch.tensor(scale_factors[i], dtype=torch.float32)).float())) | |
| for i in range(dim) | |
| ] | |
| else: | |
| assert scale_factors is not None | |
| output_size = [int(math.floor(float(input.size(i + 2)) * scale_factors[i])) for i in range(dim)] |
In both cases, this can lead to -1 pixel output size comparing to opencv, scikit-image or scipy. For example, let's rescale an image of size (32, 32) with different scale factors: [0.1234, 0.9999, 1.8]. This gives the following output sizes per library:
- check scale_factor=0.1234
Error: different shapes: opencv (4, 4) vs sk-img (4, 4) vs scipy (4, 4) vs pth (3, 3)
- check scale_factor=0.9999
Error: different shapes: opencv (32, 32) vs sk-img (32, 32) vs scipy (32, 32) vs pth (31, 31)
- check scale_factor=1.8
Error: different shapes: opencv (58, 58) vs sk-img (58, 58) vs scipy (58, 58) vs pth (57, 57)
...
Here is a gist with details: https://gist.github.com/vfdev-5/a5bd5b1477b1c82a87a0f9e25c727664
Pitch
It would be helpful to have compatible output size using pytorch interpolate op.
Alternatives
Use output size argument directly instead of scales.
Additional context
Here is equivalent code for
- opencv:
- scikit-image: https://github.com/scikit-image/scikit-image/blob/e128e0743bb2cded104654352a13ed5b4e53efb8/skimage/transform/_warps.py#L286
- scipy: https://github.com/scipy/scipy/blob/44e31b1eb202ef91caad059952ba268d2610a60b/scipy/ndimage/interpolation.py#L768-L769
cc @fmassa