Skip to content

Make interpolation output size compatible with opencv, scikit-image and scipy for floating scale factor #62396

@vfdev-5

Description

@vfdev-5

🚀 Feature

Interpolation output size is incorrectly computed with floor vs opencv/skimage/scipy are using round:

# pytorch
output_size = floor(input_size * scale)

vs

# opencv
output_size = round(input_size * scale)

Details

Currently, interpolation output size if using float scale factor and recompute_scale_factor=False is computed as

c10::SmallVector<int64_t, 3> ret;

ret.push_back(static_cast<double>(input_size[i+2]) * scale_factors.value()[i]);

if recompute_scale_factor=True:

pytorch/torch/nn/functional.py

Lines 3687 to 3695 in 9fdf7ec

if not torch.jit.is_scripting() and torch._C._get_tracing_state():
# make scale_factor a tensor in tracing so constant doesn't get baked in
output_size = [
(torch.floor((input.size(i + 2).float() * torch.tensor(scale_factors[i], dtype=torch.float32)).float()))
for i in range(dim)
]
else:
assert scale_factors is not None
output_size = [int(math.floor(float(input.size(i + 2)) * scale_factors[i])) for i in range(dim)]

In both cases, this can lead to -1 pixel output size comparing to opencv, scikit-image or scipy. For example, let's rescale an image of size (32, 32) with different scale factors: [0.1234, 0.9999, 1.8]. This gives the following output sizes per library:

- check scale_factor=0.1234
Error: different shapes: opencv (4, 4) vs sk-img (4, 4) vs scipy (4, 4) vs pth (3, 3)

- check scale_factor=0.9999
Error: different shapes: opencv (32, 32) vs sk-img (32, 32) vs scipy (32, 32) vs pth (31, 31)

- check scale_factor=1.8
Error: different shapes: opencv (58, 58) vs sk-img (58, 58) vs scipy (58, 58) vs pth (57, 57)

...

Here is a gist with details: https://gist.github.com/vfdev-5/a5bd5b1477b1c82a87a0f9e25c727664

Pitch

It would be helpful to have compatible output size using pytorch interpolate op.

Alternatives

Use output size argument directly instead of scales.

Additional context

Here is equivalent code for

cc @fmassa

Metadata

Metadata

Assignees

Labels

enhancementNot as big of a feature, but technically not a bug. Should be easy to fixmodule: interpolationtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions