In our prototype transforms tests, we have the ability to set closeness_kwargs

that will be passed to assert_close in addition to the values to be compared:

However, there is currently no option to specify the closeness_kwargs more specifically. Meaning, they will be applied to all tests regardless of the dtype and device. Especially for the dtype, this is not sufficient: images in torch.uint8 might be allowed an absolute tolerance of 1 since that is a minor difference. However, applying the same tolerance to an torch.float32 image would mean that any difference in the valid value range of [0, 1] would be fine and thus rendering the test almost useless. Thus, we use the stricter torch.float32 tolerance here, but that leads to flaky tests #6741.

We already have the TestMark functionality in, that allows us to apply pytest.mark.* to specific tests based on the inputs. We should extend this to also specify the tolerances or even the whole closeness_kwargs.

cc @vfdev-5 @datumbox @bjuncek