Context

This PR proposes to add a clamping_mode parameter for KeyPoints. This parameter is similar to what have been added for BoundingBoxes (#9128). We are adding this parameter here to avoid the issue observed in #9223, where clamping modifies the position of KeyPoints and create a misalignment with the actual locations in the transformed image. This PR will need to be supplemented with another PR to implement a class similar to SanitizeBoundingBoxes transform to remove non valid KeyPoints, e.g. outside of the image canvas.

Implementation details

In the current proposition, we align the possible values for the clamping_mode parameters with what is already existing for BoundingBoxes, that is "soft", "hard", and None. We would like KeyPoints clamping mode to be consistent with existing data structures. We propose the following behaviors for each value:

• "soft": this is the default value if not specified otherwise. This will not apply any clamping operation;
• None: This will not apply any clamping operation;
• "hard": This will apply the current default transformation with each coordinates being bounded between 0 and canvas_size - 1.

We choose this behavior as we believe that we want the default behavior to avoid creating misalignment with the transformed image. In this case, KeyPoints are not clamped unless explicitly setting clamping_mode="hard".

Illustration of the changes

We illustrate below how those changes impact the code.

# we manually set bounding boxes and key points for the demo; in a real pipeline, these would come from the data or a CV model’s output.
orig_pts = tv_tensors.KeyPoints(
    [
        [
            [445, 700],  # nose
            [320, 660],
            [370, 660],
            [420, 660],  # left eye
            [300, 620],
            [420, 620],  # left eyebrow
            [475, 665],
            [515, 665],
            [555, 655],  # right eye
            [460, 625],
            [560, 600],  # right eyebrow
            [370, 780],
            [450, 760],
            [540, 780],
            [450, 820],  # mouth
        ],
    ],
    canvas_size=(orig_img.size[1], orig_img.size[0]),
    clamping_mode="hard"
)
cropper = v2.RandomCrop(size=(128, 128))
crops = [cropper((orig_img, orig_pts)) for _ in range(4)]
plot([(orig_img, orig_pts)] + crops)

With clamping_mode="hard"

With clamping_mode="soft" or None

Testing

Please run the following unit tests

pytest test/test_transforms_v2.py -vvv -k "keypoints"
...
718 passed, 9057 deselected in 2.91s