Skip to content

Aruco marker incorrect detection for some markers of perspective transformed GridBoard near image edge #26922

@nichgaun

Description

@nichgaun

System Information

OpenCV python version: 4.11.0.86
Operating System / Platform: Ubuntu 22.04 in WSL2 on Windows 10
Python version: 3.10.12

Detailed description

When performing Aruco marker detection on an OpenCV-generated image of a board that has undergone a specific "warpPerspective" transform (as shown by the transform matrix in the code below), the resulting detections show strange, incorrect results for some markers. In the example below, the markers with id=172 and id=76 (from predefined dictionary 4x4_1000) exhibit the issue.

It's clear to me that detection could be incorrect because the lower left corners of the markers in question are right near the edge of the image. However, I would expect either detection to fail or the marker be rejected, rather than this odd incorrect detection where the corners all appear to be "inset" from the actual corners of the marker in the image.

Image
Image

Steps to reproduce

import numpy as np
import cv2

width = 19
height = 10
mrkr_size = 300
separation_ratio = 0.25
brdr_size = int(mrkr_size * separation_ratio)

# construct aruco grid board image
aruco_dict = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_4X4_1000)
gridboard = cv2.aruco.GridBoard((width, height), 1, separation_ratio, aruco_dict)

image_size = (
    int(((width)*mrkr_size)+((width+1)*brdr_size)),
    int(((height)*mrkr_size)+((height+1)*brdr_size))
)

board_image = gridboard.generateImage(image_size, 0, brdr_size, 1)

# perform perspective transform on board image
transform_matrix = np.float32([[1, -0.2, 300], [0.4, 1, -1000], [0, 0, 1]])
warped_image = cv2.warpPerspective(board_image, transform_matrix, image_size)

# detect markers
detector = cv2.aruco.ArucoDetector(aruco_dict)
corners, ids, rejected = detector.detectMarkers(warped_image)

# annotate image
color_image = cv2.cvtColor(warped_image, cv2.COLOR_GRAY2BGR)
cv2.aruco.drawDetectedMarkers(color_image, corners, ids)
cv2.imwrite("minimal.png", color_image, [cv2.IMWRITE_PNG_COMPRESSION, 9])

Issue submission checklist

  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
  • I updated to the latest OpenCV version and the issue is still there
  • There is reproducer code and related data files (videos, images, onnx, etc)

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions