Skip to content

Reimplement NMS using morphological operations#1192

Merged
matlabbe merged 1 commit intointrolab:masterfrom
borongyuan:oak-d-pro-w-dev
Jan 2, 2024
Merged

Reimplement NMS using morphological operations#1192
matlabbe merged 1 commit intointrolab:masterfrom
borongyuan:oak-d-pro-w-dev

Conversation

@borongyuan
Copy link
Copy Markdown
Contributor

This borrows from the PyTorch implementation in SuperGlue, but uses this corrected version. This implementation is relatively simple, and since OpenCV has various optimizations for morphological operations, it will theoretically have better performance. But I haven't done any performance comparison testing yet. Since this is performed on heatmap, the NMS in SuperPoint Torch can be changed to PyTorch implementation. This way we can remove this part from util2d and introduce ANMS later.

@matlabbe matlabbe merged commit 096592e into introlab:master Jan 2, 2024
@matlabbe
Copy link
Copy Markdown
Member

matlabbe commented Jan 2, 2024

The NMS version for SuperPoint Torch you are referring is this one?

std::vector<cv::KeyPoint> keypoints_no_nms;
for (int i = 0; i < kpts_cpu.size(0); i++) {
if(mask.empty() || mask.at<unsigned char>(kpts_cpu[i][0].item<int>(), kpts_cpu[i][1].item<int>()) != 0)
{
float response = prob_cpu[kpts_cpu[i][0]][kpts_cpu[i][1]].item<float>();
keypoints_no_nms.push_back(cv::KeyPoint(kpts_cpu[i][1].item<float>(), kpts_cpu[i][0].item<float>(), 8, -1, response));
}
}
detected_ = true;
if (nms_ && !keypoints_no_nms.empty()) {
int border = 0;
int dist_thresh = minDistance_;
int height = img.rows;
int width = img.cols;
std::vector<cv::KeyPoint> keypoints;
cv::Mat descEmpty;
util2d::NMS(keypoints_no_nms, descEmpty, keypoints, descEmpty, border, dist_thresh, width, height);
if(keypoints.size()>1)
{
return keypoints;
}
return std::vector<cv::KeyPoint>();
}

I am not using that much that code right now, but if you use it and know how to update it, you can make another PR.

@borongyuan borongyuan deleted the oak-d-pro-w-dev branch January 3, 2024 05:16
@borongyuan
Copy link
Copy Markdown
Contributor Author

I am not using that much that code right now, but if you use it and know how to update it, you can make another PR.

I don't use this part very much either. But I will test the new environment after upgrading my Jetson system to JetPack 6 in the near future and update this part.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants