@matlabbe @cdb0y511
Hi, when I further looked at the work related to SuperPoint, I noticed that it seemed that HF-Net could also be added. It provides global descriptors for coarse localization in addition to SuperPoint.
RTAB-Map reserves an interface for global descriptor, but does not implement related functions (#1105). Therefore, I am thinking about whether this part can be implemented universally and how to combine it with the existing memory management mechanism of RTAB-Map. My initial feeling is that it can help retrieval, and secondly it may be further used to improve loop closure hypotheses.
The difficulty in integrating HF-Net is that it is implemented in TensorFlow 1. I only have a few old devices with the required environment. So I also converted it to ONNX first. But I'm more curious about whether it will work on OAK. That would be really fun if it could. This model file looks a bit large, I guess it is mainly because of the fully connected layer at the end of the global head. Even if OAK can't run it, I will try to prune it into a MobileNet based SuperPoint implementation. Some people have tried to train SuperPoint based on MobileNet directly but did not achieve good results. If OAK can run it, I will not develop the inference solution on the host for the time being. But if you are interested, you can also develop it. This is an ONNX file that I converted but haven't tested yet.
https://drive.google.com/file/d/1xKJi1RfKZTLQ0JACD4H76HRpxr6kFgtn/view?usp=sharing
@matlabbe @cdb0y511
Hi, when I further looked at the work related to SuperPoint, I noticed that it seemed that HF-Net could also be added. It provides global descriptors for coarse localization in addition to SuperPoint.
RTAB-Map reserves an interface for global descriptor, but does not implement related functions (#1105). Therefore, I am thinking about whether this part can be implemented universally and how to combine it with the existing memory management mechanism of RTAB-Map. My initial feeling is that it can help retrieval, and secondly it may be further used to improve loop closure hypotheses.
The difficulty in integrating HF-Net is that it is implemented in TensorFlow 1. I only have a few old devices with the required environment. So I also converted it to ONNX first. But I'm more curious about whether it will work on OAK. That would be really fun if it could. This model file looks a bit large, I guess it is mainly because of the fully connected layer at the end of the global head. Even if OAK can't run it, I will try to prune it into a MobileNet based SuperPoint implementation. Some people have tried to train SuperPoint based on MobileNet directly but did not achieve good results. If OAK can run it, I will not develop the inference solution on the host for the time being. But if you are interested, you can also develop it. This is an ONNX file that I converted but haven't tested yet.
https://drive.google.com/file/d/1xKJi1RfKZTLQ0JACD4H76HRpxr6kFgtn/view?usp=sharing