Keypoints with no valid 3D points are mostly because they are too far away to be measured. filterKeypointsByDepth() will remove them. But this doesn't seem to be what we want. Should we consider retaining this part?

The features should be already filtered by depth there if _depthAsMask is true and if RGB-D data is provided. For stereo, that could be indeed added. Suggestion:

if(depthMask.empty() && (_feature2D->getMinDepth() > 0.0f || _feature2D->getMaxDepth() > 0.0f))

if(depthMask.empty() && (_feature2D->getMinDepth() > 0.0f || _feature2D->getMaxDepth() > 0.0f))

I noticed this doesn't work with ORBOctree.

Do other types of features have similar problems?

if(depthMask.empty() && (_feature2D->getMinDepth() > 0.0f || _feature2D->getMaxDepth() > 0.0f))

I noticed this doesn't work with ORBOctree.

rtabmap/corelib/src/opencv/ORBextractor.h

Line 66 in 71bb0cf

// Mask is ignored in the current implementation.

Do other types of features have similar problems?

I added a new commit to fix OrbOctree ignoring the mask. I think it was the only one ignoring it. I also added my suggestion. It seems to work as expected now.