ORB-SLAM Existing Problems

(1) FAST feature points detection using the gray threshold t is fixed, so that the SLAM process due to the light intensity changes lead to changes in contrast, the number of FAST feature points are decreased, the tracking process failed, causing a disruption to the SLAM process.
(2) Pure rotation will easy to drop frames旋转时容易丢帧
(3) Need maintain a low speed during initialization, preferably in a geometrically rich area. 初始化时需要保持低速，最好几何纹理丰富地区。
(4) Invariant feature descriptors used by ORB-SLAM are also sensitive to large perspective distortion and motion blur.

Better features

So we need a feature which is better than orb to solve these problems, this feature should be illumination, rotation and scale invariant. Some hand-crafted feature-detectors are readily available with rotation or scale invariant,such as SIFT, SURF, KAZE,AKAZE. Recently many works focused on learning a detector&descriptors and already achieve better performance than tradition hand-crafted detector&descriptors.
We use HPatches Dataset to evaluate these detector&descriptors,as follow table:

from Hpatches paper

from RFnet paper

from lfnet paper

RF-Net is the stage of the art feature, It can get better illumination, rotation and scale invariant feature than ORB or SIFT and other learned detector&descriptors . It can get more robust keypoints to help SLAM tracking, mapping and reconstruction.

Worth it?

One important question is “Is it worth to spend so much performance on feature detection which is a low-level task in SLAM? ”
I search some papers and get comparison of different features time consumption as follow table:

time comsuming from lfnet

Generally speaking this framework's(ORBSLAM2) all codes are basically around how to improve the quality of feature points. The system uses the ORB features for tracking, mapping and place recognition tasks. And use a complicated extraction process to get better features. as following picture shows:

image.png

In experiment , The way to extract orb can affect accuracy greatly, we use opencv original process to compare ORBSLAM orb extract process:

image.png

load des model 花费了1.09212秒
load det model 花费了0.010026秒
handle image   花费了0.072915秒
get keypoints  花费了0.197004秒
clip patches   花费了0.055784秒
get descriptors花费了0.095122秒

RFNET can achieve 2 fps

Conclusion

Use a powerful feature is worth and can easily enhance SLAM accuracy but our main reason to use Deep feature is not to prove it is better than orb,since orbslam choose orb is not for it's accuracy but for speed. What we do here is we want to prove deep features now is enough to undertake the SLAM task,and can do better than traditional hand-crafted features.

引用：
1、 An Improved Monocular ORB-SLAM Method Xiu-zhen WU, Liu GANG, Wei-si GONG and Yan SHI Department of Control Engineering, Naval Aeronautical Engineering Institute, Yanta, China.

2、https://zhuanlan.zhihu.com/p/57235987