Benchmark¶
We run our algorithm on EuRoC dataset on Ubuntu18.04 and macOS 10.14. And make comparisons with VINS-Mono, which is one of the state-of-the-art VIO systems. We analyze the accuracy of the algorithm by comparing the root mean squared error (RMSE) of the absolute trajectory error (ATE). ATE is given by the simple difference between the estimated trajectory and ground truth after it has been aligned so that it has a minimal error, and RMSE is the standard deviation of the residuals (prediction errors). The lower the RMSE, the better a given trajectory is able to fit its ground truth. We use “evo” tool to evaluate and compare the trajectory output of odometry and SLAM algorithms, and for more information please refer to euroc evaluation. As shown in the following table, XRSLAM has higher accuracy than VINS-Mono (without loop). The average results for visual-inertial algorithms are bolded, and the following figures show the trajectory of XRSLAM on Vicon Room 1 01 sequence.
| Sequence | APE(m) | ARE(deg) | ||
|---|---|---|---|---|
| XRSLAM | VINS-Mono | XRSLAM | VINS-Mono | |
| MH_01 | 0.147 | 0.154 | 2.516 | 1.516 |
| MH_02 | 0.077 | 0.178 | 1.707 | 2.309 |
| MH_03 | 0.154 | 0.195 | 1.554 | 1.646 |
| MH_04 | 0.269 | 0.376 | 1.344 | 1.431 |
| MH_05 | 0.252 | 0.300 | 0.740 | 0.782 |
| V1_01 | 0.063 | 0.088 | 5.646 | 6.338 |
| V1_02 | 0.097 | 0.111 | 1.877 | 3.278 |
| V1_03 | 0.102 | 0.187 | 2.190 | 6.211 |
| V2_01 | 0.066 | 0.082 | 1.301 | 2.137 |
| V2_02 | 0.092 | 0.149 | 1.521 | 3.976 |
| V2_03 | 0.193 | 0.287 | 1.592 | 3.331 |
| Average | 0.137 | 0.192 | 1.998 | 2.995 |
