Temporal up-sampling of LIDAR measurements based on a mono camera
Most of the 3D LIDAR sensors used in autonomous driving have significantly lower frame rates than modern cameras equipped to the same vehicle. This paper proposes a solution to virtually increase the frame rate of the LIDARs utilizing a mono camera, making possible the monitoring of dynamic objects with fast movement in the environment. First, dynamic object candidates are detected and tracked in the camera frames. Next, LIDAR points corresponding to these objects are identified. Then, virtual camera poses can be calculated by back projecting these points to the camera and tracking them. Finally, from the virtual camera poses, the object movement (transformation matrix transforming the object between frames) can be calculated (knowing the real camera poses) to the time moment, which does not have a corresponding LIDAR measurement. Static objects (rigid with the scene) can also be transformed to this time movement if the real camera poses are known. The proposed method has been tested in the Argoverse dataset, and it has outperformed earlier methods with a similar purpose.