HDRFusion: RGB-D SLAM with Auto Exposure

Shuda Li, Ankar Handa, Yang Zhang and Andrew Calway

The auto-exposure (AE) function on vision sensors is often a curse for vision algorithms - changing the appearance of scene content according to the current pose of the sensor and local lighting conditions, making it difficult to build reliable and invariant scene models. In this work we show how it is possible to exploit AE in order to build high dynamic range (HDR) scene reconstructions whilst at the same time maintiining reilable 3-D tracking in the face of changing appearance caused by the AE.

The key contribution is a matching function - a normalised form of correlation in exposure space derived via the inverse camera response function (CRF) - which under reasonable assumptions is independent of changes in exposure time and hence gives reliable model to frame matching. Appearance frames captured at different exposures are then fused to build an HDR reconstruction of the scene. A schematic of the complete alorithm is shown in the figure below. Results on synthetic and real data demonstrate that the method provides both improved tracking and maps with far greater dynamic range of luminosity.


HDRFusion: HDR SLAM using a low-cost auto-exposure RGB-D sensor, Shuda Li, Ankur Handa, Yang Zhang and Andrew Calway, arXiv:1604.00895.

[Source code][Data]


Video summary of the results obtained in the experiments described in the above paper

Top three images show the textured scene reconstruction obtained by fusing RGB images using our implementation of the method described in [1]. Note the highlighted artifacts. The bottom images show the textured reconstruction obtained using our method which is free of the artifacts. The HDR images were produced using Mantiuk tone mapping [2].

[1] T. Whelan, M. Kaess, H. Johannsson, M. Fallon, J. J. 947 Leonard, and J. McDonald. Real-time large-scale dense RGB-D SLAM with volumetric fusion. Intl. Journal on Robotics Research (IJRR), 34(4-5):598-626, 2015.

[2] R. Mantiuk, S. Daly, and L. Kerofsky. Display adaptive tone mapping. ACM Trans. on Graphics (ToG), 27(3):1, 2008.

The images on the left are HDR images obtained using a high-end SLR camera. The images on the right were generated using HDRFusion and rendered using Mantiuk tone mapping [2] with the colour saturation set to 1. Note that the estimated HDR texture closely matches that obtained by the high end camera.