Shuda Li, Ankar Handa, Yang Zhang and Andrew Calway
The auto-exposure (AE) function on vision sensors is often a curse for vision algorithms - changing the appearance of scene content according to the current pose of the sensor and local lighting conditions, making it difficult to build reliable and invariant scene models. In this work we show how it is possible to exploit AE in order to build high dynamic range (HDR) scene reconstructions whilst at the same time maintiining reilable 3-D tracking in the face of changing appearance caused by the AE.
The key contribution is a matching function - a normalised form of correlation in exposure space derived via the inverse camera response function (CRF) - which under reasonable assumptions is independent of changes in exposure time and hence gives reliable model to frame matching. Appearance frames captured at different exposures are then fused to build an HDR reconstruction of the scene. A schematic of the complete alorithm is shown in the figure below. Results on synthetic and real data demonstrate that the method provides both improved tracking and maps with far greater dynamic range of luminosity.
Video summary of the results obtained in the experiments described in the above paper
Top three images show the textured scene reconstruction obtained by fusing RGB images using our implementation of the method described in [1]. Note the highlighted artifacts. The bottom images show the textured reconstruction obtained using our method which is free of the artifacts. The HDR images were produced using Mantiuk tone mapping [2].
[1] T. Whelan, M. Kaess, H. Johannsson, M. Fallon, J. J. 947 Leonard, and J. McDonald. Real-time large-scale dense
RGB-D SLAM with volumetric fusion. Intl. Journal on Robotics Research (IJRR), 34(4-5):598-626, 2015.
[2] R. Mantiuk, S. Daly, and L. Kerofsky. Display adaptive tone mapping. ACM Trans. on Graphics (ToG), 27(3):1, 2008.
The images on the left are HDR images obtained using a high-end SLR camera. The images on the right were generated using HDRFusion and rendered using Mantiuk tone mapping [2] with the colour saturation set to 1. Note that the estimated HDR texture closely matches that obtained by the high end camera.
The auto-exposure (AE) function on vision sensors is often a curse for vision algorithms - changing the appearance of scene content according to the current pose of the sensor and local lighting conditions, making it difficult to build reliable and invariant scene models. In this work we show how it is possible to exploit AE in order to build high dynamic range (HDR) scene reconstructions whilst at the same time maintiining reilable 3-D tracking in the face of changing appearance caused by the AE.
