This paper proposes a depth up-sampling method using the confidence map for a fusion of a high resolution color sensor and low resolution time-of-flight depth sensor. The confidence map represents the accuracy of depth depending on the reflectance of a measured object and is estimated with amplitude, offset, and reconstructed error of a received signal. The proposed method suppresses the depth artifacts that are caused by difference between low and high reflective materials on an object at a distance. Although the surface of an object is located at the same distance, the reflectance of small regions within the surface depends on constituent materials. Weighted filter generated by confidence map is added to the modified noise-aware filter for depth up-sampling that is proposed by Derek et al., and is adaptively selected. The proposed method consists of followings; the normalization, the reconstruction, the confidence map estimation, and the modified noise-aware filtering. In the normalization, amplitudes and offsets of received signals are calculated and received signals are normalized by those. The phase shifts are measured between transmitted and received signals. In the reconstruction, received signals are reconstructed using only the values of phase shifts and the reconstruction errors are calculated. The confidence map is estimated with amplitudes, offsets, and reconstruction errors. The coefficients of a modified noise-aware filter are adaptively selected by referring to the confidence map. The proposed method shows the enhanced results of removing depth artifacts in the experiments.

Date Published: 27 January 2011
PDF: 6 pages
Proc. SPIE 7864, Three-Dimensional Imaging, Interaction, and Measurement, 786406 (27 January 2011); doi: 10.1117/12.872163

Published in SPIE Proceedings Vol. 7864:
Three-Dimensional Imaging, Interaction, and Measurement
J. Angelo Beraldin; Ian E. McDowall; Atilla M. Baskurt; Margaret Dolinsky; Geraldine S. Cheok; Michael B. McCarthy; Ulrich Neuschaefer-Rube, Editor(s)