Share Email Print

Proceedings Paper

Joining geometric and RGB features for RGB-D semantic segmentation
Author(s): Shaopeng Zhang; Ming Zhong; Gang Zeng; Rui Gan
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Depth map is a regular format of geometric data structure. Some approaches attempted to harness point cloud from depth channel to extract 3D features and demonstrated the superiority over traditional 2.5D representation approaches. However, how to add 3D features to the pixel on RGB frames in order to incorporate geometric information is a challenging task. In this paper, we propose a simple and general framework combining geometric information of depth maps and RGB information of color maps for semantic segmentation task. Specifically, we first extract geometric feature from an associated point cloud which is harnessed from depth map, and then the RGB feature from color map. Due to the regular format of depth map, the geometric feature can be easily mapped to the corresponding pixel on RGB feature. After that, we get a combination of RGB and geometric features with our Element-Max-Min-Fuse function. Additionally, an efficient decoder module is designed to refine the segmentation results. We demonstrate the effectiveness of the proposed model on S3DIS dataset, the experimental results show that our method enhances the result of using RGB image or point cloud alone.

Paper Details

Date Published: 27 November 2019
PDF: 6 pages
Proc. SPIE 11321, 2019 International Conference on Image and Video Processing, and Artificial Intelligence, 113211F (27 November 2019); doi: 10.1117/12.2541645
Show Author Affiliations
Shaopeng Zhang, Peking Univ. (China)
Ming Zhong, Peking Univ. (China)
Gang Zeng, Peking Univ. (China)
Rui Gan, Peking Univ. (China)

Published in SPIE Proceedings Vol. 11321:
2019 International Conference on Image and Video Processing, and Artificial Intelligence
Ruidan Su, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?