Share Email Print

Journal of Electronic Imaging

3D SMoSIFT: three-dimensional sparse motion scale invariant feature transform for activity recognition from RGB-D videos
Author(s): Jun Wan; Qiuqi Ruan; Wei Li; Gaoyun An; Ruizhen Zhao
Format Member Price Non-Member Price
PDF $20.00 $25.00

Paper Abstract

Human activity recognition based on RGB-D data has received more attention in recent years. We propose a spatiotemporal feature named three-dimensional (3D) sparse motion scale-invariant feature transform (SIFT) from RGB-D data for activity recognition. First, we build pyramids as scale space for each RGB and depth frame, and then use Shi-Tomasi corner detector and sparse optical flow to quickly detect and track robust keypoints around the motion pattern in the scale space. Subsequently, local patches around keypoints, which are extracted from RGB-D data, are used to build 3D gradient and motion spaces. Then SIFT-like descriptors are calculated on both 3D spaces, respectively. The proposed feature is invariant to scale, transition, and partial occlusions. More importantly, the running time of the proposed feature is fast so that it is well-suited for real-time applications. We have evaluated the proposed feature under a bag of words model on three public RGB-D datasets: one-shot learning Chalearn Gesture Dataset, Cornell Activity Dataset-60, and MSR Daily Activity 3D dataset. Experimental results show that the proposed feature outperforms other spatiotemporal features and are comparative to other state-of-the-art approaches, even though there is only one training sample for each class.

Paper Details

Date Published: 8 April 2014
PDF: 15 pages
J. Electron. Imag. 23(2) 023017 doi: 10.1117/1.JEI.23.2.023017
Published in: Journal of Electronic Imaging Volume 23, Issue 2
Show Author Affiliations
Jun Wan, Beijing Jiaotong Univ. (China)
Qiuqi Ruan, Beijing Jiaotong Univ. (China)
Wei Li, Beijing Jiaotong Univ. (China)
Gaoyun An, Beijing Jiaotong Univ. (China)
Ruizhen Zhao, Beijing Jiaotong Univ. (China)

© SPIE. Terms of Use
Back to Top