Share Email Print

Journal of Electronic Imaging

Video analysis using spatiotemporal descriptor and kernel extreme learning machine for lip reading
Author(s): Longbin Lu; Xinman Zhang; Xuebin Xu; Dongpeng Shang
Format Member Price Non-Member Price
PDF $20.00 $25.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Lip-reading techniques have shown bright prospects for speech recognition under noisy environments and for hearing-impaired listeners. We aim to solve two important issues regarding lip reading: (1) how to extract discriminative lip motion features and (2) how to establish a classifier that can provide promising recognition accuracy for lip reading. For the first issue, a projection local spatiotemporal descriptor, which considers the lip appearance and motion information at the same time, is utilized to provide an efficient representation of a video sequence. For the second issue, a kernel extreme learning machine (KELM) based on the single-hidden-layer feedforward neural network is presented to distinguish all kinds of utterances. In general, this method has fast learning speed and great robustness to nonlinear data. Furthermore, quantum-behaved particle swarm optimization with binary encoding is introduced to select the appropriate feature subset and parameters for KELM training. Experiments conducted on the AVLetters and OuluVS databases show that the proposed lip-reading method achieves a superior recognition accuracy compared with two previous methods.

Paper Details

Date Published: 6 October 2015
PDF: 11 pages
J. Electron. Imag. 24(5) 053023 doi: 10.1117/1.JEI.24.5.053023
Published in: Journal of Electronic Imaging Volume 24, Issue 5
Show Author Affiliations
Longbin Lu, Xi'an Jiaotong Univ. (China)
Xinman Zhang, Xi'an Jiaotong Univ. (China)
Xuebin Xu, Guangdong Xi'an Jiaotong Univ. Academy (China)
Dongpeng Shang, Xi'an Jiaotong Univ. (China)

© SPIE. Terms of Use
Back to Top