Share Email Print

Proceedings Paper

Lip-reading enhancement for law enforcement
Author(s): Barry J. Theobald; Richard Harvey; Stephen J. Cox; Colin Lewis; Gari P. Owen
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Accurate lip-reading techniques would be of enormous benefit for agencies involved in counter-terrorism and other law-enforcement areas. Unfortunately, there are very few skilled lip-readers, and it is apparently a difficult skill to transmit, so the area is under-resourced. In this paper we investigate the possibility of making the lip-reading task more amenable to a wider range of operators by enhancing lip movements in video sequences using active appearance models. These are generative, parametric models commonly used to track faces in images and video sequences. The parametric nature of the model allows a face in an image to be encoded in terms of a few tens of parameters, while the generative nature allows faces to be re-synthesised using the parameters. The aim of this study is to determine if exaggerating lip-motions in video sequences by amplifying the parameters of the model improves lip-reading ability. We also present results of lip-reading tests undertaken by experienced (but non-expert) adult subjects who claim to use lip-reading in their speech recognition process. The results, which are comparisons of word error-rates on unprocessed and processed video, are mixed. We find that there appears to be the potential to improve the word error rate but, for the method to improve the intelligibility there is need for more sophisticated tracking and visual modelling. Our technique can also act as an expression or visual gesture amplifier and so has applications to animation and the presentation of information via avatars or synthetic humans.

Paper Details

Date Published: 28 September 2006
PDF: 9 pages
Proc. SPIE 6402, Optics and Photonics for Counterterrorism and Crime Fighting II, 640205 (28 September 2006); doi: 10.1117/12.689960
Show Author Affiliations
Barry J. Theobald, Univ. of East Anglia, Norwich (United Kingdom)
Richard Harvey, Univ. of East Anglia, Norwich (United Kingdom)
Stephen J. Cox, Univ. of East Anglia, Norwich (United Kingdom)
Colin Lewis, Ministry of Defence SA/SD (United Kingdom)
Gari P. Owen, Ministry of Defence SA/SD (United Kingdom)

Published in SPIE Proceedings Vol. 6402:
Optics and Photonics for Counterterrorism and Crime Fighting II
Colin Lewis; Gari P. Owen, Editor(s)

© SPIE. Terms of Use
Back to Top