Share Email Print

Proceedings Paper

A robust spatio-temporal face modelling approach using 3D multimodal fusion for biometric security applications
Author(s): Girija Chetty; Michael Wagner
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

In this paper, we propose a robust spatio-temporal face modelling approach based on multilevel fusion strategy involving cascaded fusion of hybrid multimodal fusion of audio-lip-face motion, correlation and depth features for biometric security application. The proposed approach combines the information from different audio-video based modules, namely: audio-lip motion module, audio-lip correlation module, 2D+3D motion-depth fusion module, and performs a hybrid cascaded fusion in an automatic, unsupervised and adaptive manner, by adapting to the local performance of each module. This is done by taking the output-score based reliability estimates (confidence measures) of each of the module into account. The module weightings are determined automatically such that the reliability measure of the combined scores is maximised. To test the robustness of the proposed approach, the audio and visual speech (mouth) modalities are degraded to emulate various levels of train/test mismatch; employing additive white Gaussian noise for the audio and JPEG compression for the video signals. The results show improved fusion performance for a range of tested levels of audio and video degradation, compared to the individual module performances. Experiments on a 3D stereovision database AVOZES show that, at severe levels of audio and video mismatch, the audio, mouth, 3D face, and tri-module (audio-lip motion, correlation and depth) fusion EERs were 42.9%, 32%, 15%, and 7.3% respectively for biometric identity verification scenario.

Paper Details

Date Published: 17 March 2008
PDF: 14 pages
Proc. SPIE 6944, Biometric Technology for Human Identification V, 694409 (17 March 2008); doi: 10.1117/12.778631
Show Author Affiliations
Girija Chetty, Univ. of Canberra (Australia)
Michael Wagner, Univ. of Canberra (Australia)

Published in SPIE Proceedings Vol. 6944:
Biometric Technology for Human Identification V
B.V.K. Vijaya Kumar; Salil Prabhakar; Arun A. Ross, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?