Share Email Print
cover

Proceedings Paper

A multimodal temporal panorama approach for moving vehicle detection, reconstruction, and classification
Author(s): Tao Wang; Zhigang Zhu
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Moving vehicle detection and classification using multimodal data is a challenging task in data collection, audio-visual alignment, data labeling and feature selection under uncontrolled environments with occlusions, motion blurs, varying image resolutions and perspective distortions. In this work, we propose an effective multimodal temporal panorama approach for the task using a novel long-range audio-visual sensing system. A new audio-visual vehicle (AVV) dataset for moving vehicle detection and classification is created, which features automatic vehicle detection and audio-visual alignment, accurate vehicle extraction and reconstruction, and efficient data labeling. In particular, vehicles' visual images are reconstructed once detected in order to remove most of the occlusions, motion blurs, and variations of perspective views. Multimodal audio-visual features are extracted, including global geometric features (aspect ratios, profiles), local structure features (HOGs), as well various audio features (MFCCs, etc). Using radial-based SVMs, the effectiveness of the integration of these multimodal features is thoroughly and systemically studied. The concept of MTP may not be only limited to visual, motion and audio modalities; it could also be applicable to other sensing modalities that can obtain data in the temporal domain.

Paper Details

Date Published: 24 May 2012
PDF: 12 pages
Proc. SPIE 8389, Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR III, 83890V (24 May 2012); doi: 10.1117/12.918793
Show Author Affiliations
Tao Wang, The Graduate Ctr., CUNY (United States)
The City College of New York (United States)
Zhigang Zhu, The Graduate Ctr., CUNY (United States)
The City College of New York (United States)


Published in SPIE Proceedings Vol. 8389:
Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR III
Tien Pham, Editor(s)

© SPIE. Terms of Use
Back to Top