Share Email Print

Proceedings Paper

Multimodal data fusion for object recognition
Author(s): Vladimir Knyaz
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Multi-spectral imagery provides wide possibilities for improving quality of object detection and recognition due to better visibility of different scene features in different spectral ranges. To use the advantage of multi-spectral data the relation between different types of data is required. This relation is provided by capturing data using calibrated, aligned and synchronized sensors. Also geo-spatial data in form of geo-referenced digital terrain models can be used for establishing geometric and semantic relations between different types of data. The presented study considers the problem of object recognition based on two data sources: visible and thermal imagery. The main aim of the performed study was to evaluate the performance of different convolutional neural network models for multimodal object recognition. For this purpose a special dataset was collected. The dataset contains synchronized visible and thermal images acquired by several sensor based on unmanned aerial vehicle. The dataset contains synchronized color and thermal images of urban and suburb scenes gathered in different seasons, different times of day and various weather conditions. For convolutional neural network training the dataset was augmented by model images created using object 3D models textured by real visible and thermal images. Several convolutional neural network architectures were trained and evaluated on the created dataset using different splits to estimate the influence of training data on object recognition performance.

Paper Details

Date Published: 21 June 2019
PDF: 12 pages
Proc. SPIE 11059, Multimodal Sensing: Technologies and Applications, 110590P (21 June 2019); doi: 10.1117/12.2526067
Show Author Affiliations
Vladimir Knyaz, GosNIIAS (Russian Federation)
Moscow Institute of Physics and Technology (Russian Federation)

Published in SPIE Proceedings Vol. 11059:
Multimodal Sensing: Technologies and Applications
Ettore Stella, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?