Classifying videos according to their content is a common task across various contexts, as it allows effective content tagging, indexing and searching. In this work, we propose a general framework for video classification that is built on top of several neural network architectures. Since we rely on a multimodal approach, we extract both visual and textual features from videos and combine them in a final classification algorithm. When trained on a dataset of 30 000 social media videos and evaluated on 6 000 videos, our multimodal deep learning algorithm outperforms shallow single-modality classification methods by a large margin of up to 95%, achieving overall accuracy of 88%.

Date Published: 1 October 2018
PDF: 8 pages
Proc. SPIE 10808, Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2018, 108082U (1 October 2018); doi: 10.1117/12.2501679

Published in SPIE Proceedings Vol. 10808:
Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2018
Ryszard S. Romaniuk; Maciej Linczuk, Editor(s)