Share Email Print

Optical Engineering

Weighing classes and streams: toward better methods for two-stream convolutional networks
Author(s): Hoseong Kim; Youngjung Uh; Seunghyeon Ko; Hyeran Byun
Format Member Price Non-Member Price
PDF $20.00 $25.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

The emergence of two-stream convolutional networks has boosted the performance of action recognition by concurrently extracting appearance and motion features from videos. However, most existing approaches simply combine the features by averaging the prediction scores from each recognition stream without realizing that some classes favor greater weight for appearance than motion. We propose a fusion method of two-stream convolutional networks for action recognition by introducing objective functions of weights with two assumptions: (1) the scores from streams do not weigh the same and (2) the weights vary across different classes. We evaluate our method by extensive experiments on UCF101, HMDB51, and Hollywood2 datasets in the context of action recognition. The results show that the proposed approach outperforms the standard two-stream convolutional networks by a large margin (5.7%, 4.8%, and 3.6%) on UCF101, HMDB51, and Hollywood2 datasets, respectively.

Paper Details

Date Published: 23 May 2016
PDF: 5 pages
Opt. Eng. 55(5) 053108 doi: 10.1117/1.OE.55.5.053108
Published in: Optical Engineering Volume 55, Issue 5
Show Author Affiliations
Hoseong Kim, Yonsei Univ. (Korea, Republic of)
Youngjung Uh, Yonsei Univ. (Korea, Republic of)
Seunghyeon Ko, Yonsei Univ. (Korea, Republic of)
Hyeran Byun, Yonsei Univ. (Korea, Republic of)

© SPIE. Terms of Use
Back to Top