Share Email Print

Proceedings Paper

Pruning filters with L1-norm and standard deviation for CNN compression
Author(s): Xinlu Sun; Dianle Zhou; Xiaotian Pan; Zhiwei Zhong; Fei Wang
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Convolution Neural Networks (CNN) have evolved to be the state-of-art technique for machine learning tasks. However, CNNs bring a significant increase in the computation and parameter storage costs, which makes it difficult to deploy on embedded devices with limited hardware resources and a tight power budget. In recent years, people focus on reducing these overheads by compressing the CNN models, such as pruning weights and pruning filters. Compared with the method of pruning weights, the method of pruning filters does not result in sparse connectivity patterns. And it is conducive to the parallel acceleration on hardware platforms. In this paper, we proposed a new method to judge the importance of filters. In order to make the judgement more accurate, we use the standard deviation to represent the amount of information extracted by the filter. In the process of pruning, the unimportant filters can be removed directly without loss in the test accuracy. We also proposed a multilayer pruning method to avoid setting the pruning rate layer by layer. This holistic pruning method can improve the pruning efficiency. In order to verify the effectiveness of our algorithm, we do experiments with simple network VGG16 and complex networks ResNet18/34. We re-trained the pruned CNNs to compensate the accuracy loss caused by the pruning process. The results showed that our pruning method can reduce inference cost by up to 50% for VGG16 and 35% for ResNet18/34 on CIFAR10 with little accuracy loss.

Paper Details

Date Published: 15 March 2019
PDF: 9 pages
Proc. SPIE 11041, Eleventh International Conference on Machine Vision (ICMV 2018), 110412J (15 March 2019); doi: 10.1117/12.2523246
Show Author Affiliations
Xinlu Sun, Harbin Institute of Technology (China)
National Univ. of Defense Technology (China)
Dianle Zhou, National Univ. of Defense Technology (China)
Xiaotian Pan, Harbin Institute of Technology (China)
Zhiwei Zhong, National Univ. of Defense Technology (China)
Fei Wang, Harbin Institute of Technology (China)

Published in SPIE Proceedings Vol. 11041:
Eleventh International Conference on Machine Vision (ICMV 2018)
Antanas Verikas; Dmitry P. Nikolaev; Petia Radeva; Jianhong Zhou, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?