Share Email Print

Proceedings Paper

Computationally efficient training of deep neural networks via transfer learning
Author(s): Diane Oyen
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Transfer learning is a highly successful approach to train deep neural networks for customized image recognition tasks. Transfer learning leverages deep learning models that are previously trained on massive datasets and re-trains the network for a novel image recognition dataset. Typically, the advantage of transfer learning has been measured in sample efficiency, but instead, we investigate the computational efficiency of transfer learning. A good pre-trained model provides features that can be used as input to a new classifier (usually the top layers of a neural network). We show that if a good pre-trained model is selected, that training a new classifier can be much more computationally efficient than training a deep neural network without transfer learning. The first step in transfer learning is to select a pre-trained model to use as a feed-forward network for generating features. This selection process either uses human intuition/convenience or a methodical but computationally intensive validation loop. Instead, we would prefer a method to select the pre-trained model that will produce the best transfer results for the least amount of computation. To this end, we provide a computationally efficient metric for the fit between a pre-trained model and a novel image recognition task. The better the fit, the less computation will be needed to re-train the pre-trained model for the novel task. As machine learning becomes ubiquitous, highly-accurate trained models will proliferate and computationally efficient transfer learning methods will enable rapid development of new image recognition models.

Paper Details

Date Published: 14 May 2019
PDF: 9 pages
Proc. SPIE 10996, Real-Time Image Processing and Deep Learning 2019, 1099607 (14 May 2019); doi: 10.1117/12.2519097
Show Author Affiliations
Diane Oyen, Los Alamos National Lab. (United States)

Published in SPIE Proceedings Vol. 10996:
Real-Time Image Processing and Deep Learning 2019
Nasser Kehtarnavaz; Matthias F. Carlsohn, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?