Share Email Print

Proceedings Paper

Techniques for avoiding local minima in gradient-descent-based ID algorithms
Author(s): Joan L. Brierton
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

One nonlinear adaptive approach to generating target recognition algorithms, the distributed connectionist approach, is also referred to as a neural network. These algorithms frequently employ a gradient descent technique, such as the back propagation learning algorithm, to find a mapping that separates the n dimensional feature space into m recognizable classes. Gradient descent techniques are known to be limited by a characteristic referred to as the `local minima' problem. During the search for an optimum solution or global minima, these techniques can encounter local minima from which they cannot escape due to the `steepest descent' nature of the approach. However, several training techniques used to speed up training or to otherwise optimize these adaptive learning algorithms have side effects which can obviate this local minima problem. We will define a local minima problem with respect to the 1D target ID problem. Appropriate terminology and an error space relevant to the 1D range profile problem will be presented. Four techniques, dynamic architecture definition, weight pruning, adaptive learning rate selection and dynamic training set generation used to optimize training for the multilayer perceptron will be summarized. An analytical explanation of a common underlying mechanism which allows escape from local minima and is shared by these techniques is presented. Some additional advantages are provided by one of the four techniques, the dynamic training set technique. Evidence of these advantages, consistently high quality results, the automatic identification of anomalous signatures in the data base and simple implementation, will be presented.

Paper Details

Date Published: 10 June 1997
PDF: 6 pages
Proc. SPIE 3066, Radar Sensor Technology II, (10 June 1997); doi: 10.1117/12.276095
Show Author Affiliations
Joan L. Brierton, Hughes Aircraft Co. (United States)

Published in SPIE Proceedings Vol. 3066:
Radar Sensor Technology II
Robert Trebits; James L. Kurtz, Editor(s)

© SPIE. Terms of Use
Back to Top