Share Email Print
cover

Proceedings Paper

Block training methods for perceptrons and their applications
Author(s): Anibal R. Figueiras-Vidal; Angel Navia-Vasquez
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Many techniques have been developed in recent years in order to speed up the training process of Multilayer Perceptrons (MLPs), such as accelerated versions of gradient descent, second order methods, Kalman related algorithms, and block learning methods; among them, the latter offer a reasonable way of reducing training effort and getting good results with MLPs. However, they usually find suboptimal solutions and, sometimes, the values obtained for the weights are excessively large. In this paper, we analyze the drawbacks of these block methods using a very simple test model, we propose several modifications by discarding samples (DS- LSB), using controlled perturbation (CP-LSB), and by means of a Reduced Sensitivity version (RS-LSB) and we also extend their forms to include some especial characteristics: we propose a robust training algorithm relying on Total Least Squares (TLS) minimizations (the so-called RS-TLS algorithm), useful for noisy training patterns, and we also present a method for training MLPs with general output cost function (the `RS-WLSB' algorithm). The advantages of these methods with respect to related algorithms are illustrated using several test problems. Finally, we extract some conclusions and propose, as a further work, the development of recursive implementations, able to learn on-line and to deal with non-stationary problems, and the application of block training methods to recurrent networks.

Paper Details

Date Published: 4 April 1997
PDF: 11 pages
Proc. SPIE 3077, Applications and Science of Artificial Neural Networks III, (4 April 1997); doi: 10.1117/12.271522
Show Author Affiliations
Anibal R. Figueiras-Vidal, Univ. Carlos III de Madrid (Spain)
Angel Navia-Vasquez, Univ. Carlos III de Madrid (Spain)


Published in SPIE Proceedings Vol. 3077:
Applications and Science of Artificial Neural Networks III
Steven K. Rogers, Editor(s)

© SPIE. Terms of Use
Back to Top