Share Email Print

Proceedings Paper

Topological separation versus weight sharing in neural net optimization
Author(s): Omid M. Omidvar; Charles L. Wilson
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Recent advances in neural networks application development for real life problems have drawn attention to network optimization. Most of the known optimization methods rely heavily on a weight sharing concept for pattern separation and recognition. The shortcoming of the weight sharing method is attributed to a large number of extraneous weights which play a minimal role in pattern separation and recognition. Our experiments have shown that up to 97% of the connections in the network can be eliminated with little or no change in the network performance. Topological separation should be used when the size of the network is large enough to tackle real life problems such as fingerprint classification. Our research has focused on the network topology by changing the number of connections as secondary method of optimization. Our findings so far indicate that for large networks topological separation yields smaller network size which is more suitable for VLSI implementation. Topological separation is based on the error surface and information content of the network. As such it is an economical way of size reduction which leads to overall optimization. The differential pruning of the connections is based on the weight contents rather than number of connections. The training error may vary with the topological dynamics but the correlation between the error surface and recognition rate decreases to a minimum. Topological separation reduces the size of the network by changing its architecture without degrading its performance. The method also results in a network which is considerably smaller in size with a better performance.

Paper Details

Date Published: 16 December 1992
PDF: 12 pages
Proc. SPIE 1766, Neural and Stochastic Methods in Image and Signal Processing, (16 December 1992); doi: 10.1117/12.130853
Show Author Affiliations
Omid M. Omidvar, Univ. of the District of Columbia (United States)
Charles L. Wilson, National Institute of Standards and Technology (United States)

Published in SPIE Proceedings Vol. 1766:
Neural and Stochastic Methods in Image and Signal Processing
Su-Shing Chen, Editor(s)

© SPIE. Terms of Use
Back to Top