Share Email Print

Proceedings Paper

Inheritance of knowledge in neural networks via symbolic algebra
Author(s): Samir I. Sayegh
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

The theme is that of `inheritance' and the use of symbolic algebra to implement it. The notion of inheriting knowledge between networks is crucial since training a network can be exceedingly slow. Partial information acquired through long experience (learning epochs) should be `transferrable.' A technique using the notion of `gluing' networks has been pioneered by Alex Waibel of CMU. However, this technique cannot be considered true inheritance since the component networks are trained on similar but different subtasks that are later `concatenated.' The approach presented here is based on the observation that even when the problem is not separable, one can get a reasonable performance out of a 2 layer network. Its training is fast and its only minimum is often below that of many of the local minima of the corresponding multi-layer networks. After training such a net, if one can transfer the knowledge to a multi-layer net, not only would one have saved valuable training time, but one would have avoided many of the local minima associated with the multi-layer network. Training can then proceed with the task of the multi-layer net reduced to improving the performance of the 2-layer net, instead of having to start from scratch. Equations corresponding to this approach are derived. They can be written for specific topologies and solved exactly (toys) or approximately (larger problems) using symbolic algebra.

Paper Details

Date Published: 1 July 1992
PDF: 7 pages
Proc. SPIE 1710, Science of Artificial Neural Networks, (1 July 1992); doi: 10.1117/12.140117
Show Author Affiliations
Samir I. Sayegh, Purdue Univ. (United States)

Published in SPIE Proceedings Vol. 1710:
Science of Artificial Neural Networks
Dennis W. Ruck, Editor(s)

© SPIE. Terms of Use
Back to Top