Share Email Print

Proceedings Paper

Efficient activation functions for the back-propagation neural network
Author(s): Surender K. Kenue
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

The back-propagation algorithm is the most common algorithm in use in artificial neural network research. The standard activation (transfer) function is the logistic function s(x) equals 1/(1 + exp(-x)). The derivative of this function is used in correcting the error signals for updating the coefficients of the network. The maximum value of the derivative is only 0.25, which yields slow convergence. A new family of activation functions is proposed, whose derivatives belong to Sechn (x) family for n equals 1,2,.... The maximum value of the derivatives varies from 0.637 to 1.875 for n equals 1-6, and thus a member of the activation function-family can be selected to suit the problem. Results of using this family of activation functions show orders of magnitude savings in computation. A discrete version of these functions is also proposed for efficient implementation. For the parity 8 problem with 16 hidden units, the new activation function f3 uses 300 epochs for learning when compared to 500,000 epochs used by the standard activation function.

Paper Details

Date Published: 1 March 1992
PDF: 10 pages
Proc. SPIE 1608, Intelligent Robots and Computer Vision X: Neural, Biological, and 3-D Methods, (1 March 1992); doi: 10.1117/12.135110
Show Author Affiliations
Surender K. Kenue, General Motors Research Labs. (United States)

Published in SPIE Proceedings Vol. 1608:
Intelligent Robots and Computer Vision X: Neural, Biological, and 3-D Methods
David P. Casasent, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?