Share Email Print

Proceedings Paper

Statistical analysis of the effect of noise injection during neural network training
Author(s): Adel M. Abunawass; Charles B. Owen
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

It is commonly accepted that the modification of the weights during training of an Artificial Neural Network can be augmented by addition of a random element chosen from various distributions. This technique, referred to as Noise Injection, allows the training process to stochastically traverse a larger subset of the sample space, as well as escape from local minima. This paper examines the effect of noise injection on the training cycle of feedforward neural networks. Emphasis is placed on the gradient descent weight modification technique of the backpropagation model. Statistical examination is made of the distribution of the effect within the topology of the weight space, upon the inputs to individual units, upon training time, and on the total error of the network. Since the weights of the network can be considered together as an n-tuple, injection of noise can be statistically examined within that n- dimensional space. It is shown that, for stochastically independent random distributions, the effect on this weight space and on the inputs to individual units is dependent upon the number of weights in the network. The multivariate distribution of the vector modification during training becomes increasingly distorted as the network size increases, such that noise injection has a more significant, and less stable, effect. Problems with traditional approaches are examined and an alternative noise injection method based on network size is presented.

Paper Details

Date Published: 19 August 1993
PDF: 10 pages
Proc. SPIE 1966, Science of Artificial Neural Networks II, (19 August 1993); doi: 10.1117/12.152635
Show Author Affiliations
Adel M. Abunawass, Western Illinois Univ. (United States)
Charles B. Owen, Western Illinois Univ. (United States)

Published in SPIE Proceedings Vol. 1966:
Science of Artificial Neural Networks II
Dennis W. Ruck, Editor(s)

© SPIE. Terms of Use
Back to Top