This paper will review recent advances in the applications of artificial neural network technology to problems in automatic target recognition. The application of feedforward networks for segmentation feature extraction and classification of targets in Forward Looking Infrared (FLIR) and laser radar range scenes will be presented. Biologically inspired Gabor functions will be shown to be a viable alternative to heuristic image processing techniques for segmentation. The use of local transforms such as the Gabor transform fed into a feedforward network is proposed as an architecture for neural based segmentation. Techniques for classification of segmented blobs will be reviewed along with neural network procedures for determining relevant features. A brief review of previous work on comparing neural network based classifiers to conventional Bayesian and K-nearest neighbor techniques will be presented. Results from testing several alternative learning algorithms for these neural network classifiers are presented. A technique for fusing information from multiple sensors using neural networks is presented and conclusions are made. 1

Adaptive control is seen as a two part problem control of plant dynamics and control of plant noise. The two parts are treated separately. An unknown plant will track an input command signal if the plant is driven by a controller whose transfer function approximates the inverse of the plant transfer function. An adaptive inverse identification process can be used to obtain a stable controller even if the plant is nonminimum phase. A model reference version of this idea allows system dynamics to closely approximate desired reference model dynamics. No direct feedback is used except that the plant output is monitored and utilized in order to adjust the paramters of the controller. Control of internal plant noise is accomplished with an optimal adaptive noise canceller. The canceller does not affect plant dynamics but feeds back plant noise in a way that minimizes plant output noise power. Key words. Adaptive control modeling identification inverse modeling noise cancelling deconvolution adaptive inverse control.

Many neural networks are constructed to learn an input-output mapping from examples. This problem is related to classical approximation techniques including regularization theory. Regularization is equivalent to a class of threelayer networks which we call regularization networks or Hyper Basis Functions. The strong theoretical foundation of regularization networks provides us with a better understanding of why they work and how to best choose a specific network and parameters for a given problem. Classical regularization theory can be extended in order to improve the quality of learning performed by Hyper Basis Functions. For example the centers of the basis functions and the norm weights can be optimized. Many Radial Basis Functions often used for function interpolation are provably Hyper Basis Functions. 1.

A multi-functional hybrid neural net is described. It is hybrid since it uses a digital hardware Hecht-Nielsen Corporation (HNC) neural net for adaptive learning and an optical neural net for on-line processing/classification. It is also hybrid in its combination of pattern recognition and neural net techniques. The system is multi-functional. It can function as an optimization and adaptive pattern recognition neural net as well as an auto and heteroassociative processor. I . W. JTRODUCTION Neural nets (NNs) have recently received enormous attention [1 -2] with increasing attention to the use of optical processors and a variety of new learning algorithms. Section 2 describes our hybrid NN with attention to Its fabrication and the role for optical and digital processors. Section 3 details Its use as an associative processor. Section 4 highlights is use in 3 optimization NN problems (a mixture NN a multitarget tracker (MTT) NN and a matrix inversion NN). Section 5 briefly notes it use as a production NN system and symbolic NN. Section 6 describes its use as an adaptive pattern recognition (PR) NN (that marries PR and NN techniques). 2. HYBRID ARCHITECTURE Figure 1 shows our basic hybrid NN [3]. The optical portion of the system is a matrix-vector (M-V) processor whose vector output P3 is the product of the vector at P1 and the matrix at P2. An HNC digital hardware NN is used during learning determine the interconnection weights forP2. If P2 is a spatial light modulator (SLM) its contents can be updated (using gated learning) from thedigital NN. The operations in most adaptive PR NN learning algorithms are sufficiently complex thatthey are best implemented digitally. In addition the learning operations required are often not well suited for optical realization for optimization NNs the weights are fixed and in adaptive learning learning is off-line and once completed the weights can often be fixed. Four gates are shown that determine the final output or the new P1 input neurons (Depending on the application). We briefly discuss these cases now and detail how each arises in subsequent sections. In most optimization NNs an external vector a is added to the P3 output (Gate I achieves this). In all NNJs a nonlinear thresholding (P3 outputs are 0 or 1 ) truncation (allP3 outputs lie between 0 and 1 ) or maximum selection (the maximum P3 output is set to 1 and all other P3 outputs to 0) SPIE Vol. 1294 Applications of Artificial Neural Networks (1990) / 31

We have examined the use of neural networks as a potential method of solving the multiple fault diagnostics problem that is when one symptom leads to several faults many symptoms leadto one fault or many symptoms lead to many faults. Current methods addressing this problem are brittle and slow. We have approached diagnostics from a pattern classification perspective in that we have constructed an input pattern from symptoms and classified that symptom pattern to an appropriate output class that corresponds to the fault that occurred. The system description was described in the first part of this two-part paper . In this second part we will report on the performance of the system. 1.

Is the process of inferring facts using neural networks a unique form of reasoning? Is there really a different type of reasoning separate and distinct from deduction and induction? Does there exist a single fundamental form of inference for reasoning symbolically, qualitatively, quantitatively, possibilistically (about "fuzzy" concepts), and probabilistically? YES, it is called abduction. This paper presents abduction and abductory induction. Abduction not only classifies the distinct type of reasoning performed when neural networks are applied, but gives a logical framework for expanding current neural network research to include network concepts not constrained by neuron analogies. These networks are called abductive networks. In describing abductive networks, this paper unveils the true source of the "power" of networks of functional elements. A practical machine learning tool for synthesizing abductive networks from databases of examples, called the Abductory Induction Mechanism (AIMTM), is also presented.

A brief review is presented of neural network tools for Automatic Target Recognition (ATR) . These tools include collective computation for implementing a variety of computational-vision techniques learning and adaptation for pattern recognition knowledge integration for expert-system capabilities and beyondsupercomputer- level hardware. As a specific example neural networks for stereo vision are introduced as a potentially fruitful approach to ATR. Preliminary results are presented which show substantial performance improvements over previous stereo algorithms for producing accurate dense displacement maps. These maps can be used in turn to derive accurate geometrical shape information that can result in improved recognition performance. 1.

Using the bimodal optical computer for training a hetroassociative memory of a neural network is introduced. The storage capacity of the trained hetroassociative memory is shown to be much higher than that for the Hopefield model. A comparison with the pseudoinverse model shows that in the proposed method the vector recall accuracy is higher when the number of vectors is greater than their size. This method has the potential of being faster than the other methods because of its parallel processing nature. I.

In this paper we discuss the advantages of two specific neural net models for the purposes of scene analysis. Two applications of those models are also presented. One concerns the prediction of sea water depth on the basis of the intensity of reflected light. The second one concerns the characterization of biological cells from slide images. The fractal dimension is found to be a very good parameter for the latter case. 1.

We describe the design of a target recognition system. The distinctive feature of this system is the integration of model-based and data-driven approaches to target recognition. This necessitates achievement of recognition through short-time behavior as opposed to longtime behavior of a dynamical system. The system also satisfies a list of natural requirements which includes locality of inferences (for efficient VLSI implementation) incorporation of prior knowledge multi-level hierarchies and iterative improvement. The architecture is uniformly parallel for low- and mid- as well as high-level vision. Robustness depends on collective effects rather than high precision of the processing elements. 1.

Real-time visual tracking is a difficult problem requiring high speed processing. We have previously reported a fast tracking algorithm (the Line Correlator Tracker (LCT) )12 capable of estimating displacement for a sequence of images using a conventional rectangular sensor. When used with a logarithmic-spiral sensor3, changes of scale can also be estimated. Although the algorithm can be implemented using sequential or parallel digital processing, a Hopfield-Tank (HT) network implementation is potentially simpler and faster.

A neural network solution to the problem of automatic threshold selection for image segmentation is presented. A multilayer perceptron is trained on a set of feature vectors extracted from gray scale imagery. The trained network then emulates the threshold selection behavior of its teacher. The thresholds obtained are used by a region based segmentation algorithm to partition the meaningful objects in the image into regions of constant gray level. Experimental results are given for a set of infrared imagery. 1.

We investigated the possibility of using an artificial neural network as a translation invariant target detector. The one-dimensional target detection model was a linear array of 20 pixels of which three were unity and the remainder were zero. Several multi-layer back progagation networks were able to distinguish a target consisting of three contiguous pixels from a nontarget three non-contiguous pixels. Under-constrained models were not trainable. A detailed analysis was done of one network with a small number of connections. The network solution appeared to be similar to a triplet correlat ion funct ion. 1.

Spectral analysis involving the determination of atomic and molecular species present in a spectm of multi—spectral data is a very time consulTLLng task, especially considering the fact that there are typically thousands of spectra collected during each experiment. Ixie to the overwhelming amount of available spectral data and the time required to analyze these data, a robust autorratic method for doing at least some preliminary spectral analysis is needed. This research focused on the development of a supervised artificial neural network with error correction learning, specifically a three—layer feed-forward backpropagation perceptron. The obj ective was to develop a neural network which would do the preliminary spectral analysis and save the analysts from the task of reviewing thousands of spectral frames . The input to the network is raw spectral data with the output consisting of the classification of both atomic and molecular species in the source.

The primary objective of this research is the development and testing of neural network models for two fundamental computer vision tasks: edge/line detection and texture analysis. In order to test the ability of the neural network models to detect patterns in images we used both remote sensing data and medical imagery. Neural network models for edge and line detection were used to detect geological lineaments in Landsat data. Neural network models for the analysis of image texture variations were used on ultrasonic images to distinguish patients with normal liver scans from patients with diffuse liver disease. 1.

This paper considers the application of a " global" optimization scheme to the training of multilayer perceptions for signal classifications. This study is motivated by the fact that the error surface of a multilayer perceptron is a highly nonlinear function of the parameters. Therefore the backpropagation which is a gradient descent algorithm converges to locally minimum structures. As an example we consider a signal classification problem where the optimum classifier has been shown to have an exponential complexity and the optimum decision boundary to be nonlinear and nonconvex. In this example when standard backpropagation is used to train the weights of a multi-layer perception the network is shown to classify with a " linear" decision boundary which corresponds locally to a minima of the neural network configurations. In this paper we propose to enhance the learning process of the network by considering an optimization scheme referred to as simulated annealing. This optimization scheme has been proven to be effective in finding global minima in many applications. We derive an iterative training algorithm based on this " global" optimization technique using the backpropagation as the " local" optimizer. We will verify the effectiveness of the learning algorithm via an empirical analysis of two signal classification problems. 1 PRELIMINARIES Artificial Neural Networks are highly interconnected networks of relatively simple processing units (commonly referred to as nodes e. g. perceptrons) which operate in parallel.

Backpropagation-trained neural networks with optical correlation inputs are used to predict target rotation and to synthesize simplified optical correlation filters for rotated targets.

A prototype system is being developed to demonstrate the use of neural network hardware to fuse multispectral imagery. This system consists of a neural network IC on a motherboard a circuit card assembly and a set of software routines hosted by a PC-class computer. Research in support of this consists of neural network simulations fusing 4 to 7 bands of Landsat imagery and fusing (separately) multiple bands of synthetic imagery. The simulations results and a description of the prototype system are presented. 1.

A neural network architecture is presented to segment and recognize objects of interest. The architecture consists of a region growing net to segment regions of interest by propagating activity through the neural lattice formed by the image pixels using local features as synaptic weights. A supervisory net utilizes the Fourier descriptors of the segmented region to characterize its shape and control the region growing net. The neural net is applied to segment objects of varying clarity to measure its performance and robustness in the presence of cluttered backgrounds and noisy object boundaries. Finally the segmentation and supervisory nets are combined and applied to the practical problem of segmenting roads from aerial photographs. 1.

Commonly used signal recognition techniques have many drawbacks. Many signal recognition and analysis techniques rely on complex algorithms which are computationally intensive and require a man in the loop to verify and validate the analysis. Classical signal recognition techniques often are unable to function in near real time. Classical techniques include nearest neighbor classifiers parameter range-matching statistical estimation techniques and heuristic algorithms. Hard-limited decision boundaries can produce ambiguities because signals which are outside these boundaries may not be classified or may be matched to more than one class. Lastly the addition of more signals to the signal recognition database of these algorithms typically necessitates additional software or hardware. We describe the use of an artificial neural network for classifying radar signals collected by a passive receiver. We selected neural classifiers because of their ability to adapt to the environment through training which allows them to avoid many of the problems associated with traditional classifiers. We used an artificial neural network employing a multilayer perceptron with back propagation to solve two common pattern recognition problems encountered when classifying radar signals. The first problem that of pulse sorting or deinterleaving is to sort individual pulses into " bins" associated with the radar emitter each pulse is from. The second problem that of radar classification or identifying radar type is to determine the type (and function) of the radar emitter represented by each bin

This paper describes the comparison of three classifiers for use in an automatic target recognition (ATR) system for millimeter wave (MMW) radar data. The three classifiers were the quadratic (Bayesian-like), the multilayer perceptron using a backpropagation training algorithm (termed backpropagation for short), and the counterpropagation network. Two data sets, statistical with four classes and real radar data with three classes, were used for training and testing all three classifiers. Three experiments were performed including: comparing the performances between the three classifiers on both the statistical feature set and the real radar data; optimal configuration for the backpropagation network; and the number of training iterations required for optimal performance using the backpropagation network before overtraining occurred.

A system under development for automated interpretation of oceanographic satellite images includes a Gulf Stream description module which uses a neural network which produces coefficients of an empirical orthogonal function (EOF) series representation of the Gulf Stream directly from processed satellite imagery. The Gulf Stream module consists of the EOF software and the neural network with input from an innovative edge detector. The Gulf Stream is the swiftest and most energetic current in the north Atlantic and meanders with a broad spectrum of variability on several spatial and temporal scales. Satellite observations provide a means to observe the Gulf Stream''s shape although clouds in JR imagery and other types of " noise" complicate interpretation. The Gulf Stream shape at any time may be represented as a series of complex EOFs (CEOFs) i. e. principal components which can be truncated after a relatively small number of terms (10) and still describe Gulf Stream shapes well (to within 10 km). These modes can be optimized from initial values with as few as 21 fixes on the position of the Gulf Stream axis using leastsquares estimation. The CEOFs interpolate between spatially intermittent observations of portions of the Gulf Stream as might come from JR imagery with partial cloud cover. The study described here tested whether a credible Gulf Stream can be produced using a neural network (simulated in software) that has inputs derived from

A method for optical flow estimation from an image sequence using a neural network is presented. Under hypothesis based on local rigidity translational motion and smoothness constraints a neural network is designed to estimate the optical flow. Experimental results using real world I. R. images are presented to demonstrate the efficiency of this method compared to Horn and Schunck algorithm. I -

This paper deals with the analog implementation of neocognitron based neural networks. All of Fukushima''s and related work on the neocognitron is based on digital computer simulations. To fully take advantage of the power of this network paradigm an analog electronic approach is proposed. We first implemented a 6-by-6 sensor network with discrete analog components and fixed weights. The network was given weight values to recognize the characters U L and F. These characters are recognized regardless of their location on the sensor and with various levels of distortion and noise. The network performance has also shown an excellent correlation with software simulation results. Next we implemented a variable weight network which can be trained to recognize simple patterns by means of self-organization. The adaptable weights were implemented with PETs configured as voltage-controlled resistors. To implement a variable weight there must be some type of " memory" to store the weight value and hold it while the value is reinforced or incremented. Two methods were evaluated: an analog sample-hold circuit and a digital storage scheme using binary counters. The latter is preferable for VLSI implementation because it uses standard components and does not require the use of capacitors. The analog design and implementation of these small-scale networks demonstrates the feasibility of implementing more complicated ANNs in electronic hardware. The circuits developed can also be designed for VLSI implementation. 1.

A neural network breadboard consisting of 256 neurons and with 2048 5 bit programmable synaptic weights has been constructed and is in use to demonstrate as a real time adaptive mirror control. The heart of the system is an array of custom 8 wide programmable resistor chips on a reconfigurable neuron board. The current system can processes a frame of 138 slope measurements producing 69 actuator position control signals at a rate ofup to 5000 frames per second. This system was designed to replace a conventional approach using a STAR array processor which is limited to a frame rate of less than 600 frames/sec. The 5000 frame/sec data rate is limited by the digital bandwidth of the wavefront sensor but still represents an equivalent processing speed of 140 megaflops. The analog bandwidth of the resistor/neuron board is better than 90 kHz which would allow frame rates as high as 900 kHz. The system architecture is expandable with complexity proportional to the number of actuators. The control algorithm is a variation of Hudgin''s algorithm modified to allow flexibility in the hardware setup. A specialized version of the LMS algorithm is used to train a sparsened pseudo-inverse response weight matrix and a geometrically determined feedback weight matrix. The training can be run while the analog network controls the mirror in real time. This allows the wavefront control algorithm to adjust to thermally induced

This report details the implementation of the Kohonen Self-Organizing Net on a 32-node Intel iPSC/1 HyperCube and the 25 performance improvement gained by increasing the dimensionality of the net without increasing processing requirements. 1. KOHONEN SELF-ORGANIZING MAP IMPLEMENTED ON THE INTEL iPSC HYPERCUBE This report examines the implementation of a Kohonen net on the Intel iPSC/l HyperCube and explores the performance improvement gained by increasing the dimensionality of the Kohonen net from the conventional two-dimensional case to the n-dimensional case where n is the number of inputs to the Kohonen net. In this example the Kohonen net performance is improved by increasing the dimensionality of the net without increasing the number of weights or nodes in the net and without increasing processing requirements. Kohonen in his Tutorial/ICCN 1 2 states that the dimensionality of the grid is not restricted to two but that maps in the biological brain tend to be two-dimensional. It is proposed that this is not a particularly severe restriction in the brain where not all inputs are connected to all nodes and specific maps can be formed for specific functions but in the case of the massively connected Kohonen net reducing all problems to two dimensions places an unnecessary burden on the learning process and necessarily causes the loss of information regarding the interrelationship of inputs and corresponding output clusters. Indeed reducing the dimension is a projection

This paper describes the application of a neural network model the Kohonen Feature Map to tactical and sensor data fusion. The problems presented by data fusion are reviewed with illustrations taken from the modern naval environment. The tasks of target and task identification for both automated surveillance and support tasks for naval operatives are highlighted as potential application domains for neural network based systems. The Kohonen Feature Map model is reviewed its limitations for practical applications examined and a discussion of how to overcome these problems is provided. Two alternative modular network architectures which use the Kohonen Feature Map as the basic unit are then described and their application to data fusion contrasted. 1.

The performance of neural networks with Ceedback is determined by electrically addressable magneto-optical (MO) spatial light . modula-'' tors (5LM). It was shown that MO SLM may be used. in neural networks but it''s necessary to modificate traditional MO SLM in order to achi eve high switching rate. In this work we present the xnod. ificated MO SLM structure MO media''s physical properties optimized. for achieving high rate switching the MO SLM operational margin and the way$ of its widening. 1 . T- TION An important role in artificial neural networks belongs to electrically addressible programmable SLM. They are particu]ry important if feedback is considered in neural network architecture''. The major requirements to SLIvI''s are high rate switching time providing effecti ye itteration process and high reliability. Among known up to date types of SLM (liquid crystalline RZLT ceramics etc) most attractive are recently developed MO SLM based on epitaxial films of Bisubsti tut?d iron garnet (BiRIG). Two variants of MOSLM''s are conimercially produced on of which (LISA ELSP etc) is oriented for nonmechanicalprinters while the other (SIGHT MOD Semetex) is developed mainly for plane display sys tems. Some of above mentioned MOSLM types have sufficient high rate switching time (typical frame rate for 128x128 SIGHT MOD is 20 ms). In SIGHT MOD variant with 128x128 array where nominal line resistance is in the range of 40-60 Ohm thermal limitation determine

This paper describes an approach to robotic control patterned after models of human skill acquisition and the organization of the human motor control system. The intent of the approach is to develop autonomous robots capable of learning complex tasks in unstructured environments through rule-based inference and self-induced practice. Features of the human motor control system emulated include a hierarchical and modular organization antagonistic actuation and multi-joint motor synergies. Human skill acquisition is emulated using declarative and reflexive representations of knowledge feedback and feedforward implementations of control and attentional mechanisms. Rule-based systems acquire rough-cut task execution and supervise the training of neural networks during the learning process. After the neural networks become capable of controlling system operation reinforcement learning is used to further refine the system performance. The research described is interdisciplinary and addresses fundamental issues in learning and adaptive control dexterous manipulation redundancy management knowledge-based system and neural network applications to control and the computational modelling of cognitive and motor skill acquisition. 296 / SPIE Vol. 1294 Applications of Artificial Neural Networks (1990)

This paper presents a scheme that uses a feedforward neural network for the learning and generalization of the dynamic characteristics for the starting of a dc motor. The goal is to build an intelligent motor starter which has a versatility equivalent to that possessed by a human operator. To attain a fast and safe starting from stall for a dc motor a maximum armature current should be maintained during the starting period. This can be achieved by properly adjusting the armature voltage. The network is trained to learn the inverse dynamics of the motor starting characteristics and outputs a proper armature voltage. Simulation was performed to demonstrate the feasibility and effectiveness of the model. This study also addresses the network performance as a function of the number of hidden units and the number of training samples. 1.

A new form of adaptive model-based control is proposed and experimentally evaluated. An Adaptive Model-Based Neural Network Controller (AMBNNC) uses multilayer perceptron artificial neural networks to estimate the payload during high speed manipulator motion. The payload estimate adapts the feedforward compensator to umnodeled system dynamics and payload variations. The neural nets are trained through repetitive presentation of trajectory tracking error data. The AMBNNC was experimentally evaluated on the third link of a PUMA56O manipulator. Servo tracking performance was evaluated for a wide range of payload and trajectory conditions and compared to a non-adaptive model-based controller. The superior tracking accuracy of the AMBNNC demonstrates the potential of our proposed technique. 1.

In this study automation of circuit board assembly process is considered using artificial neural networks with knowledge based systems. Basic issues of achieving intelligent conirol that can adopt to changing conditions of assembly process are discussed. The feasibility of using neural networks for pattern recognition and optimum kit insertion sequence generation is examined. The study provides a basic foundation for designing a conceptual architecture for adaptive intelligent control of circuit board assembly. Component recognition section of the architecture is tested using an ART network based on real time images and promising results are obtained. 1.

By replacing the sigmoid activation function often used in neural networks with an exponential function a probabilistic neural network (PNN) can be formed which computes nonlinear decision boundaries which are asymptotically Bayes-optimal. The PNN technique offers a tremendous speed advantage for problems in which the incremental adaptation time of back propagation is a significant fraction of the total computation time. For one application the PNN paradigm was 200 times faster than back propagation. Many potential applications exist for neural networks of this type three recent investigations will be discussed in this paper. PNN has been used successfully to detect submarines based on hydrophone data. The neural network was trained to recognize characteristic spectra for both ships and submarines and was subsequently able to detect 1 00 of independent test sequences observed from the same class of submarines used for training with no false detections. PNN was applied to the problem of identifying types of ships based on analysis of electronic emissions from these ships (ELINT reports). The same technique was then applied to identification of land platforms based on ELINT reports. A combination of deterministic preprocessing and the PNN was used to deduce the underlying causes of satellite communications failures based on measurements of S/N for individual communications links. Data were supplied by the Defense Communications Agency. 1 . THE PROBABILISTIC NEURAL NETWORK There is a striking similarity between the organization

This paper presents a new extension of the probabilistic neural network which utilizes one additional training pass to obtain significantly improved performance relative to the conventional probabilistic neural network. In addition it automatically sets certain algorithm parameters. The method substantially outperforms K-nearest neighbor techniques for the same number ofnodes. and it also offers performance competitive with LVQ2 which requires much longer training periods. 1.

The Probabilistic Neural Network (PNN) as described by Specht''3 has been successfully applied to a number of emitter correlation problems involving operational data for training and testing of the neural net work. The PNN has been found to be a reliable classification tool for determining emitter type or even identifying specific emitter platforms given appropriate representative data sets for training con sisting only of parametric data from electronic intelligence (ELINT) reports. Four separate feasibility studies have been conducted to prove the usefulness of PNN in this application area: . Hull-to-emitter correlation (HULTEC) for identification of seagoing emitter platforms . Identification of landbased emitters from airborne sensors . Pulse sorting according to emitter of origin . Emitter typing based on a dynamically learning neural network. 1 .

A general neural hypercolumn architecture is applied to radiographic weld images to locate regions of strong spatial intensity gradients. The hypercolumn output provides information on both the direction and the orientation of local spatial intensity gradients. These outputs can also be used to form an enhanced decimated image which can be processed for feature recognition. Parametric tuning of the architecture is discussed with particular emphasis on the requirements of the application. The performance of this architecture is compared with that of Sobel filters and other edge-detecting convolution masks. The possible representation of these various discrete convolution masks -including hypercolumns - as generalized non-adaptive neurons is also discussed. 1.

A locally linear neural network for interpolation and extrapolation is described. Desirable characteristics of this network include exact recall of training data optimal linear generalization of testing data and training in a known number of computational steps.

Optical implementations of neural networks based on the Hopfield model have always found it difficult to produce the negative weights required for the interconnecting synaptic matrix. One solution involves the addition of a positive offset to the weights to ensure that they all become non-negative but this introduces another problem as a dynamic (or time-dependent) threshold value is then required which may be difficult to implement. The dynamic threshold arises out of an inconsistency in the implementation. To overcome this our implementation employs a biased (non-negative) interconnection matrix which is dynamically multiplied by a diagonal matrix version of the neural state vector so that the same biasing is experienced. The above problem then no longer arises and we are left with a static threshold value. The method is demonstrated in an optoelectronic system employing 50 fully interconnected neurons. This uses a laser source for the neurons a computer driven liquid crystal spatial light modulator to produce the interconnection weights and a photodiode array with appropriate electronic circuitry to introduced the summing and thresholding aspect. 1. .

We describe a knowledge base browser based on a connectionist (or neural network) architecture that employs both distributed and local representations. The distributed representations are used for input and output thereby enabling associative noise-tolerant interaction with the environment. Internally all representations are fully local. This simplifies weight assignment and facilitates network configuration for specific applications. In our browser concepts and relations in a knowledge base are represented using " microfeatures. " The microfeatures can encode semantic attributes structural features contextual information etc. Desired portions of the knowledge base can then be associatively retrieved based on a structured cue. An ordered list of partial matches is presented to the user for selection. Microfeatures can also be used as " bookmarks" they can be placed dynamically at appropriate points in the knowledge base and subsequently used as retrieval cues. A proof-of-concept system has been implemented for an internally developed Honeywell-proprietary knowledge acquisition tool. 1.

Future projects with neural networks will require multiple individual network components. Current efforts along these lines are ad hoc. This paper relates the neural network to a classical device and derives a multi-part architecture from that model. Further it provides a Predicate Calculus variant for describing the location and nature of the trainings and suggests Resolution Refutation as a method for determining the performance of the system as well as the location of needed trainings for specific proofs. 2. THE NEURAL NETWORK AND A CLASSICAL DEVICE Recently investigators have been making reports about architectures of multiple neural networksL234. These efforts are appearing at an early stage in neural network investigations they are characterized by architectures suggested directly by the problem space. Touretzky and Hinton suggest an architecture for processing logical statements1 the design of this architecture arises from the syntax of a restricted class of logical expressions and exhibits syntactic limitations. In similar fashion a multiple neural netword arises out of a control problem2 from the sequence learning problem3 and from the domain of machine learning. 4 But a general theory of multiple neural devices is missing. More general attempts to relate single or multiple neural networks to classical computing devices are not common although an attempt is made to relate single neural devices to a Turing machines and Sun et a!. develop a multiple neural architecture that performs pattern classification.

This paper describes a novel learning scheme derived from a geometrical study of a onelayer autoassociative neural net that possesses hard limited neuron response functions. It is a study of the nonlinear mapping relations based on the concept of convex cone in the Ndimensional state space. (N is the nurrber of neurons in the neural system. ) This theoretical approach then allows us to derive a new learning scheme that appears to have many advantages over the conventional systems. 1 . It is a very fast and efficient onestep learning scheme . It does not require iteration processes to achieve the learning. 2 . Learning new mappings will not destroy old mappings already learned. 3. Learning of DISCRETE (or binery) mapping relations allows us to do pattern recognition in CONTINUOUS (analog) manner. 4. The maximum capacity of learning is much larger than those of the conventional systems. 5. It should be very easy to implement with conventional electronic coriponents. I . IrRWlXTI Supervised learning of a neural net has been studied quited extensively in the past decade. Besides the classical outerproduct rule [1] and the Ilebb'' 5 rule [2] many other supervised learning rules have also been studied and applied in practical areas. Rumelhart Hinton and Williams [3] derived the delta learning rule based on the gradient descent iteration approach. Widrow Hoff al et [46] derived another learning rule that forces

We briefly review the concept of computer aided medical diagnosis and more extensively review the the existing literature on neural network applications in the field. Neural networks can function as simple expert systems for diagnosis or prognosis. Using a public database we develop a neural network for the diagnosis of a major presenting symptom while discussing the development process and possible approaches. MEDICAL EXPERTS SYSTEMS COMPUTER AIDED DIAGNOSIS Biomedicine is an incredibly diverse and multidisciplinary field and it is not surprising that neural networks with their many applications are finding more and more applications in the highly non-linear field of biomedicine. I want to concentrate on neural networks as medical expert systems for clinical diagnosis or prognosis. Expert Systems started out as a set of computerized " ifthen" rules. Everything was reduced to boolean logic and the promised land of computer experts was said to be in sight. It never came. Why? First the computer code explodes as the number of " ifs" increases. All the " ifs" have to interact. Second experts are not very good at reducing expertise to language. It turns out that experts recognize patterns and have non-verbal left-brain intuition decision processes. Third learning by example rather than learning by rule is the way natural brains works and making computers work by rule-learning is hideously labor intensive. Neural networks can learn from example. They learn the results

A modular system of neural networks and a genetic algorithm are employed at a meta level to control solvers for the traveling salesman problem. The neural networks extract features of the input problem and recommend an instantiation of the solver to apply. The genetic algorithm conducts an adaptive search that further refines the parameters that control the work of the solvers. The result is a system that consistently produces very high quality solutions to traveling salesman problems. 1.

A practical approach to dealing with combinatorial decision problems and uncertainties associated with electronic combat through the use of networks of high-level functional elements called abductive networks is presented. It describes the application of the Abductory Induction Mechanism (AIMTM) a supervised inductive learning tool for synthesizing polynomial abductive networks to the electronic combat problem domain. From databases of historical expert-generated or simulated combat engagements AIM can often induce compact and robust network models for making effective real-time electronic combat decisions despite significant uncertainties or a combinatorial explosion of possible situations. The feasibility of applying abductive networks to realize advanced combat decision aiding capabilities was demonstrated by applying AIM to a set of electronic combat simulations. The networks synthesized by AIM generated accurate assessments of the intent lethality and overall risk associated with a variety of simulated threats and produced reasonable estimates of the expected effectiveness of a group of electronic countermeasures for a large number of simulated combat scenarios. This paper presents the application of abductive networks to electronic combat summarizes the results of experiments performed using AIM discusses the benefits and limitations of applying abductive networks to electronic combat and indicates why abductive networks can often result in capabilities not attainable using alternative approaches. 1. ELECTRONIC COMBAT. UNCERTAINTY. AND MACHINE LEARNING Electronic combat has become an essential part of the ability to make war and has become increasingly complex since

We formulated special Kohonen type associative memories and studied their utilization as the fast discriminator/classifier in munition systems. These nets are pretrainable and the response time for classification is minimal. The time required for adaptation is short. The applications investigated include [1] a targetaerosol discrimination problem and [2] change detection leading to an image based decision making in a realistic system. Appropriate pre-processing of the data are required for these methods to be effective. The outline of the algorithm for each application is described and the results obtained using generic data are illustrated. This work was supported by AFOSR under RIP and URRP programs. * On leave from East Texas State University Commerce Texas. ** On leave from Jarvis Christian College Hawkins Texas. 466 / SPIE Vol. 1294 Applications of Artificial Neural Networks (1990)

The system presented in this paper is a self-training visual inspection system that detects and classifies flaws in digitized images of surfaces with known characteristics. The system is composed of a control unit a signalprocessing unit and aclassifier. The control unitmonitors the generation andplacement of simulated flaws learning schedules and provides the teaching signal to the classifier. The signal processing unit simulates an optical area-to-line transformation for high speed processing and extracts regions of interest. The classifier is a multi-layer connectionist neural network. Two inspection tasks are targeted and the system''s performance in each is analyzed in terms of the neural network''s behavior including various learning schedules and application of three diagnostic tools developed in this work. 1.

The calibration of a 2-D displacement sensor that suffers from nonlinearities and cross talking using an Artificial Neural Networks (ANN) is described. The ANN is used as a Pattern Associator that is trained to perform the mapping between the sensor''s readings and the actual sensed properties. For comparison purposes a few methods were explored: 1 ) A three-layer ANN with a different number of hidden units trained by the Back Propagation (BP) method 2) Cerebellar Model Arithmetic Computer (CMAC) with a fixed number of quantizing functions and 3) Polynomial curve fitting technique. The results of the calibration procedure and recommendations are discussed. 2.

The error control problem in digital communication and information storage is presented as a - signal/pattern classification problem and the promising area of neural networks is exploited in correction of errors introduced in channel. The initial simulation results for a (7 Hamming and a (8 extended Hamming code with this approach show dramatic improvements in the Eb/No required to achieve a given probability of bit error. 1.

The decoding and error-correction of data transmitted over a noisy channel is in principle equivalent to the operation of a neural network performing as a content-addressable memory. For a successful application however the neural network has to be capable of storing arbitrary words and it has to be guaranteed that the stored words represent the only stable attractors of the memory. In this paper we present a novel high order neural network architecture that has these characteristics. The analog nature of the network can be used to perform softdecision decoding with any block code. The performance in terms of post-decoding bit error rate versus signal-to-noise ratio is demonstrated for two exemplary block codes. The comparison with a conventional decoding algorithm for a (15 cyclic redundancy code shows for example that the bit error rate at 7dB signal-to-noise ratio can be decreased by two orders of magnitude. 1.

Modeling of artificial neural networks is shown to depend on the programming decisions made in constructing the algorithms in software. Derivation of a common neural network training rule is shown including the effect of programming constraints. A method for constructing large scale neural network models is presented which allows for efficient use of memory hardware and graphics capabilities. Software engineering techniques are discussed in terms of design methodologies. Application of these techniques is considered for large scale problems including neural network segmentation of digital imagery for target identification. 1.

We here describe a neural network simulation environment that we have been developing at the Universität Stuttgart, West-Germany. Our network simulation environment is a tool to generate, test, and visualize medium-sized artificial neural networks. It consists of 3 major components: a simulator kernel that operates on an internal representation of the neural networks, a graphical user interface to interactively construct and change neural nets, and a compiler to generate the topology of large neural networks from a high level declarative network description language.

Neural networks were applied to the classification oftwo types ofacoustic emission (AE) events crack growth andfretting a simulated aiiframejoint specimen. Signals were obtainedfromfour sensors at different locations on the test specimen. Multilayered neural networks were trained to classify the signals using the error backpropagation learning algorithm enabling AE events arisingfrom crack growth to be distinguishedfrom those caused by fretting. In thispaper we evaluate the neural network classWcationperformancefor sensor location dependent and sensor location independent training and testing sets. Further we present a new training strategy which signcantly reduces the time required to learn large training sets using the error backpropagation learning algorithm and improves the generalization performance of the network. 1.

In this study we consider a family of polynomial classifiers and compare the performance of these classifiers to the Mahalanobis Distance classifier and to two types of artificial neural networks- -multilayer perceptrons and high-order neural networks. The well-known Mahalanobis Distance classifier is based on the assumption that the underlying probability distributions are Gaussian. The neural network classifiers and polynomial classifiers make no assumptions regarding underlying distributions. The decision boundaries of the polynomial classifier can be made to be arbitrarily nonlinear corresponding to the degree of the polynomial hence comparable to those of the neural networks. Further we describe both iterative gradient descent and batch procedures by which the polynomial classifiers can be trained. These procedures provide much faster training than that achievable for multilayer perceptrons trained via backpropagation. We demonstrate that the polynomial classifier and high-order neural network can be equated thereby implying that the classification power of the multilayer perceptron can be achieved while retaining the ease of training advantages of the polynomial classifiers. 1.

In this paper, we present a study of temporal information processing using a recurrent network and speech data. The task of this neural network is the formant tracking of continuous speech, a classical but difficult problem in speech processing. For better analysis of the results, this task is divided into two sub-tasks: a qualitative task (formant presence detection) and a quantitative task (formant frequency calculation). The network performs quite well for the qualitative task (91% detection) but not as well for the quantitative task (65% of correct estimation). Future work direction is discussed and several questions raised at the end of the paper.

Feedforward networks are artifical neural networks composed of successive layers of neurons. In this paper we use mathematical and geometric analysis to investigate properties of classifiers feedforward networks composed of neurons having threshold activation functions. The focus of this investigation is the relationship between the classification power of these networks and the number of layers composing them. We show that regions classifiable by simple twolayer classifiers also known as perceptrons are closed under region complementation and a limited form of region intersection. The proof of these results leads to a method for constructing twolayer classifiers for complicated regions. 1

In a three-valued neural logic network the strengths of the nodes are confined to the ordered pairs (1 (0 and (0 The first two pairs represent TRUE and FALSE respectively. The meaning of the third pair depends on the type of logic used. In Kleene''s logic (0 represents UNKNOWN. In Bochvar''s logic it represents MEANINGLESS. In this paper we introduced neural networks that allowed the strengths to be an ordered pair of real numbers the sum of which does not exceed one. Uncertainty is expressed by having a sum ofless than one. This allows us to treat uncertainties in facts rules as well as logical operations in a unifying way. 1.

One of the key features of Recursive Error Minimization (REM) equations is the efficient computation of the second derivative of mean square error with respect to each connection. The approximate integration of this derivative provides an estimate of the effect of removing or adding connections. A network with a minimum number of connections can then be found for a specific learning task. This has two important consequences. First the explanation of network decisions is much simpler with a minimum net. Second the computational load is a function of the number of connections. Results are presented for learning the English alphabet and for a simpler task learning the first seven letters of the alphabet. 1.

Neural networks are trained using Recursive Error Minimization (REM) equations to perform statistical classification. Using REM equations with continuous input variables reduces the required number of training experiences by factors of one to two orders of magnitude over standard back propagation. Replacing the continuous input variables with discrete binary representations reduces the number of connections by a factor proportional to the number of variables reducing the required number of experiences by another order of magnitude. Undesirable effects of using recurrent experience to train neural networks for statistical classification problems are demonstrated and nonrecurrent experience used to avoid these undesirable effects. 1. THE 1-41 PROBLEM The statistical classification problem which we address is is that of assigning points in ddimensional space to one of two classes. The first class has a covariance matrix of I (the identity matrix) the covariance matrix of the second class is 41. For this reason the problem is known as the 1-41 problem. Both classes have equal probability of occurrence and samples from both classes may appear anywhere throughout the ddimensional space. Most samples near the origin of the coordinate system will be from the first class while most samples away from the origin will be from the second class. Since the two classes completely overlap it is impossible to have a classifier with zero error. The minimum possible error is known as the Bayes error and

A novel Monte Carlo learning algorithm for a nonlinear neural network model with clipped synapses has been proposed. Computer simulation has shown that the storage capacity ot can be improved after the learning process by factors of 1. 2 to 2. 5 and the add-. ressability of noise tolerance is enhanced as well. 1.

Optical implementation of three stage Cbs network for multiprocessing interconnection networks using reversible nonlinear interface devices is presented. The proposed network is nonblocking rearrangable and is based on two-by-two optical switch that is capable of performing four basic switching actions. 1 .