A family of "smart sensors" is described for implementation with very large scale integrated (VLSI) circuits and for use in information gathering, security and munitions fuzing systems. The sensors are based on a micro - signal processor technique capable of identifying, detecting, and/or classifying particular target signatures in real - time with a high degree of reliability and flexibility while meeting system constraints of low cost, small size, and very low power dissipation. A basic design is capable of accommodating multichannel inputs consisting of a variety of transducers. Specifically, applications are described using seismic and acoustic inputs. The algorithms used in these smart sensors are based on signal processing using spectral analysis. Microprocessor - based implementations, including analog conditioning, the micro itself, a special multiplier for more effective computations, and memory have been built and tested demonstrating the capability of a compact sensor system for intrusion and target recognition.

Multi-detector IR imaging focal plane arrays possess large detector-to-detector dark current (offset) and responsivity (gain) variations which can completely mask the useful thermal signatures in IR scenes. Conventional detector compensation techniques require uniform temperature references of constant radiance over the entire field of view and a mechanical/ electro-optical shutter. This detracts from the mechanical simplicity of multi-detector staring focal planes (which require no scanning). This paper describes a real-time offset and responsivity (gain) compensation technique which dispenses with temperature references and shutters in staring focal planes. The technique makes use of the IR scene itself for calibration and continuously updates the compensation coefficients without interrupting the field of view with a shutter or a temperature reference. The results of real-time simulations of this technique with a number of sensors are presented. Real-time hardware implementation considerations suggest that the technique can be implemented with the addition of very little hardware to a conventional compensation technique requiring temperature references. The technique is also suitable for multi-detector scanning focal planes and for the removal of shading in TV sensors as well.

An intensified linear array instrument has been made by fiberoptically coupling a micro-channel plate proximity focused image intensifier tube to a Reticon RL512S linear photo-diode array. One of the principal advantages of intensification is that the sensitivity of the array can be increased to the photon-counting level. The pixel (p) readout rate is adjustable from 30 kp/s to 300 kp/s, so that rapid data acquisition is possible, and correction for nonuniformities in the instrument response can easily be made by using standard data processing equipment. The measured signal transfer, uniformity, spectral response, MTF and dark current characteristics of this instrument are presented, and alternate instrument designs are discussed. The electronic gating capability of an optional image tube assembly allows optical events as fast as several nanoseconds to be captured. This instrument has been designed in modules so that it can be coupled to a wide variety of existing spectroscopic optical systems. The modular design allows the optical interface flange, the image intensifier tube assembly, and the Reticon array assembly to be easily and quickly changed. It is possible to place the input face of the image intensifier directly in the focal plane of optical systems, such as spectrographs, so that no additional optical elements are required. This versatile instrument can provide more efficient data acquisition of spectroscopic or other linear array optical signals with less distortion than other instruments which do not use microchannel plate proximity focused image intensifiers.

The Microchannel Spatial Light Modulator (MSLM) is a versatile, optically-addressed, highly -sensitive device that is well suited for low-light-level, real-time, optical information processing. It consists of a photocathode, a microchannel plate (MCP), a planar acceleration grid, and an electro-optic plate in proximity focus. A framing rate of 20 Hz with full modulation depth, and 100 Hz with 20% modulation depth has been achieved in a vacuum-demountable LiTaO3 device. A halfwave exposure sensitivity of 2.2 nJ/cm2 and an optical information storage time of more than 2 months have been achieved in a similar gridless LiTaO3 device employing a visible photocathode. Image processing operations such as analog and digital thresholding, real-time image hard clipping, contrast reversal, contrast enhancement, image addition and subtraction, and binary level logic operations such as AND, OR, XOR, and NOR can be achieved with this device. This collection of achievable image processing characteristics makes the MSLM potentially useful for a number of smart sensor applications.

The Ground-Based Electro-Optical Deep Space Surveillance (GEODSS) system will employ a SIT vidicon camera to detect satellites. Imaging charge-coupled devices (CCDs) are being evaluated as a means to improve GEODSS capability. This paper presents performance models and uses them to evaluate currently available CCDs and to determine the characteristics of the ideal CCD array for GEODSS. The analysis includes an approach for determining the CCD pixel size which maximizes signal-to-noise ratio; this approach can be used in many applications. The impact of response nonuniformity and a simple compensation method are also discussed. The combination of a suitable CCD array, response nonuniformity compensation, and moving target indicator (MTI) processing is expected to substantially increase the detection and search rate capability of the GEODSS system.

This paper describes a charge-coupled device (CCD) imaging sensor which is being developed to provide high sensitivity and high scan coverage rate for automatic detection in electro-optical, deep-space, satellite-surveillance applications. The visible-spectrum sensor focal plane is a mosaic of 100 x 400 pixel CCD imagers which are operated in the time delay and integration (TDI) mode permitting continuous high-rate scan coverage while integrating target light levels for high sensitivity. The polysilicon gate, front illumi9ated devices have 30 x 30 μm pixels. Room temperature dark current is less than 3 nA/cm and noise levels are 10-20 electrons. Several aligned rows of imaging CCD arrays located on the focal plane in the scan direction provide multiple samples of the field-of-view for automatic MTI detection processing. Five-chip focal planes have been assembled with positional errors of less than 4 μm. The MTI Signal Processor features imager defect removal, parameter estimation, adaptive thresholding, digital delay, and spatial correlation for MTI. This paper includes descriptions of the sensor system, the CCD imager design and characterization, the signal processor, and a summary of test results of the initial sensor system operated on space surveillance telescopes at the GEODSS Experimental Test System, White Sands Missile Range.

Star trackers based on the use of charge-coupled device (CCD) imaging arrays have demonstrated substantial performance gains relative to more conventional designs. For many space applications, the most important gain is the accuracy improvement associated with a stable, precisely known, CCD geometry. This inherent accuracy can be fully exploited using star image centerfinding techniques to measure the image location to a small fraction of a CCD picture element. In this paper we present the results of an extensive series of laboratory tests aimed at exploring the accuracy limits of this technique. Simulated star images were moved over the CCD, permitting measurement of tracking errors as small as 1/1000 pixel. After tests on both frontside and backside illuminated (thinned) CCDs, we conclude that tracking accuracy better than 1/50 pixel can be achieved.

Recent star tracker developments at JPL which utilize microprocessor-controlled large area imaging CCD arrays have included several novel techniques for maximizing the efficiency with which information can be extracted as well as corrected for electronic and opto/ mechanical distortions. Scanning algorithms which maximize the rate of obtaining target information, coupled with autonomous control of CCD and signal chain parameters, result from efficient task allocations between electronic hardware elements and operating software. Trade-offs between optical system and image processing software complexity allow sub-pixel star and image tracking using badly aberrated point spread functions by means of power series correction equations which define ideal image centroids from the aberrated image centroids. Star trackers employing these techniques currently under development at JPL are described.

The Information Adaptive System (IAS) is an element of the NASA End-to-End Data System (NEEDS) Phase II and is focused toward onboard image processing. Since the IAS is a data preprocessing system which is closely coupled to the sensor system, it serves as a first step in providing a "Smart" imaging sensor. Some of the functions planned for the IAS include sensor response nonuniformity correction, geometric correction, data set selection, data formatting, packetization, and adaptive system control. The inclusion of these sensor data preprocessing functions onboard the spacecraft will significantly improve the extraction of information from the sensor data in a timely and cost effective manner and provide the opportunity to design sensor systems which can be reconfigured in near real time for optimum performance. The purpose of this paper is to present the preliminary design of the IAS and the plans for its development.

In photographing the solar corona from a ground-based coronagraph, the scattered light present in the Earth's atmosphere greatly affects the exposure times. Described in this paper is a sky photometer used to sample the sky brightness to within .3 solar radius of the solar limb, crucially important for precisely sensing the coronal field background. The detector used is a commercially available MCC-401 optical detector set up to oscillate according to the amount of light falling upon it. The output of this detector is then used to regulate the exposure times via the 6502 micro-processor controlling the coronagraph.

Techniques are presented which extend pattern classification techniques beyond the traditional high-resolution, high-contrast imagery cases. In particular, smart sensors can now be designed to operate with poorer quality imagery resulting from various blurring phenomena including diffraction limiting aperture effects, poor quality optical effects, and atmospheric turbulence effects. This results in smaller aperture electro-optical smart sensors at lower cost and with increased reliability and maintainability, characteristics.

The practical scene matching problem presents certain complications which must extend classical image processing capabilities. In this paper, we consider certain aspects of the scene matching problem which must be addressed by a smart sensor for terminal homing. In the first section, we outline a philosophy for treating the matching problem for the terminal homing scenario. Later, we consider certain aspects of the feature extraction process and symbolic pattern matching.

Using spatial images as the target signatures, we evaluate qualitatively the effects of target fading and signal- and background-generated shot noise on the accuracy and precision of estimates of target angular position (TAP). The estimates are derived from the photocounts registered by an array of detectors that sense image radiation noncoherently. The semi-classical theory of photocounting is used to relate the statistics of image radiation to the statistics of photocounts at the output of photodetectors. The photocounting statistics are taken to be Poisson and negative binomial for the limiting cases of self-luminous and laser-illuminated targets, respectively; the number of degrees of freedom associated with the negative binomial distribution provides a convenient measure of signal coherence. Preliminary analysis, with the help of the computer simulations, has established that performance of the TAP estimates (or algorithms) depends upon target fading which is, in turn, a function of target coherence and geometry. This dependence on target fading is most pronounced for the class of algorithms which employ the entire image to estimate its angular position (rather than only the image edge).

The problem of multiple image registration is that of finding n subimages in a larger image (Search area S) which best match the n smaller images (windows or references) obtained from different sensors, assuming that all smaller images are completely located within the larger image. Although correlation and sequential similarity detection algorithms are commonly used, the method of moments which has been successfully used for automatic classification of an unknown pattern as one of several known patterns can also be used for digital image registration. This paper compares the computational efficiency of the three methods stated above for software as well as hardware implementations. For single image registration problem, the moments method requires more computation time than correlation and sequential similarity detection methods. However, for the multiple image registration problem the moments method becomes computationally more efficient as the number of windows increases. This paper shows that the moments method requires less computation time if implemented in software and less hardware for real time implementation when the number of windows is large.

We report on the development of a new class of parallel computation algorithm for low-level scene analysis. The algorithm is a high resolution, high speed estimator for boundary extraction of simple objects imaged under noisy conditions. We explain the algorithm structure and underlying physical models; we then present demonstrative pictorial examples of application to synthetic test imagery. We next introduce a generalization of the algorithm wherein a hierarchical variable resolution search is employed to gain major improvements in algorithm convergence speed and robustness. We discuss the importance of making the algorithm adaptive to local image statistics and show that the algorithm parallel-window topology is consonant with this goal. We present further experimental results that depict the generalized algorithm applied to real data bases; these results demonstrate that even simple adaptation models can substantially improve algorithm convergence accuracy.

Many techniques for segmenting images have been developed over the past decade. Proponents of each of the techniques feel that their method is the best, but this is generally based on subjective criteria. This paper reports progress on an effort to develop a set of factors which will allow an objective evaluation of segmenter performance on a comparative basis by analyzing the performance of several segmentation approaches on a common data base. Three generically distinct segmentation approaches were analyzed and their performance measured using the criteria selected. A comparison of techniques based on the results of the test showed the relative performance to be consistent with intuitive expectations. The greatest potential for refining this evaluation approach appears to be in the areas of selection and quantification of the data base and expansion of the set of factors.