Industrial machine vision requires speed, accuracy, and low cost. The SUPERSIGHT vision system at General Motors Research locates industrial parts with high accuracy even with low-resolution video cameras. This leads to large savings in the hardware costs of subsequent image processing electronics, and considerable speed improvements over current methods. With a prototype system employing a Reticon 100x100 photodiode array camera, the relative locations of lines, circles, and ellipses in a plane could be routinely determined to 1/10 to 1/20 pixel accuracy, with 1/50 pixel (1 part in 5000) accuracy best case. Theory indicates that there is no upper bound on the accuracy that can be achieved, with the only limiting factor being noise. Primary sources of noise were hypothesized to be fixed-pattern camera noise for within-frame location measurements, and digitization synchronizing error (frame jitter) and mechanical vibration for between-frame measurements. Gaussian digital filters, pixel-by-pixel correction of gain and bias variations, and higher quality solid-state video cameras can reduce the influence of camera fixed-pattern noise. Frame averaging and improved digitizers can reduce frame jitter. In extensions to 3-D surface and edge analysis, multiple-order Gaussian-derivative filters allowed for an efficient means of determining the regression coefficients useful for specifying the 3-D surface shapes of objects from shading, range or stereo data. The Krawtchouck polynomials, the discrete form of the Gaussian derivative functions, allowed more accurate estimates than current facet models of the properties of surface patches, and the locations of edges. Such Gaussian derivative-like operators are similar to the vision operators used by the primate brain, as measured neurophysiologically.

It is widely known that the human visual system is able to achieve super-resolution under ideal viewing conditions in its sensing of local form, local motion and stereo disparity. The practical limits appear to be something less than 0.1 receptor spacings (pixels) of spatial displacement and better than 1 degree in local orientation. Yet it seems to achieve this performance instantaneously and with utmost ease. Classical computer vision is unable to compare either in absolute capability or apparent simplicity. We have constructed a computer simulation of early human vision with the emphasis being on our particular interpretation of the important neural components and on simplicity. This simulation has now been working well and achieving super-resolution on practical images for some time. This paper presents the theory and practice of performance limits for this simulation. It is shown that theoretical and practical performance limits are in good agreement. Practical performance includes pixel by pixel edge location to within 0.03 pixels, local motion and disparity measurement between pairs of sampled frames pixel by pixel to better than 3% and pixel by pixel orientation measurement to better than 1 degree. All these are achieved by direct pipeline processing with no iterative procedures. They compare very closely with observed limits of human visual performance.

The properties and characteristics of a new model of primate spatial vision are described and compared with human vision. The model employs a multiple level stack of graduated receptive field sizes whose sampling densities progressively decrease with level while maintaining a constant space-bandwidth product. The effect of this structure is to increase receptive field size with eccentricity, whilst retaining a constant number of samples per level, coupled with a set of octave-related spatial frequency filters at the fovea. Such an architecture, which exploits the Heirarchical Discrete Correlator of Burt (1981)17, correctly mimics the visual cortical mapping function of Schwartz (1983) 14 yet it has the valuable property that it can produce invariant responses to local changes in the size and position of features in the image. This architecture requires only local connections at each level, so producing the type of uniformity that is commonl observed as a feature of neural processing in the striate cortex (Rubel & Wiesel, 1974)1-1. An intrinsic characteristic of the model is the concept of "attention area", representing the spatial extent of each level in the stack, and this concept helps to explain the high efficiency of human visual search in terms of hierarchical scanning. Our model has been simulated on an INTELLECT image processor using many different natural scene inputs. Analysis of the results has revealed possible mechanisms for human visual accommodation and neural gain control, which have enabled us to program the simulator to control reliably the focus and gain of its own TV camera input. Simple mechanisms have also been programmed that allow rapid detection of scene changes and consequent shift of attention, together with smooth pursuit of targets through natural scenes. While the simulation is slow, being a serial manifestation of a parallel original system, it has demonstrated the outstanding value of the data compression that is inherent in this form of architecture. The advantages of such a homomorphic system for intelligent machines are, we feel, obvious.

When we gaze upon a scene, our brains combine many types of locally ambiguous visual information to rapidly generate a globally unambiguous representation of form-and-color-in-depth. In contrast, many models of visual perception are specialized models which deal with only one type of information-for example, boundary, disparity, curvature, shading, color, or spatial frequency information. For such models, other types of signals are often contaminants, or noise elements, rather than cooperative sources of ambiguity-reducing information. This state of affairs raises the basic question: What new principles and mechanisms are needed to understand how multiple sources of visual information preattentively cooperate to generate a percept of 3-D form?

As vision system projects move from laboratory investigations to factory floor installations, the absolute necessity of and advantages created by optimized lighting systems is being widely acknowledged. Frequently heard phrases such as "garbage in - garbage out", "the system can be no better than the image it starts with", and "if I could only get a little better contrast in the image, I could ..." attest to this acknowledgement. Furthermore, to quote Douglas Goodman, "In general, illumination is more critical in automated optical inspection than in visual inspection, since detectors and computer programs lack the dynamic range and forgiveness of the human visual system." The Machine Vision Association of SME attests to the interest in illumination by offering an Optics and Lighting session at Vision 86 and an Optics and Lighting hands on workshop.

Multiple lines of laser light can be used in machine vision, as an illumination source for three-dimensional vision, for part registration, detecting and determining proper placement of parts against low-contrast backgrounds, gauging parts with complicated geometries, and situations currently employing a single line of light illumination. Using multiple lines of laser light in conjunction with an electronic image processor can produce real-time part registration and inspection. This technique has also been used to show part deflection or distortion in real time. This paper describes how to generate and project multiple lines of laser light and how to use these lines in machine vision.

The purpose of the lighting in an industrial machine vision application is to produce an image that is well matched to the camera and vision system. The brightest areas of the scene should cause a sensor illuminance that is just below the camera's saturation level and the video signal from the darkest significant areas of the scene should lie just above the vision system's noise level. This paper describes the fundamental principles and the techniques used to design lighting that meets these requirements. The camera's transfer function, the vision system's noise level, and the relative lens aperture are used to calculate optimal luminances for the brightest and darkest areas in the scene. The necessary reflectivity coefficients for the objects in the scene are measured and lighting is designed which produces the correct object luminances in both the specular highlights and the diffuse background areas of the scene. We show how to specify the light sources required to produce the lighting and present an example of printed circuit board inspection. The burden that this design method places on the vision algorithms is also discussed.

Landmark navigation appears promising, for the guidance of mobile robots. With this approach, a landmark target is recognized and used to update the position of the mobile robot. The purpose of this paper is to describe a color target recognition method with an omnidirectional vision system. The method using a color target arranged in a given spatial pattern is described. A recognition logic using thresholding of two color components was developed. This recognition technique is demonstrated by experiments which show the advantage of using color features. Color features add new dimensions and useful measures for target recognition.

The use of structured light techniques for obtaining three dimensional measurements of printed circuit board assemblies is examined, and a particular application for calculating the volume of solder paste that has been applied to each pad is described. The optical configuration for projecting parallel lines of light is discussed, along with machine vision algorithms for obtaining the volume measurement. By interpreting the translation of the lines with respect to a known reference point, a series of data points can be collected and used to estimate the total volume of the sample. Further investigation is made into the accuracy of the measurement, and techniques for applying LCD devices within the projection apparatus. It is concluded that these capabilities will allow for the accurate measure of solder paste volume within the manufacturing environment.

X-ray is used in many different ways in a broad variety of industries within our world today. It is used on almost everything from human beings to very large steel structures.

The advances in machine vision technology have been substantial in recent years, with the introduction of faster, more powerful processors and the improvements in sensor technology (i.e., solid state cameras). One area that is often neglected in machine vision applications is lighting considerations. Often, complicated and expensive hardware and software are used to solve a vision problem where some attention to the "front end" (lighting and optics) would have solved the problem.

A model of illumination permits classifying and analyzing illumination methods. Light sources are modeled as surfaces in space relative to an imaged object, which define source apertures. Sources may be repositioned for scanning and to overcome placement constraints. This aperture model describes collimated and diffuse illumination, brightfield and darkfield methods. With the addition of a field surface it models structured lighting. The models provide a basis for comparing methods. Literature on illumination for machine vision is reviewed, with emphasis on automatic visual inspection.

Accurate calculations of edge locations and shapes are often important in industrial machine vision applications. The accuracy of the calculations is affected by the lighting, the camera and other vision system hardware, and the choice of vision algorithms. In this paper we examine the first step in the vision analysis: the formation of an edge's image on the camera's sensor. In particular, we describe how the lighting and the camera's lens affect the location, shape, contrast, and intensity of the "edge" that the camera sees. An edge is assumed to be an abrupt, step change in an object's reflectivity, opacity, color, or other visually measurable property. The image of the edge on the camera's sensor can be modelled as a perfect image that has been modified by the lighting and the camera optics. We examine the lighting and lens effects that can cause the image on the camera's sensor to differ from an ideal edge. Those that produce measurable differences in typical industrial vision applications are described and quantified. The result is an analytical model for the resulting image on the camera's sensor. We use the predictions from this model to estimate the error contribution in experimental images.

A method of high-speed processing of images with up to 1024x1024 gray scale pixels is presented. The first step is a form of pyramid processing. The second step is a form of region of interest processing. This method is inexpensive and allows the processing of digital images much faster than in conventional ways.

Accurately locating the three dimensional position of a hole is critical for many manufacturing applications of machine vision. A novel method for locating a hole combines the use of backlighting with structured light. The centroid of the image of the backlit hole defines a line in space along which the center of the actual hole must lie. The cross of structured light is then used to determine the plane( surface ) on which the hole is located. Hence, the intersection of the line and plane defines the three dimensional position of the hole in space. The strength of the approach is the large amount of data used to obtain both the line position and the plane location. Thus, the approach is robust and has good immunity to noise, unlike many pure structured light techniques. The approach is also unaffected by changes in the orientation of the hole. It avoids the difficult problem encountered in stereo vision of having to create good images from two different viewing angles.

New optical systems containing several laterally displaced lenses or lens segments operating in parallel are discussed. Such systems are useful for high resolution imaging of small, widely separated regions of an object or for producing custom illumination patterns for applications such as triangulation based range sensing using multiple stripe illumination. Inspection problems well suited to multiple aperture imaging are described and design rules are given. Examples of both imaging and illumination applications are given.

There exists an increasing need in industry, particularly for machined parts, for higher accuracy profile measurements. Structured lighting techniques have proven valuable for such profile measurements, but have typically been limited in their accuracy by focus and sensor limitations. This paper describes a different approach to a structured light type of system which instead takes full advantage of the high accuracies and range available with precision measurement positioning stages. The resulting system, the optical guillotine, provides an optical equivalent of the commonly used mechanical guillotine and feeler gages used for making measurements, except with much greater speed and measurement capability. The optical guillotine effect is created by translating a collimated sheet of light relative to the object's surface along the line of sight of the viewing system such that the line will cut across a constant cross-section of the object at the points on the object which are at the depth as indicated by the stage. This system takes advantage of a compensated focus of both the illumination beam and the viewing system so that the system is not limited by the depth-of-focus of the viewing lens. For high slopes surfaces, very little data processing is required and the signal to noise ratio is inherently very high. These properties make the system capable of very fast measurements, covering a depth of over 15 centimeters in one second with a measurement accuracy of a few microns. The theoretical and design considerations of this approach, including the sources of errors for this type of system in general, are discussed in this paper. The results of a breadboard system, based on this design analysis, is also presented.

This paper describes a range imaging system being developed based on a high-speed space encoding projector. Space encoding allows the identity of 2 to the B projected light stripes to be unambiguously determined by the projection of B + 2 coding patterns. Range to each projected stripe element can then be determined by triangulation and a range image of the illuminated scene generated. The projector makes it possible for the coding patterns to be projected at camera frame rates. The complete system, consisting of the projector, a 60 Hz frame rate charge-coupled device (CCD) camera, and processing electronics, will be able to acquire an 8 bit, 256 by 256 pixel range image in 1/6 second.

A novel combination of image formation theory, along with optimal digital estimation theory was employed in development of a new optical probe suitable for a variety of high speed sensing applications, such as in-process manufactured part inspection. Experimental results demonstrate a robust, 40,000:1 range-to-resolution ratio. The sensing strategy also utilizes multiple, simultaneous illumination beams such that it is possible to accurately estimate surface orientation for robotic guidance applications. A significant element in this research was the development of a technique which is least influenced by surface finish roughness as commonly found on manufactured surfaces. Experimental results are provided to demonstrate the validity of the theoretical models.

The vision system described in this paper was developed to digitize the contours of smooth, featureless, curved surfaces, such as aircraft wing surfaces. The information will be used by a maintenance robot to carry out automated ultrasonic material testing on the surfaces. The vision system uses a light spot projector and video camera in a structured-light technique. This employs a 32x32 array of illuminated spots which is first projected onto the measurement surface and then followed by a set of five binary space-encoded patterns to identify the column addresses of the spots. The images of the spot patterns are captured by the video camera. Two surface reconstruction algorithms, nearest neighbour and thin plate model, are implemented to fit a surface to the scattered amplitude samples obtained by the spot projection system.

Heterodyne and "quasi"-heterodyne techniques applied to holographic interferometry provide both increased sensitivity and means for automated analysis of fringe information. Holograms may be recorded which contain optical path difference information related to surface contour, displacement, or deformation of an object of interest. Optical path difference resolution approaching 1/2000 of a wavelength (about 2.5 Angstroms) has been demonstrated using heterodyning techniques in which image data is electronically reduced to produce high resolution maps of contours or deformation of objects with specular or diffuse reflecting surfaces. This represents a 1000-fold improvement in sensitivity over that acheived using conventional holographic interferometry. The fact that dimensional data is taken directly from the image plane of the interferometer unambiguously and without the need for human interpretation, suggests the potential for the use of these or similar techniques for some machine vision applications.

Even though moire interferometry techniques have been around for some time now, they have not been implemented in very many machine vision applications. To date, the main draw back of moire interferometry has been the difficulty associated with the processing of the image. The processing time can range from several minutes to hours. This paper will concern itself with work to reduce the processing time to near frame rate speeds by using a vision flow type architecture vision system. Phase shifted moire is a full-field technique of mapping three dimensional information into two dimensional space. Unlike conventional moire interferometry, there is no ambiguity as to the sign of the z (depth) coordinate and there are no fringe centers to be located or fringe mapping to be done. Phase shifting naturally filters out noise in the image, and can even provide contrast information that can help determine if the data at a given point is good. Being a full-field technique, phase shifted moire interferometry produces a data rich set of three dimensional coordinates for all points on an object surface. By its nature, full-field data analysis is necessarily a computational intensive task. Traditionally, several images are acquired and stored away to later be processed in a serial fashion. By capturing and processing the images with a massively parallel vision engine, the total time required to process the data can be reduced significantly.

A new triangulation method for range image acquisition is presented. The scene is illu-minated with a fixed binary coded light pattern. The 64x63 range picture is determined from the information of a single camera image, which makes it possible to acquire moving scenes. The ambiguity inherent in multiple sample triangulation is solved by making each sample point identifiable by means of a binary signature, which is locally shared among its closest neighbours. The applied code is derived from pseudo-noise sequences. Analysis of an illumination-coded image yields the image coordinates of the detected sample points with sub-pixel accuracy, along with their assigned code bit. Exploration of the local topology around each sample retrieves its associated binary signature, by which it is identified. The corresponding spatial coordinates of an identified sample are determined by applying a linear transformation to its image coordinates, the appropriate matrix being selected by the identification result.

A structured light technique is utilized to perform precise, non-contact, continuous dimensional measurement of products on a moving conveyor belt. A plane of near infrared laser light is projected onto the workpiece and a triangulation method is used to reconstruct the three-dimensional properties of the object. Applications of this technique include determination of thickness, depth, length and width, as well as product contour and shape analysis. The system is robust and suitable for on-line industrial inspection of all types of products. Gray scale image analysis is used with data filtering techniques to achieve high resolution and accuracy. A gradient edge detection algorithm minimizes processing time and enhances system throughput. If desired, a stored reference position can be utilized when the conveyor belt or work surface is not available for direct computation of product depth. The advantages of this system over previous systems lie in the flexibility of the system for on-line industrial inspection and in the accuracy and reliability produced by multiple line scan, multiple frame, gray scale analysis.

We propose a new technique for robotic positioning and calibration which utilizes holographic imaging and interferometry to avoid the disadvantages associated with conventional techniques. This technique allows both gross and fine positioning of robotic devices in space and is useful in situations where it is necessary to repeatably position a moving device to a specific location. Gross positioning is accomplished by matching the robotic device to a holographic virtual image, using a computer vision system to overlap key features of the robotic device on their holographically imaged counterparts. The use of holographic images is an advantage because the robot cannot obscure them from view or collide with them. Fine positioning is achieved to better than one-micron accuracy by utilizing the interference fringes that result when the robotic device and the holographic image are aligned to within about 50 microns of each other. Elimination of the fringes indicates exact coincidence between the robot and its holographic counterpart. A computer vision system was utilized to automate the entire positioning procedure. Theory, algorithms, and experimental implementation are described.

One of the most important challenges in machine vision is 3D data acquisition and processing. It has been said that most problems are inherently three dimensional and that 2D problems rarely exist. Many applications, for instance inspection of component boards, require three dimensional information to avoid the special case engineering required to handle varying colors, shapes, and backgrounds of low contrast. Acquisition speed is a major limitation of 3D sensing devices currently available. This paper surveys a number of techniques which can be used in future high speed scanning and detector systems to overcome these limitations. Performance of each system type will be compared and examples of high resolution 3D maps will be shown.

One method for measuring the shape of a three-dimensional (3-D) object is to project light rays onto its surface, image the resulting scene with a camera, and triangulate the illuminated object points. The locations of the projected rays, and their correspondences with the camera image of their incidences with the object surface, must be known for the triangulation procedure. This paper describes how the pattern illumination technique can be refined for range mapping without the usual correspondence information, and in fact without knowledge of the projected ray locations themselves. This is done by displacing the camera and/or projector between image captures.

A geometry based on the use of a pattern projected on the scene and a mask in the aperture of an objective lens is used for the acquisition of objects shape. A standard CCD camera produces an image containing all the necessary information to measure the three spatial coordinates. A review of the sensor geometry is given. The algorithms used to extract the information and to remove the erroneous measurements are shown together with a practical hardware implementation.

The specification and design of structured light triangulation systems are discussed. Custom optical systems for optimal illumination and signal detection are described. In addition to general concepts that are applicable to all triangulation sensors, several special cases are examined. In particular, the effects of surface roughness on measurement accuracy and the problems of long range triangulation are discussed.

Vision capabilities have a significant effect on many applications of robots. The limitation of current imaging systems with regard to a limited field of view affects the performance of the robotic system. A whole world view sometimes is required and is often desirable. The general approach relies on mechanical scanning to record an entire world view; however, omnidirectional viewing by a fish eye lens can provide a full hemispherical field of view at the same instant. The development of omnidirectional vision appears to have definite advantages. The paper reviews the development of omnidirectional viewing and present the basic configuration of a fish eye lens. The projection features of omnidirectional lenses are discussed. A design of an omnidirectional robot vision system is described. The striking topics for its applications in the field of robotics, an omni-image restoration technique and an omnivision navigation technique, are briefly outlined.

Recent advances, including binary phase-only filters (BPOF) implemented with suitable real-time devices, have enhanced the potential of coherent optical correlation for pattern and object recognition and object location functions which are important in machine vision applications. Hybrid (optical/electronic) systems performing complex vision operations such as recognition, location, and discrimination at attractive rates are enabled by these developments. Computer simulations and matching experimental runs investigating the performance of BPOF correlation on video images of actual industrial objects have been performed and are reported. The results are not exhaustive and they do not provide the solution to a particular machine vision problem, but they do indicate the high potential of hybrid optical/electronic systems for this type application.

Even with incoherent (ambient light) imaging some optical processing can be performed to filter an image for machine vision applications. Apertures with useful transfer functions which are still light efficient can be formed from pseudonoise sequences. These pseudonoise apertures can be implemented on spatial light modulators to make adaptive imaging systems.

This paper deals with filtering techniques, both optical and electrical for suppressing background light interference in a wireless infrared (IR) diffuse-light channel, operating in closed working areas. Edge and bandpass optical filters are discussed, and a variety of electrical filtering circuits is presented covering low, medium, and high bit-rates that may be required by an IR system, depending on its application.