SPIE Membership Get updates from SPIE Newsroom
  • Newsroom Home
  • Astronomy
  • Biomedical Optics & Medical Imaging
  • Defense & Security
  • Electronic Imaging & Signal Processing
  • Illumination & Displays
  • Lasers & Sources
  • Micro/Nano Lithography
  • Nanotechnology
  • Optical Design & Engineering
  • Optoelectronics & Communications
  • Remote Sensing
  • Sensing & Measurement
  • Solar & Alternative Energy
  • Sign up for Newsroom E-Alerts
  • Information for:

SPIE Photonics West 2019 | Register Today

SPIE Defense + Commercial Sensing 2019 | Call for Papers



Print PageEmail PageView PDF

Electronic Imaging & Signal Processing

Biologically inspired visual systems for autonomous robots

Prototypes of autonomous visual perception systems that attempt to mimic biological vision can be fabricated using small, low-power chips and a depth perception design based on motion parallax.
14 February 2007, SPIE Newsroom. DOI: 10.1117/2.1200701.0598

Perceiving the world around us is something most of us take for granted. However, capturing huge quantities of data and understanding it as we walk, run, drive, or pedal a bike is a task of incredible complexity. To perceive the world, we use our ‘built-in’ nervous system and sensory organs. Similarly, to perceive the world around them, autonomous systems (robots, planetary explorers, etc.) require sensory systems that are not overly burdensome in terms of weight, power consumption, and size.

Since vision is such an important sense for perception, an autonomous system must also have an efficient functioning visual system that is appropriate for its intended use. For example, there is a difference between mounting a visual system on an aircraft carrier and outfitting a Mars Rover with artificial vision.

The main goal of our research is to develop novel methods for improving visual processing for autonomous systems. We feel that the best approach is to use nature as our model. Indeed, animal visual systems perform extremely well under the constraints of limited power (food for energy) and space (eye and brain size). Thus, our group is creating biologically inspired silicon vision systems, combining analog integrated circuit (chip) design, optoelectronics, and neural networks.

Analog CMOS VLSI (complementary metal oxide silicon and very large scale integration) technology is a combination of analog signal processing and high-density transistor circuits that represents an active research area in biologically based vision and hearing processing systems.1–7 CMOS is the semiconductor technology used to create standard computer chips, and their fabrication is accordingly straightforward and inexpensive. VLSI is the process used to place tens of thousands of transistors on a chip. The perception functions performed by animals require the real-time processing of huge amounts of data, but not at very high levels of precision. Hence, analog VLSI is ideally suited to implement biologically based processors.

Our initial work was based on the silicon retina developed by Mead and Mahowald.8 Our system had a parallel optical output that eliminated the need to sequentially read out the data from each pixel.9 Subsequently, we developed a custom system that could determine depth perception.10 Our method was based on stereopsis, a visual process that results in depth perception owing to the different physical position of the two eyes.

In our work, we have investigated alternate methods for obtaining depth information from motion. We have designed an integrated circuit embedded in a larger system that computes the velocity of a light source using motion parallax, the apparent shift of an object against a background due to a change in observer position. The velocity information is obtained from an output current pulse whose duration in time is proportional to the differences in the speed of light sources across the detector array.11,12 Our work has also defined the conditions under which using motion parallax for depth perception is valid. Finally, we have used our system to validate these theories for the first time.

We have also approached the problem of visual perception by attempting to mimic the visual processing capabilities of specific animals. We selected the octopus visual system as our model by mimicking two of its interesting aspects, namely, its orientation and polarization sensitivities.13,14 Orientation sensitivity refers to its inability to distinguish between mirror oblique objects. We have demonstrated this aspect using our octopus-retina chip (see Figure 1) and in subsequent neural network studies.14 Polarization sensitivity in the octopus is mainly based on the orthogonal arrangement of its photoreceptors. Our implementation uses a birefringent micropolarizer made of YVO4 (yttrium vanadate) and mounted on a CMOS chip with neuromorphic circuitry to process linearly polarized light. Our prototype has an 8×5 array with two photodiodes per pixel.

Figure 1. A photograph of one version of our octopus retina chip. The array of processing circuitry is on the 2×2mm chip shown next to a one-cent coin.

We have demonstrated that visual perception systems can be fabricated using small, low-power chips. But to date, our work has only scratched the surface of the problem of visual perception. Future work will include combining different visual systems to create uniquely specialized perception systems, fusing data from our visual systems with other stimuli or a priori knowledge (see Figure 2), and the development of improved neural network hardware for back-end (or high-level) information processing.

Figure 2. A conceptual structure combining vision chips (labeled 1–5) with different primary functions on a single circuit board that can be used as the front-end visual system of a robot.

Albert Titus
University at Buffalo
Department of Electrical Engineering, USA

Albert H. Titus is an assistant professor in the Department of Electrical Engineering at the University at Buffalo, the State University of New York. He earned his PhD from the Georgia Institute of Technology in 1997. His research interests include analog VLSI implementations of artificial vision, and hardware and software for artificial neural networks. He has obtained numerous research grants from federal and private sources, including an NSF CAREER award.