Robot vision using biological pigments

Autonomous mobile robots respond to various situations and perform tasks without external commands. To detect objects, these robots are equipped with navigation systems composed of mechanical tools such as cameras, gyroscopes, ultrasonic sensors, and radar. All these systems, however, have substantial power and space requirements. To address these problems, we turn to the insect world. Insects are capable of processing visual information and rapidly responding to environmental changes. Robotic sensors derived from insects may provide lower-cost solutions by saving batteries and mount space. We are therefore aiming to develop an insect-mimetic visual sensor using a light-sensitive protein found in a bacterial cell and applying it to robot vision.

Our ‘biophotosensor’ is based on the purplish bacteriorhodopsin (bR) pigment found in the cell membrane of Halobacterium salinarum,¹ a bacterium that thrives in extremely high salt concentrations. The bR is similar to the light-sensitive pigment rhodopsin in the rod cells of human eyes. When excited by light, the bR molecule transports protons out of the bacterial cell. The resulting pH change generates a transient photocurrent only when the light switches on or off. This unique photocurrent property is similar to that seen in retinal neurons and quite different from that of conventional semiconductors. Therefore, there have been a variety of efforts to incorporate the bR into novel photovoltaic devices. Many of these experiments report that bR-based artificial retinas can yield edge enhancement and motion detection capabilities.^2–7

We incorporated an implementation of these revolutionary sensors into a robot developed for the 31st All Japan Micromouse Contest. In this contest, robots compete by negotiating a maze. The robot must be self-contained: it should be fully autonomous, and may not receive any external control or data during the competition. Our micromouse robot, shown in Figure 1, received a Special Award for its innovative use of biophotosensors. A video showing the robot navigating a practice maze is available online.⁸

Figure 1. A ‘micromouse’ robot with two biophotosensors (dotted circle) based on the bacteriorhodopsin (bR) pigment found in the microorganism Halobacterium salinarum.

The photocell consists of a bR spin/dip-coated thin film and electrolyte sandwiched between glass plates with indium tin oxide (ITO) electrodes. The electrolyte solution (KCl, pH 8.6) is filled in a spacer. The photocurrent response is converted to a voltage and amplified. One sensing element consists of a photocell and its associated circuitry.

We illuminate the sensing element with a beam of light and measure the photosensor's time and frequency response. The light is chopped by a compact optical shutter containing a polarization beam splitter and liquid crystal. A function generator controls the frequency and time period of illumination. Our tests confirm that the performance of the photosensor is comparable to that of the human eye.

Figure 2. (a) Structure of a bR-based photocell with a circular sensing pattern. (b) Photocurrent responses produced by light spots (2×10mm) scanned across the sensing area. Scan speeds were 8 and 80mm/sec, respectively. The signal profiles depend on the speed and cannot distinguish scan directions.

We fabricated a simple direction sensor by patterning the bR films on electrodes. When light scans the sensing area, the edges of each pattern produce a positive or negative current depending on the light's speed and direction. Figure 2(a) shows a circularly shaped bR-based photocell with an inside diameter of 9.8mm. Figure 2(b) presents the photocurrent responses to a 2×10mm light spot scanned across the sensor at 8 and 80mm/sec, respectively. In the low-speed case, there is no output signal. At the higher speed, there is a sharp spike in the photocurrent. The circular shape of the sensor prevents it from detecting the direction of the incident light beam.

Figure 3. (a) Structure of a bR-based photocell with a triangular sensing pattern. (b) Photocurrent responses produced by the moving light source. The light is scanned, at 8mm/sec, from left to right and then reversed. The asymmetrical shape of the sensing area allows direction of the scan to be determined

To measure both speed and direction, we adopted a triangular-shaped sensor: see Figure 3(a). Figure 3(b) shows the photocurrent response to a change in direction of a light spot scanned at 8mm/sec. This triangular design was the same proposed by Miyasaka,⁹ but his sensor was composed of eight sensing elements.

Our design of a photosensor consisting of only one sensing element is illustrated in Figure 4(a). Two rectangular patterns with different sizes are coated on the front and rear ITO plates, respectively. Two edges—the right side of the pattern on the front plate and the left side on the rear plate—are carefully aligned. Each pattern produces polarity-reversed photocurrents. The intensity of each signal is proportional to the edge length. Due to the asymmetry of the sensing pattern, we can distinguish four directions of the moving light—up, down, left, and right—as shown in Figure 4(b). We also obtained different profiles when scanning the light diagonally across the sensor in all directions.

Figure 4. (a) Structure of an L-shaped bR-based photocell that allows for direction and speed sensing. (b) The photocurrent response when a light spot is scanned across the detector in four different directions.

In summary, we designed photosensors based on the bR pigment found in the cell membrane of Halobacterium salinarum and showed that motion detection is possible using only one sensing element. These sensors have been successfully demonstrated on an autonomous maze-navigating robot. Biophotosensors also offer many advantages: pre-processing functions at the material level, environmental friendliness in terms of material production and disposal, and lower costs. They therefore can provide efficient devices for robot vision applications. The bR-based sensors also respond strongly to rapidly moving objects, though not at all to slow ones. We intend to exploit this unique property in risk-avoidance systems for wheelchairs and nursing care robots.

This work was supported by a Japan Society for the Promotion of Science Grant-in-Aid for Scientific Research (23560004).