Protein-based optical filters for image processing

A receptive field (i.e., a limited field of view) is an important functional unit for processing visual information. If the shape of a receptive field is understood, it can be used to predict the time response of a cell to arbitrary spatiotemporal stimulation patterns. Although silicon-based artificial receptive field devices have previously been proposed, these have been only partially successful and they do not satisfy requirements for simplicity and resolution.¹ New materials for such devices therefore need to be developed.

In humans, visual information is first processed within retinal ganglion cells (RGCs). The information is then transmitted to the lateral geniculate nucleus (LGN) of the brain, and finally to the primary visual cortex in the cerebrum. Indeed, certain sensory neurons have receptive fields. For instance, neurons in RGCs and the LGN have coaxial-shaped, center-surrounding receptive field structures that are composed of excitatory and inhibitory regions. An on-center receptive field, in which the center is sensitive to bright stimuli and the surrounding region is sensitive to dark stimuli, is shown in Figure 1(a). It is possible to model the shapes of such receptive fields with the Difference of Gaussians (DOG). Cells in the visual cortex—unlike those at earlier stages in the visual path—respond much better to a moving line than to a stationary line. These ‘simple cells’—see Figure 1(b)—have spatially oriented receptive fields with elongated alternating excitatory and inhibitory domains.² In addition, they can be described accurately by a Gabor function,³ which is simply a Gaussian-modulated sinusoid. The DOG and Gabor filters are regarded as an important tool for a variety of image processing activities (e.g., edge detection, image enhancement, and pattern recognition).

Figure 1. Schematic illustrations of on-center receptive fields of (a) retinal ganglion cells and the lateral geniculate nucleus, and of (b) simple cells. The centers of these cells are responsive to bright stimuli (green), and the surrounding regions are responsive to dark stimuli (orange). The shapes of these receptive fields can be accurately described by Difference of Gaussian (DOG) and Gabor functions, respectively.

In some previous experiments,^4–7 it has been reported that protein-based artificial receptive fields can yield edge-detection capabilities. In our work, we have thus developed a new artificial receptive field with the use of a photoinduced protein that can simulate both excitatory and inhibitory responses.⁸ The unique protein that we use—bacteriorhodopsin (bR)—is found in the cell membrane of Halobacterium salinarum and is similar to the visual pigment rhodopsin, which is found within the rod cells of human eyes. When the bR is illuminated, it transports protons out of the cell membrane. The resulting pH change generates a transient photocurrent that lasts only for the instant of the light switching (i.e., on or off), and which exhibits a polarity reversal (i.e., positive or negative). In this work we propose the use of a ‘patterned’ 2D-Gabor filter (i.e., a description of a simple cell's receptive field) in addition to a 1D DOG filter. This builds on our earlier study in which we produced a simple motion sensor by patterning bR films on electrodes.⁹ In our work, we make use of mask patterns, which are easy to print—with the use of a graphics program and a laser printer—in desired patterns on transparency films. With these mask patterns we are able to dramatically improve the image processing capabilities of receptive fields.

The experimental configuration of our bR receptive field—see Figure 2(a)—is based on a model of a binarized 1D DOG function. Our receptive field consists of a bR dip-coated thin film, as well as an electrolyte that is sandwiched between glass plates with indium tin oxide (ITO) electrodes. We coat two rectangular patterns—with different size bR films—on the front and rear ITO plates, which correspond to the excitatory and inhibitory regions, respectively. In this setup, we also fill the electrolyte solution (potassium chloride, pH 8.3) with a spacer. In addition, we use a liquid-crystal display projector, with a metal-halide lamp, to display images onto the sensing area. Various input images are produced (and the scanning speed is controlled) using the presentation software Keynote. The responses of the edge and of the thin light bar (with a width of 1.3mm) when they are scanned across the sensing area of the 1D DOG filter are shown in Figure 2(b). For the edge case, we observe a zero-crossing profile when the position of the edge coincides with the center of the filter. Both of these results are in excellent agreement with the theoretically predicted responses of X-cells to various stimuli (i.e., a moving edge and a thin bar).¹⁰ These predicted responses are images convolved with a Laplacian filter, i.e., ∇²G (where ∇² is a Laplacian operator and G is a 2D Gaussian distribution).

Figure 2. (a) Configuration of the binarized 1D DOG filter in which thin films of bacteriorhodopsin (bR) are used. The cross-sectional view is shown at the top, and the front view of the filter is shown at the bottom. Light blue bars: Glass plates with indium tin oxide electrodes. (b) Photoresponses when the edge (red line) and thin (width of 1.3mm) light bar (blue line) are scanned across the 1D DOG filter sensing area.

A 2D Gabor can be expressed mathematically in terms of the wavelength of the cosine function (λ), the variance of the Gaussian function (σ), and the spatial aspect ratio (γ). To make a binarized Gabor pattern, we chose values of 9.0mm, 4.5, and 1.0 for λ, σ, and γ, respectively. In addition, we set the intensity values to zero (i.e., less than a threshold value of 0.4). The structure of an on-center Gabor filter that mimics a simple cell receptive field is shown in Figure 3(a). In this structure, the surface of a bR 1D DOG filter is covered with a vertically directed Gabor pattern. In our experiments, we verified that our bR Gabor filter works well for line detection. We also scanned a thin light bar on the tilted filter to determine the selective orientation characteristics. We measured a maximum response at 0° and almost no response beyond 45°. These results therefore indicate that our bR Gabor filter only responds to spatial intensity changes along certain orientations.

Figure 3. (a) Structure of the on-center Gabor filter. The surface of the bR 1D DOG filter is covered with a binarized Gabor pattern (shown in black). (b) Spatial frequency sensitivity of the bR Gabor filter, measured with the use of a stripe that has six light bars. These input stripes are shown in (c). In the top version, all the lines have widths of 4.9mm. In the bottom set, the fourth and fifth lines have widths of 2.9mm (the rest have widths of 4.9mm). (d) Photocurrent response profiles of the bR Gabor filter to the input stripes. The red line shows the response to the stripe with six light bars (all with widths of 4.9mm), and the blue line shows the response to the stripes in which the fourth and fifth lines have widths of 2.9mm (the rest have widths of 4.9mm).

Furthermore, we used a stripe with six bars to analyze the spatial frequency sensitivity for different bar widths (1–20mm). We are able to predict the preferred spatial frequency from the Fourier transform of a Gabor function. We measured peak responses at about 0.11mm⁻¹, which corresponds to the reciprocal of λ equal to 9.0mm: see Figure 3(b). Our bR Gabor filter responds well to the predicted spatial frequency and therefore acts as a bandpass filter. If there is partial modulation in the spatial frequency of the input stripe, it leads to a significant decrease in the photocurrent intensity: see Figure 3(d). With this filter, we are therefore able to detect small differences of pitch (e.g., for a printed circuit) just by scanning projected images.

We have designed new receptive-field-type filters with the use of the photoreceptor protein bR, and we have experimentally shown that real-time image filtering is possible with the use of only one sensing element. We have also demonstrated that our DOG filter—which mimics on-center RGC receptive fields—has the function of a Laplacian filter and can act as an edge detector. In addition, our Gabor filter exhibits orientation and spatial-frequency selectivity, which is similar to that of simple cell receptive fields in the human primary visual cortex. Image processing with bR receptive fields is therefore similar to perception in the human visual system. The use of bR instead of silicon in image processing techniques can offer many advantages (e.g., visual functions at the material level, environmental friendliness, simple preparation processes, no need for additional external connections, and lower costs). We are now planning to explore the possibility of using our bR filters within digital image processing systems so that their operating times and power consumption can be reduced.

This work was supported by the Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research (KAKENHI) program (grant 15K00226).