Visual display with multi-finger tactile interaction
The development and widespread implementation of touchscreens in recent years has enabled intuitive human-display interaction to be achieved with a variety of electronic interfaces (e.g., mobile phones and tablets). More recently, this technology has been enhanced to enable multi-touch interaction. Interaction has also been made possible via means of a tactile sensory channel (i.e., vibrations and haptic feedback). This advancement succeeds in meeting user demand for rich feedback from mobile devices.
Among the tactile display types achievable today, three can be integrated with a visual display. The first type is based on a touch surface that vibrates due to feedback from mechanical actuators.1, 2 Conversely, in an electrostatic tactile display, the dynamic frictional force between the user's finger and the touch surface is varied by means of an electrostatic force.3, 4 Finally, an electrotactile display directly activates the sensory nerves of the user's finger via an electrical current.5 An additional important feature for next-generation touch devices is multi-touch tactile interaction. Of the approaches described, the first two (i.e., those based on mechanical actuators or electrostatic forces) present the same sensation over the entire surface: see Figure 1(a). Thus, all fingers that come into contact with the surface are subject to the same sensation. In the third type (i.e., in which sensory nerves are electrically stimulated), each electrode is connected to a wire. Because of this, the technology cannot be scaled by increasing the number of electrodes, and the number of tactile pixels therefore tends to be low.
We have developed a novel electrostatic tactile display that presents regional stimulation, thus accommodating multi-touch or multi-person tactile interaction: see Figure 1. Our tactile display panel consists of a glass substrate, multiple electrodes made of indium tin oxide (ITO) in a matrix arrangement, and an acrylic insulator layer that covers the electrodes: see Figure 2(a). The panel presents localized stimulus in the region where excited horizontal (i.e., X) electrodes cross excited vertical (i.e., Y) electrodes, causing users to experience the tactile texture of fine roughness. This approach enables the selection of electrodes to be excited, and our display panel can thus present the stimulus at arbitrary positions on the screen.
A cross-sectional view—see Figure 2(b)—illustrates how our interface elicits tactile sensations in the finger of the user. When a finger (i.e., electrode P) comes into contact with the surface, multiple X and Y electrodes create parallel plate capacitors with capacitance C. The voltage of electrode P (Vp) can be estimated as Vp= (V1+V2)/2 (assuming R is a large resistance), where V1 and V2 are the AC voltages applied to the Y and X electrodes, respectively. This voltage causes the induction of an electrostatic force (Ftotal) between electrode P and the opposing four electrodes. When the user slides his/her finger, the dynamic frictional force (Fr) is varied according to the electrostatic force.
Figure 2(c) shows calculated waveforms of Vp and Ftotal, where V1 and V2 are sinusoidal waves with a frequency of 1000 and 1240Hz, respectively.6, 7 The envelope of Ftotal has a frequency of 240Hz (i.e., the beat frequency that arises as a result of the combination of the frequencies of V1 and V2, f1and f2). This attractive force is not enough to be felt alone, but a dynamic friction fluctuation is induced when a finger slides across the touch surface. The user detects the horizontal deformation of the skin that is produced by this fluctuation (which is, in turn, transformed from the electrostatic force variation) as texture. The sensitivity of our tactile sensory system therefore depends on the signal frequency, and its design is crucial for the realization of regional stimulation.
Our measurements show that the detection threshold voltage (i.e., the minimum voltage amplitude that creates a barely detectable sensation for the user) is lower than 20V at a frequency of 200Hz, and higher than 40V at 1000Hz.6,7 Based on these findings, we selected a frequency of 1000Hz for f1 (i.e., the Y-electrode excitation) and 1240Hz for f2 (i.e., the X-electrode excitation). Hence, users perceive localized tactile sensation in a region where excited X electrodes cross excited Y electrodes.
We fabricated a 26.4cm tactile display panel—see Figure 3(a)—for our visual-tactile integrated display: see Figure 3(b). We then ran demonstration software on the system to test its efficacy. Figure 4(a) shows one such demonstration, in which a texture of fine roughness is felt by the user in a specific region of the touchscreen (i.e., the back of a crawling tortoise), whereas smoothness is felt in the area of the white background (even when both regions are interacted with simultaneously). Multiple fingers, or users, can simultaneously feel this localized tactile texture. Haptic feedback such as this increases the realism of visual environments and creates a feeling of direct interaction (i.e., ‘hyperpresence’). In another demonstration, a tactile texture is presented on each numerical key of an array of virtual elevator buttons: see Figure 4(b). This enables users to identify key locations not only visually but also via tactile response. Our results confirm that this response increases the efficiency of the interface by enabling the user to rely on familiar haptic cues to accomplish even the most basic of interactive tasks.
In summary, we have proposed an electrostatic tactile display that enables stimulus localization. The tactile sensation occurs as a result of the beat phenomenon, which arises from the combination of voltage waveforms between adjacent electrodes. Our work enhances user experience by combining multi-touch or multi-person tactile interaction with visual information, thereby opening up new possibilities for human-display interaction. To present a wide variety of tactile sensation, however, further research is required. In particular, we plan to investigate the relation between excited waveforms and tactile sensation by carrying out user studies, which are a crucial component in the development of interactive devices that induce a feeling of hyperpresence.
Hiroshi Haga received his BEng degree in image science and technology from Chiba University, Japan, in 1994. He subsequently joined the NEC Corporation, where he was engaged in the research and development of polysilicon thin-film-transistor circuits and devices. He joined NLT Technologies, Ltd. in 2011, where he has been developing human-display interaction technologies.
