Restoring the perception of eye contact in the visually impaired

For years, research and development resources for the blind have centered on restoring sight. Conventional methods for restoring sight focus on engineered solutions that use acoustic landscaping or 2D stimulation of the brain cortex.¹ However, this technological challenge may take decades to produce a working and affordable solution. By reframing the problem of lost sight, we found that the essence of sight can be restored with optical processing technology that is available today. We are fascinated by how product design can change people's lives. For example, practical understanding of how people process information can help us to design objects for cognitive impairments.² In a similar way, understanding the role of light in our daily life can result in meaningful products for visual impairments.

We live in a world in which everything is revealed by light, and so visual impairment and blindness severely impact daily life. Not only does blindness complicate orientation and communication, but it also creates social integration problems in our sighted society. In other words, blindness cuts people off from a major social and physical environment.³ Globally, it is estimated that 38 million people are blind and more than 110 million people have low vision and are at high risk of blindness.⁴Cutting off or segregating these people from society often forces the visually impaired into a dependent role that can influence their behavior. The result is stereotypical attitudes and responses to the sighted person, who reacts accordingly.

‘Sett,’ Norwegian for ‘see(n),’ is a prototype that extends and supports the other senses to improve quality of life for the blind.⁵ Without the ability to observe facial expressions and gestures, it is difficult to recognize another's meaning and intentions during a conversation, or to judge the meaning of a silence. This obstacle makes blind people less confident in evaluating emotional behavior. The loss of certainty in recognizing the location of who is speaking, especially in a group, makes it even harder. In other words: the blind should be able to perceive the other as part of the environment in addition to observing the activity of the other perceiving him.⁶ This kind of perceptual feedback is essential during communication between two people. Using this insight, Sett captures the gaze of people in the line of ‘sight’ of a blind person, and converts the information into a tactile signal.

Sett consists of a pair of sunglasses, a haptic feedback bracelet and a unit for signal processing (see Figure 1). The sunglasses with dimensions 141×140×46mm—length (l) × breadth (b) × height (h)—consist of an integrated HD camera with a resolution of 1280×720 (720p, 16:9). The HD webcam is directly linked to the computer through a USB connection and the captured images are directly loaded into the processing program (which reduces the images to 510×340). The software program for face-recognition uses OpenCV libraries and is written in Java,⁷ and the vibration motor is controlled by an Arduino UNO microcontroller board.⁸The bracelet—with dimensions 35×25×8.5mm (l×h×b)—consists of one small vibration motor with a start current of 85mA. Figure 2 shows the corresponding elements.

Figure 1. The complete Sett system consists of sunglasses and a haptic bracelet.

Figure 2. Three details of the Sett concept. (Left) Signal processing to detect the position of the eyes. (Middle) Integration of a camera in sunglasses to capture the image and to detect line of sight. (Right) Haptic feedback bracelet.

Once we had built a prototype, we investigated the extent to which it arouses emotions and provides feedback. Did Sett stimulate and enrich social interaction through touch as intended? We conducted a study that examined four participants' reactions to the prototype as well as an interviewer and participant's feeling of involvement during an interaction. The tests consisted of conversations between a totally blind person and a seeing interviewer. Situations were created with and without Sett, and with and without a desktop activity moving a simple slider. In this way, we could study the perception of someone looking at your hands while conversing.

The study's findings showed that the simplistic feedback generated by Sett was able to trigger some positive emotions as well as slight changes in the behavior of some of the participants. These small positive experiences are an important contribution for the stimulation of social interactions as well as for integration of the visually impaired in society.

Sett technology is based on building blocks that are available right now. We constructed a research prototype quite easily, and have already created and tested three iterations of functionality. To make it available to the market, the next steps are to reduce the number of false positives to an acceptable level, and engineer a fully integrated system for consumer use.