Head-worn displays on commercial flight decks

While head-up displays (HUDs), which project images onto a small glass screen located in front of the pilot(s), are becoming commonplace on the flight decks of commercial aircraft, there is increasing industry interest in the use of head-worn displays (HWDs). HWDs are a type of wearable display that projects images onto one or both lenses of a pair of glasses (see Figure 1). When coupled with a head-tracking system, they can offer two important benefits over HUDs:¹ first, a larger field of regard (total possible visual-display area) and second, the capability to present conformal imagery that is not obstructed by physical hardware as in, for example, the ‘see-through’ flight deck. Essentially, HWDs offer significantly more visual-display space and enable new ways of information provision. This is particularly appealing since future developments in aviation are predicted to increase crew information needs.²

Figure 1. A head-worn display integrated into a flight simulator.

Although HWD use is promising, there are four barriers to implementation on commercial flight decks. The first is technological. Current HWDs are heavy, awkward to wear for extended periods, have noticeable latencies in updating displayed information,³ and vary greatly in terms of transmissivity, resolution, and optical quality. To some degree, this may be the easiest challenge to meet. Advances in technology have increased processing speed and power compared to systems available just a few years ago, while new optical systems have been created that improve image quality. In addition, miniaturized components reduce hardware size and weight. However, more research is required to determine if HWD technology has improved sufficiently to mitigate all of the remaining issues and, if not, how much further there is to go.

The second barrier is physiological. To some degree, HWDs have been associated with ill effects such as nausea and eyestrain.^4–6 In their review of thirty years of operational use of a monocular helmet-mounted display, Hiatt and collaborators report that pilots have experienced physical and visual symptoms such as headaches, blurred or distorted vision, and visual illusions that occur while using the display and in some cases persist for some time after use.⁵ HWDs have also been found to affect attentional switching between the information displayed on the lens and that available only in the outside world.⁶ A proposed justification for using monocular instead of biocular or binocular units has been that only one eye is presented with augmented information while the other continues to view the external environment unaided and thus unimpeded. The intention is that focus may be more easily switched between the near-to-eye display and the real world, reducing the effects of immersion. However, in practice research indicates that binocular rivalry and eye dominance create difficulties for the observer in focusing on one or the other image at will.⁶

The third barrier is perceptual. HWDs allow for a wide variety of display parameters, including focal distance, optical quality, degree of stereopsis, and fields of regard, all of which affect one's ability to perceive and extract information effectively.⁷ Complex factors such as display overlap and ambient-lighting conditions can affect how the image is perceived. Any constraints as to the layout or format of information in a HWD must be developed with respect to expanded fields of regard that include areas of the flight deck where visual information is already displayed (i.e., the standard head-down displays, instruments, etc.). It is important to ensure that no critical information is obscured by overlapping displays. Focal-length and visual-accommodation issues with presenting information at different focal distances must also be managed. Augmented information projected onto the outside world is typically set to optical infinity while any augmented information displayed over flight-deck displays would necessarily need to be set to a much closer focal length to avoid problems of accommodation.

The fourth barrier is operational. Commercial pilots would have to accept the new display format for it to be usable. The issue concerns individual acceptance of the new display as well as the effects that it may have on crew resource management.

However, despite the technological drawbacks of currently available devices, at least one study suggests that flight performance using a HWD is equivalent to that with a HUD.⁸ In addition, pilots in a separate survey also responded positively to the concept of near-to-eye displays for flight-deck use.⁹ This suggests that the fourth barrier, adoption of the new display, would not be a significant hurdle as long as the display is comfortable, well-designed, and offers clear performance benefits. Similarly, concerns that explicitly stem from limitations of the optics (e.g., display resolution) or processing speed and power (e.g., latencies and boresighting accuracy) are likely to be reduced or overcome by technological advances over time.

Thus, although a number of barriers must still be overcome before HWDs can be incorporated into commercial flight decks, the data and trends to date suggest that the concept is viable and that these challenges can be met. Continued research targeting relevant environmental conditions will help isolate those factors that contribute to the negative effects experienced with HWD use. It can also be used to determine design guidelines and recommendations that will reduce or eliminate the risk of detrimental physiological or perceptual effects. These, in turn, will help inform the development of accepted standards and requirements, which will aid in the certification process.