The US National Airspace System (NAS) demonstrates almost daily that, while aircraft can take off and land in bad weather at near zero visibility and zero ceiling, the operational tempo— especially within the airport terminal area—deteriorates in instrument flight conditions. That leads to airport delays that cascade across the nation. Enabling the US Next Generation Air Transportation System (NextGen)1 requires developing and implementing technologies and operational concepts that are safe, efficient, and reliable. These innovations must remove many of the existing constraints and support a wider range of operations to meet the NextGen goal of three times the capacity of today's airspace system.
One NextGen operational concept—called equivalent visual operations (EVOs)—addresses the capacity bottleneck and safety concerns caused by instrument flight conditions. When the weather is good and pilots fly in visual flight conditions, airspace capacity, operational flexibility, and safety are at their best since the flight crew can see the terrain, other air traffic, and runways and taxiways. The crew can self-navigate and remain separate from the other traffic, without being directed by air traffic controllers. Aircraft can fly direct, point-to-point, fuel-efficient routes and noise-abatement trajectories. Once instrument flight conditions occur, however, these benefits largely vanish. Flights must now be tracked and directed by air traffic control to maintain safe traffic flow and separation. Technology and operating procedures severely limit operational flexibility and capacity.
The EVO concept goal is to recreate the safety and operational tempos of visual flight operations. One way to achieve this is through the use of new electronic cockpit displays and aircraft sensors that create an ‘equivalent vision’ flight deck environment for the flight crew. NASA and its industry partners have led the work on these technologies, particularly enhanced vision (EV) and synthetic vision (SV) for commercial and business aircraft applications. Both SV and EV are intended to create, supplement, or enhance the natural vision of the pilot.2 These concepts are currently in use with conventional head-up and head-down displays. The next challenge lies in coupling these ‘equivalent vision’ technologies with head-worn displays (HWDs) to create spatially integrated, ‘unlimited’ field-of-regard EVO displays for pilots (see Figure 1).3 NASA is focusing on foundational research, development, testing, and evaluation of these emerging technologies as well as the extent to which they may enable EVO, or even ‘better than visual’ operations.
Figure 1. Notional equivalent visual operations flight deck.
HWDs are not new technology, particularly for military operations. However, component miniaturization and maturation are now progressing to the point where HWDs can be considered in commercial and business aircraft operations. Costs are reaching affordable levels, and comfort and convenience for the pilot could approach that of wearing sunglasses.4 NASA's research began with the vast background of military application data that exists for helmet-mounted displays, but quickly diverged. Information demands and operating constraints between military and commercial applications are radically different. The military may accept pilot workload increases to complete a mission that was previously unachievable. In contrast, the commercial aviation ‘mission’ can be done today, just not as efficiently as needed for NextGen. Thus, commercial technology must provide cost or capability improvements for airlines and other users since they must literally ‘buy into’ the concepts.
NASA has been conducting laboratory tests and pilot-in-the-loop simulation experiments, primarily using commercial off-the-shelf HWD technology, to characterize the state of the art and to assess operational and human factor implications.5,6 For instance, what kind of visual flight condition information must be reproduced in an EVO to be effective? Can commercially available technology meet these requirements? If not, are the commercial market trends favorable? NASA is targeting innovative development to explore revolutionary, high pay-off investments to fill technology gaps. To date, exploratory work has focused on aircraft surface operations (see Figure 2). Here, the flight crew relies almost exclusively on looking outside the cockpit for navigation and traffic separation, since limited surface operations instrumentation exists. Consequently, full field of regard visual information will be critical for the pilot, who must look for traffic and maintain airport navigation and taxiway awareness. Overall, the results have shown that suitably designed HWD concepts have significant advantages over head-up displays for enhancing taxiway traffic conflict detection with commensurate pilot workload and performance.
Figure 2. HWD concept showing traffic model and taxi guidance insert display.
While these results are encouraging, HWD concepts are not yet ‘ready for prime time.’ Our first challenge has been to create a viable HWD form factor with stability, yet without encumbering the pilot. NASA has partnered with a commercial vendor and developed a proof-of-concept, monocular HWD with a total head-borne weight of less than 0.5lbs. While not perfect, this work shows that a miniaturized tracking system integrated with custom eyewear and display optics provides a viable approach. With this substantially reduced form factor, laboratory testing will assess whether the HWD imagery can meet existing commercial accuracy standards. More important, NASA human-in-the-loop testing this year will evaluate whether these commercial standards—developed for head-up displays—are applicable and appropriate for HWD applications. These tracking accuracy standards, and also those for image parallax, critically determine the conformality (i.e., how well the HWD “overlays” the real world) of the HWD imagery. This research will lay the groundwork for these technologies.
Another challenge must address the more fundamental question of optics design. Testing to date shows that a monocular design did not induce binocular rivalry and that it exhibited less degradation in visual acuity compared with binocular designs. However, these results were derived during fixed-based simulation. Flight validation of the findings is needed, and other confounds must be explored such as whether these results are influenced by the presence or absence of synthetic and enhanced vision HWD imagery. Another confound being investigated is a so-called semiconformal optics design in contrast to a binocular design. The semiconformal concept is simply the case where the HWD image is rotated slightly above the eye away from the normal line of sight. It can be viewed by looking up, and the conformal image by tilting the head down. The key advantage is that the pilots do not have an image continually obscuring their normal vision. This concept promises to mitigate certification risk, which might otherwise prohibit the use of a continual binocular HWD design for commercial and business aviation.
The challenges to meet NextGen's EVO capability are formidable, both in terms of technology and proving the cost-benefit for customer buy-in. NASA is conducting research, development, testing, and evaluation of advanced concepts that hold the potential for significant benefits in capability and safety. HWDs are showing promising results and offer significant weight savings over a head-up display installation. With sufficient technology maturation, HWDs may provide the performance, safety, and cost-benefit advantages for commercial and business aviation operators to meet the challenges of NextGen. Further steps include deriving an acceptable HWD form factor and required informational needs for the NextGen flight crew, and developing data that may define the HWD resolution, image accuracy (i.e., tracking and sensor sources), and optics design requirements for this mission.
This work is being funded as part of NASA's Integrated Intelligent Flight Deck (IIFD) project, one of four projects within NASA's Aviation Safety Program.7
Randall Bailey, Jarvis Arthur
NASA Langley Research Center