Sunglass-like displays become a reality with free-form optical technology

An augmented reality (AR) display allowing the overlay of computer-generated imagery on a person's real-world view has long been considered a transformative technology to redefining the way we perceive and interact with digital information. A desired AR display form is a lightweight optical see-through (OST), head-mounted display (HMD). It enables optical superposition of digital information onto the direct physical-world view while maintaining see-through vision. With the increased bandwidth of wireless networks and electronics' miniaturization, one goal is to realize a sunglass-style and size, unobtrusive AR display that integrates the functions of OST-HMDs, mobile devices, and miniature global positioning system technologies. Such a display will make access to wireless networks, video displays, 3D content, and location-specific information easier than ever. It also has the potential to revolutionize many fields. One example is in medicine, where AR technology may let physicians see 3D anatomical structures or computed tomography images superimposed onto the patient's abdomen during surgery.

Still, a true sunglass-like, unobtrusive, high-performance OST-HMD has yet to become a reality. The stereotypical image of HMDs is that they have a cumbersome, helmet-like form factor. One major barrier to a sleeker form is the lack of a viable optical method that enables high-performance display functions, achieves the desired sunglass form, and maintains intact OST capability. Conventional design methods using a rotationally symmetric optical structure fundamentally limit designing a compact package.

But efforts to overcome the cumbersome form factor have been revived recently. Sony has demonstrated a full-color eyewear display prototype¹ and Lumus Optical has developed a geometrical light guide approach.² With the increasing sophistication of diamond turning and optics molding technologies, free-form optics show promise for developing lightweight, high-performance, and low-cost AR displays. In particular, free-form, waveguide-like prisms formed by multiple free-form optical surfaces are a very promising approach. The prisms have a waveguide-like geometry that guides the light propagation from a microdisplay engine hidden in the sidebands of glasses or in the space above the eyebrow. In collaborations with academic and industrial partners, my group has been exploring free-form optics for lightweight HMD system designs. We have developed various tools, ranging from new mathematical descriptions of free-form surfaces to surface shape controls, that have enabled a few HMDs to be developed.

Okuyama and Yamazaki of Canon Japan created the wedge-shaped, free-form prism.³Following Yamazaki's pioneering work, a wide variety of wedge-shaped, free-form prisms have been designed, including Olympus's Eye-Trek immersive HMDs. Yamazaki and colleagues further extended the wedge prism work with an attached free-form compensation lens to achieve see-through capability.⁴ These prior works were mostly based on low-resolution, large-sized microdisplays and resulted in systems with fairly high (greater than 4) f-number (f/#, the light collection capability of an optical system, defined as the ratio of focal length to optics diameter). To overcome these limitations, in collaboration with Yongtian Wang's group at the Beijing Institute of Technology, we demonstrated an f/1.875 prototype system based on a 0.61-inch organic LED microdisplay.⁵ Figure 1 shows the optical layout and the prism prototype. The overall optical system, with dimensions of 25×22×12mm, achieved a diagonal field of view (FOV) of 53.5^° and a spatial resolution of 30 line pairs/mm (lps/mm). The optics assembly weight per eye was about 8 grams. More recently, we applied this design to a low-cost, see-through HMD system to achieve a larger FOV and to support high-definition-resolution microdisplays. We also developed a free-form optics tiling method to achieve HMDs with very wide FOV and high resolution.⁶

Figure 1. The optical layout (a) and prism (b) of a prototype free-form, head-mounted display (HMD).

The prisms are precursors to free-form waveguides, but they are better suited for packaging the microdisplays and electronics above the eyebrow. They typically do not offer enough freedom for a sunglass-style system package. Recently, Augmented Vision Inc. (Tucson, AZ) has achieved a sunglass-like wraparound form factor.⁷ Figure 2 shows one of their free-form waveguide designs based on a 0.4-inch liquid-crystal-on-silicon microdisplay panel. The spatial resolution was 80lps/mm for a pixel size of about 6.5μm. The design was optimized so all the components except the waveguide were placed and balanced within the HMD's sideband frame.

Figure 2. A free-form waveguide system for a see-through HMD by Augmented Vision Inc. (Courtesy of Chunyu Gao.)

Free-form optics has enabled high-performance HMD optics and provided opportunities to give HMD systems more sophisticated capabilities. A recent system fully integrated the HMD optics with eye illumination and imaging optics for eye movement tracking.⁸ One design, shown in Figure 3, used a wedge-shaped, free-form prism to combine four optical paths: eye illumination by near-IR (NIR) LEDs, a sensor for capturing NIR-illuminated eye images, a virtual display for viewing microdisplay images, and a see-through path for maintaining a real-world view. These optical innovations enable eye-tracking, virtual display, and see-through in a much more compact package than existing eye-tracked HMD solutions.

Figure 3. Optical layout of a compact, optical see-through, eye-tracked HMD. NIR: Near infrared.

In summary, free-form optical surfaces offer much larger degrees of freedom for optical design than rotationally symmetric optical surfaces. They consequently lead to opportunities for simplifying the overall optical structure, reducing system size and weight, and controlling the system's form factor. Combined with moldable plastic optics, free-form waveguide prisms have been used to successfully achieve low-cost, high-performance, lightweight HMD optics. Compared with other waveguide-form optics based on holographic optical elements (HOE), free-form waveguides have the advantages of high optical performance, excellent scalability in terms of system FOV and resolution, a low mass-production cost, and durability and robustness. The main disadvantage lies in the relatively thicker profile than HOE-based waveguides, which can achieve about 2–3mm thickness with a moderate FOV.

In the future, we plan to develop more innovative methods for free-form surface description, design, and tolerance analysis, and to develop more system-level innovations, with the goal of achieving thin-profile, free-form waveguides in the range of a few millimeters thick.

This work is partially supported by the National Science Foundation awards IIS-1115489 and 06-44446.