Metamaterials Tackle Real-world Challenges

Development of a metalens depth sensor and research into hyperbolic metamaterials indicate the diversity of potential applications.
26 March 2020
By Tim Hayes
Compact single-shot metalens depth sensor inspired by eyes of jumping spiders
“Spiders” Compact single-shot metalens depth sensor inspired by eyes of jumping spiders. (Abstract notes: Jumping spiders (Salticidae) rely on accurate depth perception for predation and navigation. They accomplish depth perception, despite their tiny brains, by using specialized optics. Each principal eye includes a multi-tiered retina that simultaneously receives multiple images with different amounts of defocus, and from these images, distance is decoded with relatively little computation. We introduce a compact depth sensor that is inspired by the jumping spider. It combines novel metalens optics, which modifies the phase of incident light at a subwavelength scale, with efficient computations to measure depth from image defocus.) Credit: Capasso Group, Harvard University.

Metamaterials were once again a focus of discussion at this year's SPIE Photonics West, thanks to the range of real-world applications that might benefit from their novel optical properties, and the current technical challenges that they could solve.

A forecast published in late 2019 indicated that the overall market for metamaterials could exceed $10 billion by 2030, stimulated initially by demand for antenna technologies from 5G wireless networks. A widening applications base should then follow, with the proliferation of autonomous systems and platforms likely to spur growth in metamaterial- based sensing devices.

The Harvard University group of Federico Capasso has been a hub of activity in the metamaterials field, and is presenting research into one such sensing device here, this week. This novel technology could significantly improve the performance of microrobots and microsensor networks, addressing the shortcomings of the conventional visual depth sensors that form a vital part of many autonomous platforms.

Depth sensing has traditionally involved a combination of cameras, algorithms and light sources to determine the 3D shapes of surrounding objects, whether through Lidar, time-of-flight cameras or other methods. But these techniques are often inherently unsuited to low-power mobile platforms, where size, weight and power consumption are critical considerations. Harvard's new device employs a design inspired by the eyes of jumping spiders, a species where each principal eye includes a multitiered retina that simultaneously receives multiple images with different amounts of defocus. From these defocused images, distance is decoded by the animal's tiny brain with relatively little computation.

Building from this basic principle, the Harvard group employed a metalens to split incoming light into two differently defocused images, falling on distinct regions of a planar photosensor.

"Defocus is the essential concept here," commented Zhujun Shi of Harvard's John A. Paulson School of Engineering and Applied Sciences. "Previous implementations of similar algorithms have required mechanical adjustment of the sensors, either using an electrically tunable liquid lens or a tunable aperture. This not only makes the hardware and control mechanisms complicated, but also limits the temporal resolution, as well as introducing unwanted artifacts such as motion blur."

Using a metamaterial instead allowed the team to get rid of those moving parts. A single metasurface, consisting of patterned subwavelength-spaced nanostructures of titanium dioxide on a glass substrate, can function as two off-axis lenses in a single structure, encoding two complementary lens phase profiles with distinct focal lengths and lateral offsets. This allows the depth sensor to become compact, single-shot and completely passive; according to the Harvard project, its device is the first demonstration of a depth sensor that demonstrates all those qualities.

Microrobots and microsensor networks

The sensor is also intended to be computationally efficient. Alongside the metalens and its desired focal lengths, the team developed a depth-reconstruction algorithm allowing accurate depth maps to be computed from the two simultaneous images, with calculations that are spatially localized and few in number. The goal was to allow the depth computations for each specific image pixel to involve only a small spatial neighborhood, and require no additional correspondence search after initial calibration.

As a result, the project believes that it has reduced the amount of computation involved in a depth sensing operation by a factor of ten, compared to previous traditional implementations. In trials, a device using a 3 mm-diameter metalens proved able to calculate depth over a distance range of 10 cm, and did so using fewer than 700 floating point operations per output pixel.
Future iterations of the device could reduce the computation burden further and make the depth sensor still more efficient. The current design predominantly uses established metalens design principles, and in the future the team intends to develop an end-to-end optimization pipeline that co-optimizes both the metalens and the algorithm. This could lead to a paradigm shift in metalens design and computational sensing.

"This integration of nanophotonics and efficient computation brings artificial depth sensing closer to being feasible on millimeter-scale, microwatt platforms such as microrobots and microsensor networks," said Zhujun Shi. "Currently, the biggest challenge is the operating bandwidth. Our proof-of-principle demo device works only for green light, and to extend the bandwidth and realize broadband white light operation, we will utilize the dispersion engineering method to design an achromatic metalens. Our group has already published several papers on dispersion engineering of metasurfaces for a different application."

The potential applications for such a depth sensor could ultimately be broad, especially as autonomous systems intended to operate in real-world environments become more sophisticated.

"Compared to conventional approaches, our device does not require active lighting as time-of-flight sensors do, or the heavy computational load that a learning- based method involves," Zhujun Shi noted. "The ultra-lightweight and compact architecture could make it particularly suitable for small platforms that have a limited power budget, such as microrobots and wearable devices, where conventional approaches do not work. Many companies have expressed interest in our research, and we definitely believe that these principles are going to have a real-world impact in the future."

Artistic visualizations of the metalens
Artistic visualizations of the metalens

Artistic visualizations of the metalens. Nature provides diverse solutions to passive visual depth sensing. Evolution has produced vision systems that are highly specialized and efficient, delivering depth perception capabilities that often surpass those of existing artificial depth sensors. Here, we learn from the eyes of jumping spiders and demonstrate a metalens depth sensor that shares the compactness and high computational efficiency of its biological counterpart. Our device combines multifunctional metalenses, ultrathin nanophotonic components that control light at a sub-wavelength scale, and efficient computations to measure depth from image defocus. Compared with previous passive artificial depth sensors, our bioinspired design is lightweight, single-shot and requires a small amount of computation. The integration of nano-photonics and efficient computation establishes a new paradigm for design in computational sensing. Credit: Capasso Group, Harvard University.

Hyperbolic metamaterials: extreme anisotropy put to good use

Another form of metamaterial under discussion at SPIE Photonics West for its novel optical properties is the class termed hyperbolic metamaterials (HMMs), now becoming attractive candidates for applications in imaging, waveguiding and sensing.

The label of "hyperbolic" derives from plots of the propagating wave vectors, or k-vectors, allowed by the materials. When drawn as iso frequency contours, the k-vectors allowed naturally by a conventional isotropic material form a simple closed sphere, but for HMMs the same plot becomes an open hyperboloid shape. In real-world practice, this means that HMMs demonstrate extreme anisotropy in their electromagnetic properties, and can effectively act as a dielectric in one direction and a metal in another.

"In many ways, the HMM platform is an alternate design approach to the traditional way of designing a metamaterial structure for effective media properties, which is to construct it from individual ‘meta-atom' cells," commented Augustine Urbas of the Air Force Research Laboratory, chair of a Photonics West conference session on hyperbolic metamaterials. "In that aspect they parallel other metamaterial approaches, but with some distinct characteristics. They can be made to exhibit many of the properties that other metamaterials systems show, but the geometries are very distinct."

While approaches based on meta-atom cells usually rely on sub-wavelength and typically resonant unit cells, this need not be the case for HMMs, which instead only require that the material be effectively both conductive and nonconductive in different directions. Pioneers in the HMM field realized that a simple multilayer of metal and dielectric could achieve this, and early work on these systems indicated their many unique properties.

"The building block materials can be the same as typical metamaterials and plasmonic metamaterials, but the geometries are simpler and, in many cases, non-resonant," said Urbas. "Typically, HMMs are multilayers or arrays of parallel wires, for example, to achieve the anisotropic metallic behavior. I am not sure I would call it a radical departure from other metamaterials, but the simplicity of fabrication and the flexibility of the single platform make it distinct."

That ability to tune and change the properties of HMMs will be a factor in their ultimate real-world usage, with one focus of discussion at Photonics West being how changes in their effective properties can either trigger a non-linear response, or switch and tune a linear response.

Dynamic optical systems

Urbas believes that the possibility of integrating HMMs into photonic systems and their subsequent use in real-world applications is one of the most appealing aspects of these materials.

"Their simple geometries, and fact that HMM properties have been observed in nanostructured and natural materials as well, offer direct ways to integrate them into technological platforms where more traditional meta-atom systems may be difficult or costly," he noted. "Explorations on using them in dynamic optical systems and photonics are ongoing and the initial steps toward this will be among the reports at this Photonics West. There may not be an individual property or effect that HMMs uniquely exhibit that other meta systems do not, but I think they offer some technological integration potential that may lead to early applications."

Some challenges on this path still remain, however. One is how to describe the behavior of dynamic and nonlinear HMM systems in terms of their effective properties, and identifying whether this homogenization is useful for engineering these systems, or for capturing their interactions and effects. Experimental and theoretical studies are currently exploring this question, alongside other HMM hot topics such as whether there is a role for the materials in quantum effects.

"Since the properties are broadband, it is possible that these materials can influence and mediate interactions between quantum systems," said Urbas. "Coupled with their tunable properties, this may prove enabling. Some initial reports have suggested these effects, and this is an area of future study."

Tim Hayes is a freelance writer based in the UK. He was previously industry editor of and Optics & Laser Europe magazine. A version of this article appeared in the 2020 Photonics West Show Daily.

Related SPIE content:

Through a Metalens, Lightly

Federico Capasso: Flat optics: from high-performance metalenses to structured light

Federico Capasso: Metasurface flat optics: Unifying semiconductor manufacturing and lens making

Using microfluidics for tunable metamaterials

Plasmonics and metamaterials: looking beyond gold and silver

Enjoy this article?
Get similar news in your inbox
Get more stories from SPIE
Recent News