It’s a 3D world: Sensing a bright future for lidar

In January, the SPIE Photonics West panel “From Consumer to Space, 3D Sensing Adoption is Accelerating” went well beyond lidar, to MEMS, beam steering, and spectrum scan technology. Its audience also heard about the software stack, sensor fusion and perception, and other topics expected to help deliver accurate 3D models, said Pierrick Boulay, of Yole Développement and 3D Sensing conference chair.

The panel also included Luis Dussan, Chief Technology Officer and founder of AEye; Pierre Olivier, CTO of LeddarTech; and Gleb Akselrod, CTO and founder of Lumotive.

Key steps forward owe much to new photonics technologies on the emitter side, such as VCSEL arrays for lidar now using multijunction structures, Boulay told Show Daily. With the same input current, the output power can be drastically increased. Regarding edge emitting lasers, or EELs, manufacturers are developing a limited wavelength deviation (EEL wavelength usually deviates with temperature). On the optics side, AR coatings are being developed to reduce signal loss between the laser emitter and receiver all along the optical path.

In short, lidar is popping up everywhere, said Dussan. He predicts that AEye’s version of adaptive lidar will experience steadily accelerating adoption across markets — from automotive and rail to trucking, from intelligent transportation systems (ITS) to construction and mining.

Lidar, which stands for Light Detection and Ranging, uses pulses of laser light to take precise 3D measurements of road conditions for AD-equipped vehicles. Radar-based devices use radio waves.

Several new photonics and optics technologies are enabling key progress on lidar and 3D sensing, noted Akselrod. Lumotive is a leader in all-silicon lidar systems for consumer electronics, industrial automation, robotics and automotive applications.

“One of the big trends that we see in lidar,” Akselrod said, “is the drive to higher levels of integration enabled by semiconductor technologies, and the move away from discrete and mechanical components, which brings down the cost of lidar.

“This means large detector arrays integrated with CMOS logic for processing, smart VCSEL arrays, and of course CMOS-integrated solid-state beam steering — which is what Lumotive pioneered.”

Olivier of LeddarTech said the company is working on how to deliver better perception. “Our research projects aim at exploring different development axes, such as for instance beam steering, which is a key enabling technology for lidar,” he said. They orient activities around the key areas of uncertainty, like instance optical efficiency.

As a technology that’s been around, in principle, in parallel with the laser, for 60 years, lidar’s history is daunting.

“The main challenge has been to find applications that, taken individually, could generate enough volume to justify heavily cost-optimized solutions, and this is why platform solutions are so critical,” said Olivier. He considers lidar technology still under-utilized.

Since 1971, lidar has been a mainstay of space missions. That year, a laser ranger was on the Apollo 15 mission. Other such systems are working in space today.

A NASA report called lidar “a powerful tool in remote sensing from space.” It recalled that lidars operate at a much shorter wavelength with a narrower beam and smaller transmitter and receiver compared to radar (for “radio detection and ranging.”) It noted that lidars carry their own light sources and can continue measuring day and night. Some lidars are known by their targets, such as cloud, wind or CO₂ lidars.

Semiconductor technology is at the heart of the Lumotive Meta-lidar platform. An engineer in the lab with a wafer of the company’s disruptive Light Control Metasurface (LCM™) solid-state beam steering chips. Credit: Lumotive

On the ground, experts predict lidar will play the leading role in everything from advanced driving support systems, known as ADAS, to autonomous driving, or AD, to autonomous mobility applications.

Thus, AEye has built its lidar platform to be flexible. For example, AEye’s lidar enables software-configurable scanning patterns, allowing the sensor to adapt in real-time to different circumstances, such as city vs. highway situations.

Previously, Dussan explained, a driver would be stuck with one sensor, in a static pattern for everything. Different hardware was needed in front and at the back for the car to operate safely in the city and on the highway. “That’s no longer a limitation. With our system, you can just program it so when you are in the city you have a wide field of view. On the highway you have a narrow field of view, usually 10 hertz with really long ranges. That allows our customers to have the best of both worlds.”

Generally, said Olivier, the attractiveness of the market for ADAS and AD, both for fleets and for privately owned vehicles, justifies “developing an optimized platform.” But industry experts see some limitations and glitches in radar-based sensing applications.

‘Autopilot, traffic jam-assist, and automatic braking’

“As a person who has kids,” Dussan said, “one of the things I like about what I’m seeing now, about future vehicles, ones that are going to come out soon, maybe 2024 or 2025, is that we are talking a lot about autopilot, we talk about traffic jam assist, automatic braking. Those things are a minimum expectation of what people want from a smart vehicle.

“But when was the last time you were in an ADAS-equipped vehicle? Those are offering these things, but recent tests show that they only work in great lighting conditions, or when the person is wearing bright clothing. They just are not very accurate. There are a lot of false positives.

“And that’s fundamentally due to the fact that the cameras are interpretive systems. They are great. We have one in our system. But radars have a lot of issues with respect to their resolution. So you are not getting reliable systems,” said Dussan.

The good thing about lidar, the experts said, is that it’s so much more deterministic — consistent, with no randomness.

“What you are going to see is, when there’s a pedestrian in front, when it’s dark, or light, whether he’s wearing dark or light clothes, lidar brings its own light to the party,” Dussan said. “So, it’s going to stop, because it’s going to be able to identify things, from the background, without confusion, and it is going to be able to make that decision.”

What’s exciting about the future of lidar is that all the features that the market is talking about are just going to “lock in and get much, much better,” he said.

So people are going to say, “This actually does work, and it works all the time, when I need it to work. And it doesn’t give me these false positives.”

The more advanced lidar systems will likely reach automotive markets, Dussan predicted, by 2024 or 2025, at the dealerships for new autos. It will show up in industrial markets sooner than that.

Boulay agreed. “Today, we clearly see that mechanical scanning lidars are used mainly by automotive players, and this is expected to remain the case for a few years,” Boulay said. “But in the meantime, solid-state technologies (MEMS-based lidar and flash lidars) are starting to be implemented by a few OEMs (original equipment manufacturers). However, other technologies based on FMCW ranging technology are not expected in the automotive industry before 2025.”

Toyota is coming up with a model with some of the new short-range lidar features — working at about 30 meters, but the longer range lidar systems — ones that can spot a brick at 120 meters — will hit the market in the next few years.

With lidar being “the only deterministic sensor out there,” it’s like the cameras you see on your iPads or iPhones. The complexity of a lidar seeing objects in front of it is minimal, Dussan said. “It’s always going to be there for you. It’s reliable.”

Furthermore, the cost is going down. “It’s paralleling how radar started,” Dussan said. That’s expected to continue in rockets and spaceships as well as in smart lights at intersections.

For consumers, Boulay said, 3D sensing is being increasingly implemented in smart phones, vacuum robots, and payment applications. 3D sensing is progressing rapidly in logistics applications, such as warehouse AMR (autonomous maneuverable robots) and AGV (automated guided vehicles), delivery robots, and autonomous trucks. “Regarding smart infrastructure,” he said, “applications such as intelligent transport systems (ITS), people-flow monitoring, and highway tolling are emerging.”

For now, Olivier said, automotive has the potential to be the “killer app” for lidar. Some challenges remain unmet, such as achieving a larger field of view or sufficiently long detection range, but, Olivier said, LeddarTech, for one, has “unique and innovative solutions.”

Lidar’s industry circles are expected to grow even wider, to include the new installations on the streets of “smart cities.”

“We see a lot of analogues between what happened with cameras and what will happen with lidar,” said Lumotive’s Akselrod.

“Smart phones drove camera technology, and now there are billions of cameras enabling every kind of application imaginable. I think something similar will happen with lidar, where initially the interest is driven by automotive, but as lidars become cheap and tiny, 3D sensing will proliferate everywhere from robots to smartphones, to city infrastructure.”

Added Dussan, “We know that construction wants to make vehicles safer, so they want the smart phones to be able to sense everything around them. We see cocoon lidars, an important ask from the general public — to surround everything with lidar, to be aware of everything around them.

“Our approach is helping to make the adoption of lidar happen faster. So we’ve removed the complexity from the hardware.”

Lidar beyond automotive

That is going to lead to smoother adoption of lidar beyond the automotive space.

“We can apply that same idea of change in the software across different markets,” said Dussan. “Trucking, heavy machinery, construction, mining, rail, intelligent transportation, retail, just to name a few markets that will benefit from lidar.

In short, construction apps seem to be a natural for growth.

AEye, Dussan said, offers “one of the most solid state systems out there,” in terms of performance range and cost. One of the most exciting developments, he said, is that in AEye’s work with the construction industry the systems can survive major vibration shock. “We are basically as solid state as your phone, and that allows these environments to be even rougher and harder, and the lidar systems will survive.”

Sensors used in the AEye lidar system are installed on top of this Jaguar, seen at AEye headquarters in Dublin, California. Credit: AEye

With that increased reliability, experts said the industry seems certain to see migration of these heavy-duty vehicles in harsh environments to lidar systems.

Three other companies were given high marks for integrating complete lidar based systems. They included Seoul Robotics, based in South Korea; TuSimple, based in San Diego; and Robotic Research, based in Maryland.

Seoul Robotics’ creates software products for robotic vehicle systems, intelligent transportation systems, ADAS apps, and processing of lidar data as a partner with major lidar companies.

TuSimple concentrates on trucking perception and vehicle movement generally.

Robotics Research develops systems for autonomous perception by vehicles, along with robotic technology. It says its platforms can be widely employed, from tugs, UAVs, and shuttles, to heavy-duty transit buses and full-size logistics trucks.

Calculating the cost

“Of course cost and functional safety are also important topics,” said Olivier. “And once the industry addresses these issues the potential to leverage the technology in other markets is very large.”

“Smaller, faster, cheaper, no question,” said Dussan. “Lidar is emulating what’s happened in the camera and radar space.

“The power of the market economy is going to drive down costs. That’s expected and welcome, especially in our supply chain. As we increase volume, we are going to see lower component costs as well,” Dussan said.

“Newer methods and techniques will change how we do lidar, or cameras, or radar. And that’s going to further enhance the economies of the robotaxi or ADAS markets. That means smaller, faster, cheaper, and higher performing in general.”

What about insurance discounts? When there’s enough statistical evidence, the discounts will follow, the experts said, but there’s not yet enough sample data.

“All you have is GM’s Supercruise system, and Tesla’s FSD (full self-driving) to some extent,” said Dussan.

In all, the 3D panelists made it clear that it’s not a matter of if but when lidar will be adopted across markets, and everybody should prepare for that. “We have collected a lot of data on autonomous systems now,” said Dussan, “people are realizing its value.”

Coping with ‘corner cases’

Meanwhile, the auto industry is facing challenges for ADAS or autonomous driving systems from “corner cases” — situations where an autonomous vehicle has difficulty addressing or identifying an object or circumstance, which in turn can result in an accident or fatality. An example might be a brick in the road, at a distance, at night.

 “People used to say 90 percent of the problems were solvable by a certain set of features,” Dussan said. “And then you have the other 10 percent that are really hard.

“But now what we are seeing is that there is a set of corner cases that make up 80 percent of the problem in robotaxis or autonomous vehicles — and that 80 percent of corner cases is different all the time, and they can’t be solved with just one type of pattern. And you need an adaptive, intelligent lidar system, to overcome them.”

For all of these industries, said Olivier, the Photonics West event is crucial, since optics and photonics are an essential part of lidar. It gives, he said, an opportunity “to go down the different rabbit holes that may lead to new research activities. That is what makes it a must-attend event.”

Ford Burkhart is a science and technology writer based in the US. A version of this article appeared in the 2022 Photonics West Show Daily.