Improving car safety with low-light, high-speed cameras

Public safety and security could benefit from the ability to acquire images at very low light levels (e.g., no illumination), at high frame rates (possibly thousands of frames per second), and also with distance resolution (possibly millimeter precision). Nowadays, the imager market offers a broad portfolio of either commercial- or scientific-grade cameras, ranging from consumer CMOS cameras up to high-end CCD imagers, but none of them simultaneously offers both high speed and ultra-high sensitivity. Our goal is to process pixel-level intensity data (for 2D imaging) and depth information (for 3D ranging, i.e., an object's distance) of rapidly changing scenes in light-starved environments. Our consortium has focused on two principal applications: automotive safety systems and security surveillance.

Our technology is based on solid-state single-photon avalanche diodes (SPADs), which are detectors able to provide a pulse when a single visible or near-IR photon hits its active area.¹ As ‘trigger’ detectors,² a SPAD's output is a digital pulse, and it is much more sensitive than a ‘linear’ detector with an analog output current (i.e., proportional to the flux of a huge number of photons). We have developed advanced arrays of SPAD-based pixels, able not only to count single photons but also to accurately tag them with their arrival time and thus provide a full image of the scene under observation.³

Initially, we designed and developed novel state-of-the-art, high-performance SPAD detectors, smart pixels, and mini-arrays with unprecedented performance compared to the best SPAD detectors with 30μm active area diameter, in terms of sensitivity (60% at 400nm wavelength), noise (100cps for 50μm diameter SPADs),⁴ and very large active areas⁵ with a diameter of up to 0.5mm. These properties result from our design criteria for above-breakdown SPAD operation of confining the avalanche process within the active area, avoiding issues at the junction edges, and tailoring the high-field region thickness. It is also a proof of the very high quality and reliability of the high-voltage CMOS processing we have developed, which is based on 0.35μm CMOS technology optimized for automotive and optoelectronic applications.

When acquiring 3D depth-resolved images for short-range applications, it is usual to actively illuminate the scene, either by pulsed-light or continuous-wave-modulated LEDs or by lasers. However, the required photon signal depends on the desired depth resolution and scene condition (background illumination, object reflectivity, and active light power), and no single illumination method is consistently superior. Moreover, sometimes it is important to acquire a ‘passive’ 2D image of the scene (i.e., with no active illumination). We therefore subsequently developed a fully functional ‘universal’ SPAD camera (see Figure 1), able to acquire both 2D images and 3D ranging videos of objects up to 40m from the camera, with 64×32 pixel resolution (see Figure 2).⁶ We have now developed a second 3D ranging camera, working with both pulsed-light and continuous-wave modulation or none, able to compute the time of flight of incoming photons,⁷ and hence objects' depth distance. Unlike other cameras, our imager is based on smart pixels containing a SPAD detector and a sensing front-end. It offers in-pixel time-to-digital conversion of the time of flight of each incoming photon with 290ps single-shot resolution (corresponding to 9cm), which improves further after repeated acquisitions at high frame-rate laser pulses.

Figure 1. Left: Chip and camera. Right: 2D and 3D frames of a high-frame-rate movie made with the camera.

Our systems include high frame-rate (>200 images/s), short-range (10–40m) 3D ranging cameras for prompt intervention in front and back pre-crash automotive safety systems (see Figure 2) and also long-range (200m–1km) 3D ranging systems for security surveillance, with centimeter resolution. In addition, our components offer single-photon sensitivity over the whole visible and near-IR wavelength range (300–900nm), microsecond image acquisition times for videos of fast optical transients, and reduced-pixel-density, cost-effective on-chip processing. Picosecond resolution for further advanced time-of-flight investigations is intrinsic to the SPAD detector.

Figure 2. Automotive pre-crash scenario for our short-range single-photon avalanche diode-based safety camera.

We are now working to disseminate and exploit our devices to wider communities and markets. The principal applications will be in safety (e.g., environmental surveillance, traffic and workplace safety monitoring, product safety analysis, food and agriculture quality, and safety assessment) and security (access control, biometrics, surveillance systems, dangerous agents monitoring, homeland security, and fire hazards). Other applications will include microscopy, biology, and adaptive optics.

This work is supported by the MiSPiA (microelectronic single-photon 3D imaging arrays) project under the ICT theme of the European Commission's 7th Framework Programme (FP7, 2007–2013), grant 257646.⁸The consortium consists of seven partners from the Politecnico di Milano, Italy, the Fraunhofer Institute for Microelectronic Circuits and Systems, Germany, Heriot-Watt University, Scotland, Micro Photon Devices srl, Italy, Centro Ricerche Fiat, Italy, EMZA Visual Sense, Israel, and CF Consulting, Italy.