High-speed, ultracompact, and flexible 3D optical sensing systems are greatly needed in space-constrained applications involving motion, such as 3D endoscopic imaging. One way to achieve high-quality 3D sensing with high speeds is with the state-of-the-art ‘digital fringe projection’ (DFP) technique.1A DFP system typically uses a digital video projector to project computer generated sinusoidal patterns onto the object surface. A camera captures the patterns, distorted by the object surface, from another viewing perspective. Once correspondence between the projector and the camera is established using the sinusoidal patterns and the system is calibrated, the 3D coordinates for each camera point can be reconstructed based on triangulation. However, DFP systems are usually bulky and their complicated design means that it is nontrivial to miniaturize them.2
We have recently developed a binary-defocusing method that could simplify DFP system development and simultaneously achieve unprecedentedly high (kHz) speeds.3 However, mechanical motion of digital micromirrors in this system imposes a speed bottleneck and its transient effect (when the digital micromirror of the projector is still switching between on and off states) is non-negligible.4The transient effect could be problematic for high-speed applications, such as capturing rapidly occurring mechanical and biological events (e.g., high-speed vibrations, explosions, and live beating hearts).
Unlike DFP, fiber-optic interferometric fringe projection uses only two very small (μm) fibers to generate a sinusoidal pattern.5 For high-quality 3D sensing, phase-shifting methods are usually adopted, and one of the most popular approaches is to use a piezoelectric transducer (PZT) to introduce phase shifts.6, 7 In such methods, the optical fibers are wrapped around the PZT tube. By modulating the input voltage, the PZT tube is deformed to stretch the fibers and thus modulate the optical path of the light to introduce phase shifts. Alternatively, phase shifts can be introduced by modulating the frequency of a tunable laser through varying injection current.8, 9 These approaches have demonstrated their success for compact system design. However, their measurement speeds are limited either by the modulation rate of the mechanical moving part (in the case of the PZT), or by the tuning rate of the injection current. To the best of our knowledge, none of them have been able to match even the kilohertz-rate 3D sensing of the DFP systems.
To address the speed bottleneck in fiber-optic interferometric fringe projection, we have introduced a lithium niobate electro-optic phase modulator, which can generate phase shifts at megahertz, even gigahertz, rates.10 Lithium niobate is a kind of crystal whose refractive index is a function of the strength of the local electric field. Once the lithium niobate is exposed to the electric field, the time that the light takes to pass through it will change, which is directly proportional to the phase delay of the light. Therefore, high-speed phase modulation can be realized by applying ultrashort electronic voltage pulses to the crystal.
This modulator could achieve unprecedented high speeds while maintaining a miniature system comparable in size with the fiber-optic interference system. In a prototype system with only limited laser output power and a large measurement range, we achieved 100Hz 3D shape measurement speeds (see Figure 1). Although 100Hz is clearly several orders of magnitude lower than our desired speeds, it is adequate proof of principle.
Figure 1. Schematic diagram of the proposed fiber optical system.
In our system, laser light emitted from the laser diode is split into two different optical paths by a coupler, and polarization controllers alter one beam's polarization state. The phase modulator generates phase shifts in the other beam and, finally, phase-shifted sinusoidal patterns are generated by fiber interference. A function generator controls the input voltage to the phase modulator for generating the phase shifts. It also triggers a high-speed camera to capture images of the fringes simultaneous with projection of the phase-shifted fringe pattern onto the subject of the study. In a similar way to DFP systems, 3D information about the subject can be obtained by analyzing the captured fringe patterns.
We evaluated the performance of our prototype system by measuring a small complex 3D statue: see Figure 2(a). To reduce the random phase-shift error caused by using single-mode fibers, we applied a 10-step phase-shifting algorithm for phase retrieval. Figure 2(b) shows one of the phase-shifted fringe patterns. After applying the phase-shifting algorithm, we used a spatial phase unwrapping algorithm to obtain a continuous phase map.11The unwrapped phase map was further converted to a 3D shape by adopting a single reference-plane-based calibration method: see Figure 2(c) for the successful 3D measurement result.
Figure 2. Preliminary experimental results. (a) The small complex statue we measured. (b) A captured fringe pattern. (c) The reconstructed 3D shape, shown with a color scale of blue to red representing -30mm to 10mm.
In summary, we have integrated a superfast lithium niobate electro-optic phase modulator into a miniature fiber-optic fringe projection system that could achieve a 3D sensing rate on the megahertz or gigahertz scale. Preliminary experimental results demonstrated the promise of the proposed method. Our future work will focus on solving some technical challenges of the proposed technique (such as phase drifting and system calibration), as well as the applications. We are currently developing next-generation 3D endoscopy technology for both medical and industrial applications.
This study was sponsored by the National Science Foundation (NSF) under grants CMMI-1150711 and CMMI-1300376. The views expressed are those of the authors and not necessarily those of the NSF.
Beiwen Li, Song Zhang
Iowa State University (ISU)
Beiwen Li is a doctoral student at ISU. He received a masters from ISU in 2014 and a BS from Beihang University in 2012, and his research interests include 3D optical metrology, image processing, and fiber optics.
Song Zhang is an associate professor of mechanical engineering at ISU, where he is the principal investigator of the 3D machine-vision laboratory. His major research focuses on high-speed, high-resolution 3D optical sensing, 3D sensing data analysis, human-computer interaction, and 3D video telecommunications.
Pan Ou is an associate professor. He received his PhD from Tsinghua University, China, in 2005. His research interests include fiber optics, image processing, and machine vision.
1. S. Zhang, Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques, Opt. Laser Eng. 48(2), p. 149-158, 2010.
2. J. Geng, J. Xie, Review of 3-D endoscopic surface imaging techniques, IEEE Sens. J. 14(4), p. 945-960, 2014.
3. S. Zhang, D. van der Weide, J. Oliver, Superfast phase-shifting method for 3-D shape measurement, Opt. Express 18(9), p. 9684-9689, 2010.
4. Y. Wang, B. Bhattacharya, E. H. Winer, P. Kosmicki, W. H. El-Ratal, S. Zhang, Digital micromirror transient response influence on superfast 3D shape measurement, Opt. Laser Eng. 58, p. 19-26, 2014.
5. T. L. Pennington, H. Xiao, R. May, A. Wang, Miniaturized 3-D surface profilometer using a fiber optic coupler, Opt. Laser Tech. 33(5), p. 313-320, 2001.
6. C. Lv, F. Duan, F. Zhang, X. Duan, E. Bo, F. Feng, Fiber optic interferometer fringe projector using sinusoidal phase-modulating, Proc. SPIE
8916, p. 89160V, 2013. doi:10.1117/12.2035862
7. Z. Chao, D. Fa-Jie, Phase stepping methods based on PTDC for fiber-optic projected-fringe digital interferometry, Opt. Laser Tech. 44(4), p. 1089-1094, 2012.
8. B. Wang, X. Wang, O. Sasaki, Z. Li, Sinusoidal phase-modulating interferometer insensitive to intensity modulation of a laser diode for displacement measurement, Appl. Opt. 51(12), p. 1939-1944, 2012.
9. R. Zhu, Q. Song, R. Zhu, J. Li, Three-dimensional shape measurement with wavelength-modulated laser base on fiber optic interferometer projection, Proc. SPIE
8769, p. 876938, 2013. doi:10.1117/12.2018935
10. B. Li, P. Ou, S. Zhang, High-speed 3D shape measurement with fiber interference, Proc. SPIE
9203, p. 920310, 2014. doi:10.1117/12.2060562
11. S. Zhang, X. Li, S.-T. Yau, Multilevel quality-guided phase unwrapping algorithm for real-time three-dimensional shape reconstruction, Appl. Opt. 46(1), p. 50-57, 2007.