Restoring turbulence-degraded underwater images
Underwater imaging is essential for a variety of scientific and military applications, such as optical communications, situational awareness, diver visibility, and mine detection. Unfortunately, underwater imaging is severely impaired by scattering and optical turbulence associated with refraction index fluctuations. Because of the similarities between underwater and atmospheric image degradation, approaches taken to solve the underwater image restoration problem originate in atmospheric turbulence compensation algorithms.1, 2 However, applying unmodified atmospheric techniques to underwater images fails because of the differences in the scale of the refraction index fluctuations.
Scattering may come from the water itself, particulates suspended in the water column, or turbulence present along the propagation path. Particulates contribute more significantly to image degradation in general, especially at larger viewing angles. Near-forward turbulent scattering dominates near the surface, mixed layer, and bottom of the water column. As a result of large index of refraction fluctuations, underwater imaging is subject to stronger degradation than atmospheric imaging. Examples of degradation include significant loss of signal, time-dependent blur, noise, and feature motion or distortion. The multitude of techniques designed to image through atmospheric turbulence has led to advances in imaging through water turbulence and allowed us to develop a multi-frame underwater restoration algorithm.3
Our algorithm, based on the ‘lucky patch’ atmospheric compensation technique, uses a sequence of consecutive short-exposure underwater images as input. We begin by denoising each frame in the sequence to avoid noise amplification during further processing. Next, we estimate the optical flow between the reconstructed image and the current frame. Simultaneously, we compute an uncertainty measure of the optical flow estimation to control for possible errors in this step.4 We then use this optical flow estimate for motion compensation by warping the current frame. Fourth, we compute a gradient-based image quality for the current and reconstructed frame. We then calculate local gain coefficients based on the difference between the computed image quality maps. Finally, we apply a lucky patch fusion process using the current frame, the reconstructed frame, and the local gain coefficients.2 The fusion is not performed in areas of the image where the optical flow uncertainty measure exceeds a certain threshold.
We tested our algorithm on data collected in July 2010 at Lake Skaneateles, where we imaged a target over a 5m path length at various depths in the water column. Figure 1 shows a typical image of the US Air Force 1951 resolution chart collected with a monochromatic electro-optical camera during the exercise. Figure 2 compares a 10-frame mean image with the restored image displaying the resolution advantage of at least two vertical line bands.
Additionally, we collected laboratory imagery in a convective tank that was designed and implemented at the Oceanography Division of the Naval Research Laboratory at Stennis Space Center. The tank houses highly conductive stainless-steel heat dissipation plates on the top and bottom. Temperature-controlled water is used to regulate the plate temperature and, thus, the intensity of the turbulent structure within the tank. Underwater imagery from the laboratory tank, collected with the same camera used in the field exercise, yields a simple pattern of horizontal lines with spatial frequencies of 1250 and 2500 cycles per radian (see Figure 3). Figure 4 compares a mean image to the restored image. The restored image shows better definition in the high-frequency pattern and sharper low-frequency lines.
In summary, we developed and tested an algorithm to restore degraded underwater imagery obtained in the field and in laboratory conditions. Our approach uses a sequence of short exposure frames to obtain a single higher-resolution image. The algorithm is based on the lucky patch atmospheric compensation technique modified with nonlinear gain, nonlocal filtering, and optical flow motion compensation with uncertainty measure control. For future work, we plan to use and modify even more powerful atmospheric imaging techniques like adaptive optics in a challenging underwater environment.
This work is sponsored by Office of Naval Research/Naval Research Laboratory program element 62782N.
Andrey Kanaev received his PhD from the University of Rochester, NY (2001). During his career he worked as a scientist at Corning Inc., Polymath Research Inc., and the Center for Optoelectronics and Optical Communications. He has held his current position for the last seven years.
Weilin (Will) Hou is an oceanographer. He received his PhD from the College of Marine Science, University of South Florida (1997). He helps SPIE organize the Ocean Sensing and Monitoring conference as part of the SPIE Defense and Security Symposium.
