Share Email Print

Proceedings Paper

Deep learning for software-based turbulence mitigation in long-range imaging
Author(s): Robert Nieuwenhuizen; Klamer Schutte
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Optical imaging over long horizontal paths often suffers from the effects of atmospheric turbulence. Dynamic density variations in the air result in random spatiotemporally varying shifts and blurs in the recorded images. Software based turbulence mitigation algorithms provide a means of computationally reducing these turbulence effects in video. They provide camera operators with sharper and more stable imagery which supports them on visual recognition and identification tasks. Most turbulence mitigation algorithms rely on a form of temporal low pass filtering to remove the turbulence induced fluctuations. This filtering also suppresses high spatial frequency information and thus limits the ability of these algorithms to recover fine details. Here we propose a turbulence mitigation algorithm which employs a deep neural network to recover high spatial frequency information from turbulence degraded video. The proposed algorithm builds on our previous approach, where frame-to-frame estimates of image shifts are used to combine multiple frames. This approach is amended by using a deep neural network to deblur output images. For the related task of single image super-resolution, we have previously shown that such neural networks provide state-of-the-art image reconstruction performance. Here our neural network is trained using semi-synthetic image sequences of static scenes with simulated turbulence. We show that our deep learning based approach provides on semi-synthetic test data a substantial performance increase compared to our previous sharpening approach. We also apply our algorithm to real long range imagery of ships at sea and find a perceptually similar improvement in image quality as for semi-synthetic data.

Paper Details

Date Published: 19 September 2019
PDF: 10 pages
Proc. SPIE 11169, Artificial Intelligence and Machine Learning in Defense Applications, 111690J (19 September 2019); doi: 10.1117/12.2532603
Show Author Affiliations
Robert Nieuwenhuizen, TNO Defence, Security and Safety (Netherlands)
Klamer Schutte, TNO Defence, Security and Safety (Netherlands)

Published in SPIE Proceedings Vol. 11169:
Artificial Intelligence and Machine Learning in Defense Applications
Judith Dijk, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?