# American Institute of Mathematical Sciences

August  2013, 7(3): 839-861. doi: 10.3934/ipi.2013.7.839

## Video stabilization of atmospheric turbulence distortion

 1 Department of Mathematics, University of California Los Angeles, Los Angeles, CA, 90095, United States, United States 2 School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160 3 Computer Science Department, University of California Los Angeles, Los Angeles, CA, 90095, United States

Received  April 2012 Revised  March 2013 Published  September 2013

We present a method to enhance the quality of a video sequence captured through a turbulent atmospheric medium, and give an estimate of the radiance of the distant scene, represented as a latent image,'' which is assumed to be static throughout the video. Due to atmospheric turbulence, temporal averaging produces a blurred version of the scene's radiance. We propose a method combining Sobolev gradient and Laplacian to stabilize the video sequence, and a latent image is further found utilizing the lucky region" method. The video sequence is stabilized while keeping sharp details, and the latent image shows more consistent straight edges. We analyze the well-posedness for the stabilizing PDE and the linear stability of the numerical scheme.
Citation: Yifei Lou, Sung Ha Kang, Stefano Soatto, Andrea L. Bertozzi. Video stabilization of atmospheric turbulence distortion. Inverse Problems & Imaging, 2013, 7 (3) : 839-861. doi: 10.3934/ipi.2013.7.839
##### References:
 [1] L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590. doi: 10.1137/0731032. [2] M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009). doi: 10.1117/12.828332. [3] A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490. doi: 10.1137/040616024. [4] A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123. [5] K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471. doi: 10.1364/AO.43.000471. [6] J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981. doi: 10.1137/090771260. [7] J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53. doi: 10.1109/ICIP.2002.1038901. [8] L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998). [9] D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881. doi: 10.1109/ICASSP.2001.941311. [10] D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372. doi: 10.1364/JOSA.56.001372. [11] S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004). [12] J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11. doi: 10.1117/12.917234. [13] P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973). [14] M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607. doi: 10.1109/CVPR.2010.5540158. [15] R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52. doi: 10.1364/JOSA.54.000052. [16] J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008). [17] D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340. doi: 10.1109/LGRS.2007.895691. [18] D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244. [19] Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531. doi: 10.3934/ipi.2012.6.531. [20] A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64. doi: 10.1137/080724289. [21] M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013). doi: 10.1007/s10851-012-0410-7. [22] A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999). [23] P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629. doi: 10.1109/34.56205. [24] J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794. doi: 10.1364/JOSAA.19.001794. [25] M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996). doi: 10.1117/1.601043. [26] M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1. [27] G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345. [28] D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011). doi: 10.1117/1.3532999. [29] M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312. doi: 10.1364/JOSAA.18.001312. [30] P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010). doi: 10.1103/PhysRevA.82.033817. [31] X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010). doi: 10.1117/12.840127. [32] X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1. doi: 10.1109/ICCPHOT.2011.5753122. [33] X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157.

show all references

##### References:
 [1] L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590. doi: 10.1137/0731032. [2] M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009). doi: 10.1117/12.828332. [3] A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490. doi: 10.1137/040616024. [4] A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123. [5] K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471. doi: 10.1364/AO.43.000471. [6] J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981. doi: 10.1137/090771260. [7] J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53. doi: 10.1109/ICIP.2002.1038901. [8] L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998). [9] D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881. doi: 10.1109/ICASSP.2001.941311. [10] D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372. doi: 10.1364/JOSA.56.001372. [11] S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004). [12] J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11. doi: 10.1117/12.917234. [13] P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973). [14] M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607. doi: 10.1109/CVPR.2010.5540158. [15] R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52. doi: 10.1364/JOSA.54.000052. [16] J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008). [17] D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340. doi: 10.1109/LGRS.2007.895691. [18] D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244. [19] Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531. doi: 10.3934/ipi.2012.6.531. [20] A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64. doi: 10.1137/080724289. [21] M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013). doi: 10.1007/s10851-012-0410-7. [22] A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999). [23] P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629. doi: 10.1109/34.56205. [24] J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794. doi: 10.1364/JOSAA.19.001794. [25] M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996). doi: 10.1117/1.601043. [26] M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1. [27] G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345. [28] D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011). doi: 10.1117/1.3532999. [29] M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312. doi: 10.1364/JOSAA.18.001312. [30] P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010). doi: 10.1103/PhysRevA.82.033817. [31] X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010). doi: 10.1117/12.840127. [32] X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1. doi: 10.1109/ICCPHOT.2011.5753122. [33] X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157.

2018 Impact Factor: 1.469