2013, 7(3): 839-861. doi: 10.3934/ipi.2013.7.839

Video stabilization of atmospheric turbulence distortion

1. 

Department of Mathematics, University of California Los Angeles, Los Angeles, CA, 90095, United States, United States

2. 

School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160

3. 

Computer Science Department, University of California Los Angeles, Los Angeles, CA, 90095, United States

Received  April 2012 Revised  March 2013 Published  September 2013

We present a method to enhance the quality of a video sequence captured through a turbulent atmospheric medium, and give an estimate of the radiance of the distant scene, represented as a ``latent image,'' which is assumed to be static throughout the video. Due to atmospheric turbulence, temporal averaging produces a blurred version of the scene's radiance. We propose a method combining Sobolev gradient and Laplacian to stabilize the video sequence, and a latent image is further found utilizing the ``lucky region" method. The video sequence is stabilized while keeping sharp details, and the latent image shows more consistent straight edges. We analyze the well-posedness for the stabilizing PDE and the linear stability of the numerical scheme.
Citation: Yifei Lou, Sung Ha Kang, Stefano Soatto, Andrea L. Bertozzi. Video stabilization of atmospheric turbulence distortion. Inverse Problems & Imaging, 2013, 7 (3) : 839-861. doi: 10.3934/ipi.2013.7.839
References:
[1]

L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590. doi: 10.1137/0731032.

[2]

M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009). doi: 10.1117/12.828332.

[3]

A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490. doi: 10.1137/040616024.

[4]

A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123.

[5]

K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471. doi: 10.1364/AO.43.000471.

[6]

J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981. doi: 10.1137/090771260.

[7]

J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53. doi: 10.1109/ICIP.2002.1038901.

[8]

L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998).

[9]

D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881. doi: 10.1109/ICASSP.2001.941311.

[10]

D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372. doi: 10.1364/JOSA.56.001372.

[11]

S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004).

[12]

J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11. doi: 10.1117/12.917234.

[13]

P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973).

[14]

M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607. doi: 10.1109/CVPR.2010.5540158.

[15]

R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52. doi: 10.1364/JOSA.54.000052.

[16]

J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008).

[17]

D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340. doi: 10.1109/LGRS.2007.895691.

[18]

D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244.

[19]

Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531. doi: 10.3934/ipi.2012.6.531.

[20]

A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64. doi: 10.1137/080724289.

[21]

M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013). doi: 10.1007/s10851-012-0410-7.

[22]

A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999).

[23]

P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629. doi: 10.1109/34.56205.

[24]

J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794. doi: 10.1364/JOSAA.19.001794.

[25]

M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996). doi: 10.1117/1.601043.

[26]

M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1.

[27]

G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345.

[28]

D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011). doi: 10.1117/1.3532999.

[29]

M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312. doi: 10.1364/JOSAA.18.001312.

[30]

P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010). doi: 10.1103/PhysRevA.82.033817.

[31]

X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010). doi: 10.1117/12.840127.

[32]

X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1. doi: 10.1109/ICCPHOT.2011.5753122.

[33]

X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157.

show all references

References:
[1]

L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590. doi: 10.1137/0731032.

[2]

M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009). doi: 10.1117/12.828332.

[3]

A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490. doi: 10.1137/040616024.

[4]

A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123.

[5]

K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471. doi: 10.1364/AO.43.000471.

[6]

J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981. doi: 10.1137/090771260.

[7]

J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53. doi: 10.1109/ICIP.2002.1038901.

[8]

L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998).

[9]

D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881. doi: 10.1109/ICASSP.2001.941311.

[10]

D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372. doi: 10.1364/JOSA.56.001372.

[11]

S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004).

[12]

J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11. doi: 10.1117/12.917234.

[13]

P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973).

[14]

M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607. doi: 10.1109/CVPR.2010.5540158.

[15]

R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52. doi: 10.1364/JOSA.54.000052.

[16]

J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008).

[17]

D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340. doi: 10.1109/LGRS.2007.895691.

[18]

D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244.

[19]

Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531. doi: 10.3934/ipi.2012.6.531.

[20]

A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64. doi: 10.1137/080724289.

[21]

M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013). doi: 10.1007/s10851-012-0410-7.

[22]

A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999).

[23]

P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629. doi: 10.1109/34.56205.

[24]

J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794. doi: 10.1364/JOSAA.19.001794.

[25]

M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996). doi: 10.1117/1.601043.

[26]

M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1.

[27]

G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345.

[28]

D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011). doi: 10.1117/1.3532999.

[29]

M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312. doi: 10.1364/JOSAA.18.001312.

[30]

P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010). doi: 10.1103/PhysRevA.82.033817.

[31]

X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010). doi: 10.1117/12.840127.

[32]

X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1. doi: 10.1109/ICCPHOT.2011.5753122.

[33]

X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157.

[1]

Zhong Wan, Chaoming Hu, Zhanlu Yang. A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1157-1169. doi: 10.3934/dcdsb.2011.16.1157

[2]

Moulay Rchid Sidi Ammi, Ismail Jamiai. Finite difference and Legendre spectral method for a time-fractional diffusion-convection equation for image restoration. Discrete & Continuous Dynamical Systems - S, 2018, 11 (1) : 103-117. doi: 10.3934/dcdss.2018007

[3]

Nicolas Lermé, François Malgouyres, Dominique Hamoir, Emmanuelle Thouin. Bayesian image restoration for mosaic active imaging. Inverse Problems & Imaging, 2014, 8 (3) : 733-760. doi: 10.3934/ipi.2014.8.733

[4]

Yuhong Dai, Ya-xiang Yuan. Analysis of monotone gradient methods. Journal of Industrial & Management Optimization, 2005, 1 (2) : 181-192. doi: 10.3934/jimo.2005.1.181

[5]

Shenglong Hu, Zheng-Hai Huang, Hong-Yan Ni, Liqun Qi. Positive definiteness of Diffusion Kurtosis Imaging. Inverse Problems & Imaging, 2012, 6 (1) : 57-75. doi: 10.3934/ipi.2012.6.57

[6]

Antoni Buades, Bartomeu Coll, Jose-Luis Lisani, Catalina Sbert. Conditional image diffusion. Inverse Problems & Imaging, 2007, 1 (4) : 593-608. doi: 10.3934/ipi.2007.1.593

[7]

Jianjun Zhang, Yunyi Hu, James G. Nagy. A scaled gradient method for digital tomographic image reconstruction. Inverse Problems & Imaging, 2018, 12 (1) : 239-259. doi: 10.3934/ipi.2018010

[8]

Wanyou Cheng, Zixin Chen, Donghui Li. Nomonotone spectral gradient method for sparse recovery. Inverse Problems & Imaging, 2015, 9 (3) : 815-833. doi: 10.3934/ipi.2015.9.815

[9]

Timothy Blass, Rafael De La Llave, Enrico Valdinoci. A comparison principle for a Sobolev gradient semi-flow. Communications on Pure & Applied Analysis, 2011, 10 (1) : 69-91. doi: 10.3934/cpaa.2011.10.69

[10]

Robert I. McLachlan, G. R. W. Quispel. Discrete gradient methods have an energy conservation law. Discrete & Continuous Dynamical Systems - A, 2014, 34 (3) : 1099-1104. doi: 10.3934/dcds.2014.34.1099

[11]

Giacomo Frassoldati, Luca Zanni, Gaetano Zanghirati. New adaptive stepsize selections in gradient methods. Journal of Industrial & Management Optimization, 2008, 4 (2) : 299-312. doi: 10.3934/jimo.2008.4.299

[12]

Richard A. Norton, David I. McLaren, G. R. W. Quispel, Ari Stern, Antonella Zanna. Projection methods and discrete gradient methods for preserving first integrals of ODEs. Discrete & Continuous Dynamical Systems - A, 2015, 35 (5) : 2079-2098. doi: 10.3934/dcds.2015.35.2079

[13]

Jianqing Chen. Best constant of 3D Anisotropic Sobolev inequality and its applications. Communications on Pure & Applied Analysis, 2010, 9 (3) : 655-666. doi: 10.3934/cpaa.2010.9.655

[14]

Yunho Kim, Luminita A. Vese. Image recovery using functions of bounded variation and Sobolev spaces of negative differentiability. Inverse Problems & Imaging, 2009, 3 (1) : 43-68. doi: 10.3934/ipi.2009.3.43

[15]

Samuel Amstutz, Antonio André Novotny, Nicolas Van Goethem. Minimal partitions and image classification using a gradient-free perimeter approximation. Inverse Problems & Imaging, 2014, 8 (2) : 361-387. doi: 10.3934/ipi.2014.8.361

[16]

Juan C. Moreno, V. B. Surya Prasath, João C. Neves. Color image processing by vectorial total variation with gradient channels coupling. Inverse Problems & Imaging, 2016, 10 (2) : 461-497. doi: 10.3934/ipi.2016008

[17]

Nam-Yong Lee, Bradley J. Lucier. Preconditioned conjugate gradient method for boundary artifact-free image deblurring. Inverse Problems & Imaging, 2016, 10 (1) : 195-225. doi: 10.3934/ipi.2016.10.195

[18]

Tingting Wu, Yufei Yang, Huichao Jing. Two-step methods for image zooming using duality strategies. Numerical Algebra, Control & Optimization, 2014, 4 (3) : 209-225. doi: 10.3934/naco.2014.4.209

[19]

Laurent Amour, Jérémy Faupin. Inverse spectral results in Sobolev spaces for the AKNS operator with partial informations on the potentials. Inverse Problems & Imaging, 2013, 7 (4) : 1115-1122. doi: 10.3934/ipi.2013.7.1115

[20]

Gaohang Yu, Shanzhou Niu, Jianhua Ma. Multivariate spectral gradient projection method for nonlinear monotone equations with convex constraints. Journal of Industrial & Management Optimization, 2013, 9 (1) : 117-129. doi: 10.3934/jimo.2013.9.117

2016 Impact Factor: 1.094

Metrics

  • PDF downloads (0)
  • HTML views (0)
  • Cited by (4)

[Back to Top]