• Previous Article
    Mathematical imaging using electric or magnetic nanoparticles as contrast agents
  • IPI Home
  • This Issue
  • Next Article
    A fast direct imaging method for the inverse obstacle scattering problem with nonlinear point scatterers
June 2018, 12(3): 607-634. doi: 10.3934/ipi.2018026

Morozov principle for Kullback-Leibler residual term and Poisson noise

CREATIS, CNRS UMR 5220, INSERM U1044, INSA de Lyon, Université de Lyon 1, Université de Lyon, 69621, Villeurbanne Cedex, France

Received  June 2017 Revised  November 2017 Published  March 2018

We study the properties of a regularization method for inverse problems corrupted by Poisson noise with Kullback-Leibler divergence as data term. The regularization parameter is chosen according to a Morozov type principle. We show that this method of choice of the parameter is well-defined. This a posteriori choice leads to a convergent regularization method. Convergences rates are obtained for this a posteriori choice of the regularization parameter when some source condition is satisfied.

Citation: Bruno Sixou, Tom Hohweiller, Nicolas Ducros. Morozov principle for Kullback-Leibler residual term and Poisson noise. Inverse Problems & Imaging, 2018, 12 (3) : 607-634. doi: 10.3934/ipi.2018026
References:
[1]

V. AlbaniA. De Cezaro and J. P. Zubelli, On the choice of the Tikhonov regularization parameter and the discretization level: A discrepancy-based strategy, Inverse Problems and Imaging, 10 (2016), 1-25. doi: 10.3934/ipi.2016.10.1.

[2]

A. Antoniadis and J. Bigot, Poisson inverse problems, Ann. Stat., 34 (2006), 2132-2158. doi: 10.1214/009053606000000687.

[3]

S. Anzengruber and R. Ramlau, Morozov discrepancy principle for Tikhonov-type functionals with nonlinear operators, Inverse Problems, 26 (2010), 025001, 17pp.

[4]

S. Anzengruber and R. Ramlau, Convergence rates for Morozov's discrepancy principle using variational inequalities, Inverse Problems, 27 (2010), 105007, 18pp.

[5]

A. BanerjeeS. MeruguI. Dhilon and J. Ghosh, Clustering with Bregman divergences, Journal of Machine Learning Research, 6 (2005), 1705-1749.

[6]

J. M. Bardsley and J. Goldes, Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation, Inverse Problems, 25 (2009), 095005, 18pp.

[7]

J. M. Bardsley, A theoretical framework for the regularization of Poisson Likelihood estimation problems, Inverse Problems and Imaging, 4 (2010), 11-17. doi: 10.3934/ipi.2010.4.11.

[8]

0. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Publishers, 1978.

[9]

M. BerteroP. BoccacciG. TalentiR. Zanella and L. Zanni, A discrepancy principle for Poisson data, Inverse Problems, 26 (2010), 105004, 20 pp.

[10]

G. L. BesneraisJ. F. Bercher and G. Demoment, A new look at entropy for solving linear inverse problems, IEEE Transactions on Information Theory, 45 (1999), 1566-1578. doi: 10.1109/18.771159.

[11]

T. Bonesky, Morozov's discrepancy principle and Tikhonov-type functionals, Inverse Problems, 25 (2009), 015015, 11pp.

[12]

J. M. Borwein and A. S. Lewis, Convergences of best entropy estimates, SIAM Journal on Optimization, 1 (1991), 191-205. doi: 10.1137/0801014.

[13]

L. M. Bregman, The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming, USSR Computational Mathematics and Mathematical Physics, 7 (1967), 200-217.

[14]

M. Burger and S. Osher, Convergence rates of convex variational reglarization, Inverse Problems, 20 (2004), 1411-1421. doi: 10.1088/0266-5611/20/5/005.

[15]

C. Byrne, Iterative Optimization in Inverse Problems, Taylor and Francis, 2014.

[16]

C. Byrne and P. Eggermont, EM algorithms, Handbook of Mathematical Methods in Imaging, Vol. 1, 2, 3,305-388, Springer, New York, 2015.

[17]

L. Cavalier and J. Y. Koo, Poisson intensity estimation for tomographic data using a wavelet shrinkage approach, IEEE Trans. on Information Theory, 48 (2002), 2794-2802. doi: 10.1109/TIT.2002.802632.

[18]

A. Das Gupta, On a Differential Equation and one Step Recursion for Poisson Moments, Purdue University, Technical Report 98-02.

[19]

N. DucrosJ. F. P. AbascalB. SixouS. Rit and F. Peyrin, Regularization of nonlinear decomposition of spectral X-ray projection images, Med.Phys., 44 (2017), 174-187.

[20]

P. P. B. Eggermont, Maximum entropy regularization for Fredholm integral equations of the first kind, SIAM Journal on Mathematical Analysis, 24 (1993), 1557-1576. doi: 10.1137/0524088.

[21]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers: Dordrecht, 1996.

[22]

F. FavatiG. LottiO. Menchi and F. Menchi, Performance analysis of maximum likekihood methods for regularization problems with nonnegativity constraints, Inverse Problems, 26 (2010), 085013, 18pp.

[23]

I. M. Gelfand and G. E. Shilov, Generalized Functions, Academic Press, San Diego, 1964.

[24]

R. M. Gray, Probability, Random Processes and Ergodic Properties, Second edition. Springer, Dordrecht, 2009.

[25]

G. Grubb, Distributions and Operators, Springer-Verlag, New York, 2009.

[26]

T. Hohage and F. Werner, Inhomogeneous Poisson intensity estimation via information projections onto wavelet subspaces, Inverse Problems, 32 (2016), 093001, 56pp.

[27]

K. Ito, Fundations of Stochastic Differential Equations in Infinite Dimensional Spaces, SIAM, 1984.

[28]

J. Y. Koo and W. C. Kim, Inhomogeneous Poisson intensity estimation via information projections onto wavelet subspaces, Journal of Korean Statistical Society, 31 (2002), 343-357.

[29]

S. Kullback and R. A. Leibler, On information and sufficiency, Annals of Mathematical Statistics, 22 (1951), 79-86. doi: 10.1214/aoms/1177729694.

[30]

Y. Long and J. A. Fessler, Multi-material decomposition using statistical image reconstruction for spectral CT, Trans. Med. Imaging, 33 (2014), 1614-1626.

[31]

R. D. Nowak and E. D. Kolaczyk, A Bayesian multiscale framework for Poisson inverse problems, IEEE Trans. on Information Theory, 46 (2000), 1811-1825. doi: 10.1109/18.857793.

[32]

M. H. Neumann, Absolute regularity and ergodicity of Poisson count processes, Bernoulli, 17 (2011), 1268-1284. doi: 10.3150/10-BEJ313.

[33]

J. M. Ollinger and J. A. Fessler, Positon-emission tomography, IEEE Signal Processing Magazine, 14 (1997), 43-55.

[34]

C. Pöschl, Tikhonov Regularization with General Residual Term, PhD Thesis, Univeristat Innsbruck, 2008.

[35]

E. Resmerita and R. S. Anderssen, Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problem, Mathematical Methods in the Applied sciences, 30 (2007), 1527-1544. doi: 10.1002/mma.855.

[36]

E. Resmerita, Regularization of ill-posed inverse problems in Banach spaces: Convergence rates, Inverse Problems, 21 (2005), 1301-1314. doi: 10.1088/0266-5611/21/4/007.

[37]

P. Reynaud-Bouret, Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities, Probab. Theory Relat. Fields, 126 (2003), 103-153. doi: 10.1007/s00440-003-0259-1.

[38]

R. J. Santos and A. R. De Pierro, A new parameters choice method for ill-posed problem with Poisson data and its application to emission tomographic imaging, International Journal of Tomography and Statistics, 11 (2009), 33-52.

[39]

O. Scherzer, M. Grassmair, H. Grossauer, M. Haltmaier and F. Lenzen, Variational Methods in Imaging, Springer Verlag, New York, 2009.

[40]

J.P. SchlomkaE. RoesslR. DorscheidS. DillG. MartensT. IstelC. BumerC. HerrmannR. SteadmanG. ZeitlerA. Livne and R. Proska, Experimental feasiblitiy of multi-energy photon-counting k-edge imaging in pre-clinical computed tomography, Physics in Medicine and Biology, 53 (2008), p4031.

[41]

L. Schwarz, Theorie des Distributions, Hermann, Paris, 1966.

[42]

Spray tollbox, https://www.creatis.insa-lyon.fr/~ducros/WebPage/spray.html

[43]

J. L. Stark and F. Murtagh, Astronomical Image and Data Analysis, Springer Verlag, New York, 2006.

[44]

A. N. Tikhonov and V. Y. Arsenin, Solutions to Ill-Posed Problems, Winston-Wiley, New York, 1977.

[45]

H. Wang and P. C. Miller, Scaled heavy-ball acceleration of the Richardson-Lucy algorithm for 3D microscopy image restoration, IEEE Transactions on Image Processing, 23 (2014), 848-854. doi: 10.1109/TIP.2013.2291324.

[46]

F. Werner and T. Hohage, Convergence rate in expectation ofr Tikhonov-type regularization of inverse problems with Poisson data, Inverse Problems, 28 (2012), 104004, 15pp.

[47]

F. Werner, Inverse problems with Poisson data: Tikhonov-type Regularization and Iteratively Regularized Newton Methods, PhD thesis, University of Göttingen, 2012.

[48]

R. ZanellaP. BoccacciL. Zani and M. Bertero, Efficient gradient projection methods for edge-preserving removal of poisson noise, Inverse Problems, 25 (2009), 045010, 24pp.

show all references

References:
[1]

V. AlbaniA. De Cezaro and J. P. Zubelli, On the choice of the Tikhonov regularization parameter and the discretization level: A discrepancy-based strategy, Inverse Problems and Imaging, 10 (2016), 1-25. doi: 10.3934/ipi.2016.10.1.

[2]

A. Antoniadis and J. Bigot, Poisson inverse problems, Ann. Stat., 34 (2006), 2132-2158. doi: 10.1214/009053606000000687.

[3]

S. Anzengruber and R. Ramlau, Morozov discrepancy principle for Tikhonov-type functionals with nonlinear operators, Inverse Problems, 26 (2010), 025001, 17pp.

[4]

S. Anzengruber and R. Ramlau, Convergence rates for Morozov's discrepancy principle using variational inequalities, Inverse Problems, 27 (2010), 105007, 18pp.

[5]

A. BanerjeeS. MeruguI. Dhilon and J. Ghosh, Clustering with Bregman divergences, Journal of Machine Learning Research, 6 (2005), 1705-1749.

[6]

J. M. Bardsley and J. Goldes, Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation, Inverse Problems, 25 (2009), 095005, 18pp.

[7]

J. M. Bardsley, A theoretical framework for the regularization of Poisson Likelihood estimation problems, Inverse Problems and Imaging, 4 (2010), 11-17. doi: 10.3934/ipi.2010.4.11.

[8]

0. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory, Wiley Publishers, 1978.

[9]

M. BerteroP. BoccacciG. TalentiR. Zanella and L. Zanni, A discrepancy principle for Poisson data, Inverse Problems, 26 (2010), 105004, 20 pp.

[10]

G. L. BesneraisJ. F. Bercher and G. Demoment, A new look at entropy for solving linear inverse problems, IEEE Transactions on Information Theory, 45 (1999), 1566-1578. doi: 10.1109/18.771159.

[11]

T. Bonesky, Morozov's discrepancy principle and Tikhonov-type functionals, Inverse Problems, 25 (2009), 015015, 11pp.

[12]

J. M. Borwein and A. S. Lewis, Convergences of best entropy estimates, SIAM Journal on Optimization, 1 (1991), 191-205. doi: 10.1137/0801014.

[13]

L. M. Bregman, The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming, USSR Computational Mathematics and Mathematical Physics, 7 (1967), 200-217.

[14]

M. Burger and S. Osher, Convergence rates of convex variational reglarization, Inverse Problems, 20 (2004), 1411-1421. doi: 10.1088/0266-5611/20/5/005.

[15]

C. Byrne, Iterative Optimization in Inverse Problems, Taylor and Francis, 2014.

[16]

C. Byrne and P. Eggermont, EM algorithms, Handbook of Mathematical Methods in Imaging, Vol. 1, 2, 3,305-388, Springer, New York, 2015.

[17]

L. Cavalier and J. Y. Koo, Poisson intensity estimation for tomographic data using a wavelet shrinkage approach, IEEE Trans. on Information Theory, 48 (2002), 2794-2802. doi: 10.1109/TIT.2002.802632.

[18]

A. Das Gupta, On a Differential Equation and one Step Recursion for Poisson Moments, Purdue University, Technical Report 98-02.

[19]

N. DucrosJ. F. P. AbascalB. SixouS. Rit and F. Peyrin, Regularization of nonlinear decomposition of spectral X-ray projection images, Med.Phys., 44 (2017), 174-187.

[20]

P. P. B. Eggermont, Maximum entropy regularization for Fredholm integral equations of the first kind, SIAM Journal on Mathematical Analysis, 24 (1993), 1557-1576. doi: 10.1137/0524088.

[21]

H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers: Dordrecht, 1996.

[22]

F. FavatiG. LottiO. Menchi and F. Menchi, Performance analysis of maximum likekihood methods for regularization problems with nonnegativity constraints, Inverse Problems, 26 (2010), 085013, 18pp.

[23]

I. M. Gelfand and G. E. Shilov, Generalized Functions, Academic Press, San Diego, 1964.

[24]

R. M. Gray, Probability, Random Processes and Ergodic Properties, Second edition. Springer, Dordrecht, 2009.

[25]

G. Grubb, Distributions and Operators, Springer-Verlag, New York, 2009.

[26]

T. Hohage and F. Werner, Inhomogeneous Poisson intensity estimation via information projections onto wavelet subspaces, Inverse Problems, 32 (2016), 093001, 56pp.

[27]

K. Ito, Fundations of Stochastic Differential Equations in Infinite Dimensional Spaces, SIAM, 1984.

[28]

J. Y. Koo and W. C. Kim, Inhomogeneous Poisson intensity estimation via information projections onto wavelet subspaces, Journal of Korean Statistical Society, 31 (2002), 343-357.

[29]

S. Kullback and R. A. Leibler, On information and sufficiency, Annals of Mathematical Statistics, 22 (1951), 79-86. doi: 10.1214/aoms/1177729694.

[30]

Y. Long and J. A. Fessler, Multi-material decomposition using statistical image reconstruction for spectral CT, Trans. Med. Imaging, 33 (2014), 1614-1626.

[31]

R. D. Nowak and E. D. Kolaczyk, A Bayesian multiscale framework for Poisson inverse problems, IEEE Trans. on Information Theory, 46 (2000), 1811-1825. doi: 10.1109/18.857793.

[32]

M. H. Neumann, Absolute regularity and ergodicity of Poisson count processes, Bernoulli, 17 (2011), 1268-1284. doi: 10.3150/10-BEJ313.

[33]

J. M. Ollinger and J. A. Fessler, Positon-emission tomography, IEEE Signal Processing Magazine, 14 (1997), 43-55.

[34]

C. Pöschl, Tikhonov Regularization with General Residual Term, PhD Thesis, Univeristat Innsbruck, 2008.

[35]

E. Resmerita and R. S. Anderssen, Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problem, Mathematical Methods in the Applied sciences, 30 (2007), 1527-1544. doi: 10.1002/mma.855.

[36]

E. Resmerita, Regularization of ill-posed inverse problems in Banach spaces: Convergence rates, Inverse Problems, 21 (2005), 1301-1314. doi: 10.1088/0266-5611/21/4/007.

[37]

P. Reynaud-Bouret, Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities, Probab. Theory Relat. Fields, 126 (2003), 103-153. doi: 10.1007/s00440-003-0259-1.

[38]

R. J. Santos and A. R. De Pierro, A new parameters choice method for ill-posed problem with Poisson data and its application to emission tomographic imaging, International Journal of Tomography and Statistics, 11 (2009), 33-52.

[39]

O. Scherzer, M. Grassmair, H. Grossauer, M. Haltmaier and F. Lenzen, Variational Methods in Imaging, Springer Verlag, New York, 2009.

[40]

J.P. SchlomkaE. RoesslR. DorscheidS. DillG. MartensT. IstelC. BumerC. HerrmannR. SteadmanG. ZeitlerA. Livne and R. Proska, Experimental feasiblitiy of multi-energy photon-counting k-edge imaging in pre-clinical computed tomography, Physics in Medicine and Biology, 53 (2008), p4031.

[41]

L. Schwarz, Theorie des Distributions, Hermann, Paris, 1966.

[42]

Spray tollbox, https://www.creatis.insa-lyon.fr/~ducros/WebPage/spray.html

[43]

J. L. Stark and F. Murtagh, Astronomical Image and Data Analysis, Springer Verlag, New York, 2006.

[44]

A. N. Tikhonov and V. Y. Arsenin, Solutions to Ill-Posed Problems, Winston-Wiley, New York, 1977.

[45]

H. Wang and P. C. Miller, Scaled heavy-ball acceleration of the Richardson-Lucy algorithm for 3D microscopy image restoration, IEEE Transactions on Image Processing, 23 (2014), 848-854. doi: 10.1109/TIP.2013.2291324.

[46]

F. Werner and T. Hohage, Convergence rate in expectation ofr Tikhonov-type regularization of inverse problems with Poisson data, Inverse Problems, 28 (2012), 104004, 15pp.

[47]

F. Werner, Inverse problems with Poisson data: Tikhonov-type Regularization and Iteratively Regularized Newton Methods, PhD thesis, University of Göttingen, 2012.

[48]

R. ZanellaP. BoccacciL. Zani and M. Bertero, Efficient gradient projection methods for edge-preserving removal of poisson noise, Inverse Problems, 25 (2009), 045010, 24pp.

Figure 1.  Evolution of the Kullback-Leibler functional for $I = 3 $ and $P = 6820$. The value $m/2$ is displayed for comparison
Figure 2.  Maps of the reconstructed bone and soft tissues projected mass for $I = 3 $ and $P = 6820$
Figure 3.  Evolution of the Kullback-Leibler functional for $I = 10 $ and $P = 1737984$.The value $m/2$ is displayed for comparison
Figure 4.  Maps of the reconstructed bone and soft tissues projected mass for $I = 10 $ and $P = 1737984$
Figure 5.  Evolution of the Kullback-Leibler functional for $I = 3 $ and $P = 108624$. The value $m/2$ is displayed for comparison
Figure 6.  Maps of the reconstructed bone and soft tissues projected mass for $I = 3$ and $P = 108624$
[1]

Jaan Janno, Kairi Kasemets. A positivity principle for parabolic integro-differential equations and inverse problems with final overdetermination. Inverse Problems & Imaging, 2009, 3 (1) : 17-41. doi: 10.3934/ipi.2009.3.17

[2]

Raphaël Danchin, Piotr B. Mucha. Divergence. Discrete & Continuous Dynamical Systems - S, 2013, 6 (5) : 1163-1172. doi: 10.3934/dcdss.2013.6.1163

[3]

Gary Lieberman. Nonlocal problems for quasilinear parabolic equations in divergence form. Conference Publications, 2003, 2003 (Special) : 563-570. doi: 10.3934/proc.2003.2003.563

[4]

Maria Rosaria Lancia, Valerio Regis Durante, Paola Vernole. Asymptotics for Venttsel' problems for operators in non divergence form in irregular domains. Discrete & Continuous Dynamical Systems - S, 2016, 9 (5) : 1493-1520. doi: 10.3934/dcdss.2016060

[5]

Colin Guillarmou, Antônio Sá Barreto. Inverse problems for Einstein manifolds. Inverse Problems & Imaging, 2009, 3 (1) : 1-15. doi: 10.3934/ipi.2009.3.1

[6]

Sergei Avdonin, Pavel Kurasov. Inverse problems for quantum trees. Inverse Problems & Imaging, 2008, 2 (1) : 1-21. doi: 10.3934/ipi.2008.2.1

[7]

Maciej Zworski. A remark on inverse problems for resonances. Inverse Problems & Imaging, 2007, 1 (1) : 225-227. doi: 10.3934/ipi.2007.1.225

[8]

Janne M.J. Huttunen, J. P. Kaipio. Approximation errors in nonstationary inverse problems. Inverse Problems & Imaging, 2007, 1 (1) : 77-93. doi: 10.3934/ipi.2007.1.77

[9]

Masoumeh Dashti, Stephen Harris, Andrew Stuart. Besov priors for Bayesian inverse problems. Inverse Problems & Imaging, 2012, 6 (2) : 183-200. doi: 10.3934/ipi.2012.6.183

[10]

Xiaosheng Li, Gunther Uhlmann. Inverse problems with partial data in a slab. Inverse Problems & Imaging, 2010, 4 (3) : 449-462. doi: 10.3934/ipi.2010.4.449

[11]

Zaidong Zhan, Shuping Chen, Wei Wei. A unified theory of maximum principle for continuous and discrete time optimal control problems. Mathematical Control & Related Fields, 2012, 2 (2) : 195-215. doi: 10.3934/mcrf.2012.2.195

[12]

François Monard, Guillaume Bal. Inverse diffusion problems with redundant internal information. Inverse Problems & Imaging, 2012, 6 (2) : 289-313. doi: 10.3934/ipi.2012.6.289

[13]

Gabriel Peyré, Sébastien Bougleux, Laurent Cohen. Non-local regularization of inverse problems. Inverse Problems & Imaging, 2011, 5 (2) : 511-530. doi: 10.3934/ipi.2011.5.511

[14]

Simopekka Vänskä. Stationary waves method for inverse scattering problems. Inverse Problems & Imaging, 2008, 2 (4) : 577-586. doi: 10.3934/ipi.2008.2.577

[15]

Sergei Avdonin, Fritz Gesztesy, Konstantin A. Makarov. Spectral estimation and inverse initial boundary value problems. Inverse Problems & Imaging, 2010, 4 (1) : 1-9. doi: 10.3934/ipi.2010.4.1

[16]

Johnathan M. Bardsley. Gaussian Markov random field priors for inverse problems. Inverse Problems & Imaging, 2013, 7 (2) : 397-416. doi: 10.3934/ipi.2013.7.397

[17]

Anna Doubova, Enrique Fernández-Cara. Some geometric inverse problems for the linear wave equation. Inverse Problems & Imaging, 2015, 9 (2) : 371-393. doi: 10.3934/ipi.2015.9.371

[18]

Yannick Fischer, Benjamin Marteau, Yannick Privat. Some inverse problems around the tokamak Tore Supra. Communications on Pure & Applied Analysis, 2012, 11 (6) : 2327-2349. doi: 10.3934/cpaa.2012.11.2327

[19]

Kenrick Bingham, Yaroslav Kurylev, Matti Lassas, Samuli Siltanen. Iterative time-reversal control for inverse problems. Inverse Problems & Imaging, 2008, 2 (1) : 63-81. doi: 10.3934/ipi.2008.2.63

[20]

Martin Hanke, William Rundell. On rational approximation methods for inverse source problems. Inverse Problems & Imaging, 2011, 5 (1) : 185-202. doi: 10.3934/ipi.2011.5.185

2016 Impact Factor: 1.094

Metrics

  • PDF downloads (44)
  • HTML views (76)
  • Cited by (0)

Other articles
by authors

[Back to Top]