• Previous Article
    Delay-range dependent $H_\infty$ control for uncertain 2D-delayed systems
  • NACO Home
  • This Issue
  • Next Article
    Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization
2015, 5(1): 25-36. doi: 10.3934/naco.2015.5.25

On the global convergence of a parameter-adjusting Levenberg-Marquardt method

1. 

School of Mathematical Sciences, Dalian University of Technology, Dalian 116023, China, China

Received  January 2015 Revised  March 2015 Published  March 2015

The Levenberg-Marquardt (LM) method is a classical but popular method for solving nonlinear equations. Based on the trust region technique, we propose a parameter-adjusting LM (PALM) method, in which the LM parameter $\mu_k$ is self-adjusted at each iteration based on the ratio between actual reduction and predicted reduction. Under the level-bounded condition, we prove the global convergence of PALM. We also propose a modified parameter-adjusting LM (MPALM) method. Numerical results show that the two methods are very efficient.
Citation: Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447. doi: 10.1090/S0025-5718-2011-02496-8. Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173. doi: 10.1090/S0025-5718-2013-02752-4. Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47. doi: 10.1007/s10589-005-3074-z. Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23. doi: 10.1007/s00607-004-0083-1. Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164. Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431. Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258. Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17. doi: 10.1145/355934.355936. Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006). Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239. doi: 10.1007/978-3-7091-6217-0_18. Google Scholar

show all references

References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447. doi: 10.1090/S0025-5718-2011-02496-8. Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173. doi: 10.1090/S0025-5718-2013-02752-4. Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47. doi: 10.1007/s10589-005-3074-z. Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23. doi: 10.1007/s00607-004-0083-1. Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164. Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431. Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258. Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17. doi: 10.1145/355934.355936. Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006). Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239. doi: 10.1007/978-3-7091-6217-0_18. Google Scholar

[1]

Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223

[2]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199

[3]

Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227

[4]

Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems & Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335

[5]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013

[6]

Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial & Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030

[7]

Chunlin Hao, Xinwei Liu. Global convergence of an SQP algorithm for nonlinear optimization with overdetermined constraints. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 19-29. doi: 10.3934/naco.2012.2.19

[8]

Zhong Tan, Qiuju Xu, Huaqiao Wang. Global existence and convergence rates for the compressible magnetohydrodynamic equations without heat conductivity. Discrete & Continuous Dynamical Systems - A, 2015, 35 (10) : 5083-5105. doi: 10.3934/dcds.2015.35.5083

[9]

Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721

[10]

Zhiyou Wu, Fusheng Bai, Guoquan Li, Yongjian Yang. A new auxiliary function method for systems of nonlinear equations. Journal of Industrial & Management Optimization, 2015, 11 (2) : 345-364. doi: 10.3934/jimo.2015.11.345

[11]

Martin Burger, José A. Carrillo, Marie-Therese Wolfram. A mixed finite element method for nonlinear diffusion equations. Kinetic & Related Models, 2010, 3 (1) : 59-83. doi: 10.3934/krm.2010.3.59

[12]

Thierry Colin, Boniface Nkonga. Multiscale numerical method for nonlinear Maxwell equations. Discrete & Continuous Dynamical Systems - B, 2005, 5 (3) : 631-658. doi: 10.3934/dcdsb.2005.5.631

[13]

Carlos E. Kenig. The method of energy channels for nonlinear wave equations. Discrete & Continuous Dynamical Systems - A, 2019, 0 (0) : 1-15. doi: 10.3934/dcds.2019240

[14]

Walter Allegretto, Yanping Lin, Zhiyong Zhang. Convergence to convection-diffusion waves for solutions to dissipative nonlinear evolution equations. Conference Publications, 2009, 2009 (Special) : 11-23. doi: 10.3934/proc.2009.2009.11

[15]

José A. Carrillo, Jean Dolbeault, Ivan Gentil, Ansgar Jüngel. Entropy-energy inequalities and improved convergence rates for nonlinear parabolic equations. Discrete & Continuous Dynamical Systems - B, 2006, 6 (5) : 1027-1050. doi: 10.3934/dcdsb.2006.6.1027

[16]

Yang Li, Yonghong Ren, Yun Wang, Jian Gu. Convergence analysis of a nonlinear Lagrangian method for nonconvex semidefinite programming with subproblem inexactly solved. Journal of Industrial & Management Optimization, 2015, 11 (1) : 65-81. doi: 10.3934/jimo.2015.11.65

[17]

W. Wei, H. M. Yin. Global solvability for a singular nonlinear Maxwell's equations. Communications on Pure & Applied Analysis, 2005, 4 (2) : 431-444. doi: 10.3934/cpaa.2005.4.431

[18]

Daniela Giachetti, Maria Michaela Porzio. Global existence for nonlinear parabolic equations with a damping term. Communications on Pure & Applied Analysis, 2009, 8 (3) : 923-953. doi: 10.3934/cpaa.2009.8.923

[19]

Monica Conti, Elsa M. Marchini, V. Pata. Global attractors for nonlinear viscoelastic equations with memory. Communications on Pure & Applied Analysis, 2016, 15 (5) : 1893-1913. doi: 10.3934/cpaa.2016021

[20]

Zuzana Došlá, Mauro Marini, Serena Matucci. Global Kneser solutions to nonlinear equations with indefinite weight. Discrete & Continuous Dynamical Systems - B, 2018, 23 (8) : 3297-3308. doi: 10.3934/dcdsb.2018252

 Impact Factor: 

Metrics

  • PDF downloads (8)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]