# American Institute of Mathematical Sciences

• Previous Article
Applications of a nonlinear optimization solver and two-stage comprehensive Denoising techniques for optimum underwater wideband sonar echolocation system
• JIMO Home
• This Issue
• Next Article
An outcome space algorithm for minimizing the product of two convex functions over a convex set
January  2013, 9(1): 227-241. doi: 10.3934/jimo.2013.9.227

## On the Levenberg-Marquardt methods for convex constrained nonlinear equations

 1 Department of Mathematics, and MOE-LSC, Shanghai Jiao Tong University, Shanghai 200240, China

Received  January 2012 Revised  July 2012 Published  December 2012

In this paper, both the constrained Levenberg-Marquardt method and the projected Levenberg-Marquardt method are presented for nonlinear equations $F(x)=0$ subject to $x\in X$, where $X$ is a nonempty closed convex set. The Levenberg-Marquardt parameter is taken as $\| F(x_k) \|_2^\delta$ with $\delta\in (0, 2]$. Under the local error bound condition which is weaker than nonsingularity, the methods are shown to have the same convergence rate, which includes not only the convergence results obtained in [12] for $\delta=2$ but also the results given in [7] for unconstrained nonlinear equations.
Citation: Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
##### References:
 [1] S. Bellavia, M. Macconi and B. Morini, An affine scaling trust-region approach to bound-constrained nonlinear systems,, Appl. Numer. Math., 44 (2003), 257. doi: 10.1016/S0168-9274(02)00170-8. [2] S. Bellavia and B. Morini, An interior global method for nonlinear systems with simple bounds,, Optim. Methods Software, 20 (2005), 1. [3] H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions,, Optim. Meth. Software, 17 (2002), 605. [4] J. E. Dennis and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations,", Prentice-Hall, (1983). [5] S. P. Dirkse, M. C. Ferris, MCPLIB: a collection of nonlinear mixed complementarity problems,, Optim. Meth. Software, 5 (1995), 319. doi: 10.1080/10556789508805619. [6] M. E. El-Hawary, "Optimal Power Flow: Solutions Techniques, Requirements, and Challenges,", IEEE Service Center, (1996). [7] J. Y. Fan and J. Y. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Computational Optimization and Applications, 34 (2006), 47. [8] J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23. doi: 10.1007/s00607-004-0083-1. [9] C. A. Floudas, P. M. Pardalos, C. S. Adjiman, W. R. Esposito, Z. H. Gumus, S. T. Harding, J. L. Klepeis, C. A. Meyer and C. A. Schweiger, "Handbook of Test Problems in Local and Global Optimization, Nonconvex Optimization and its Applications,", Vol. 33, (1999). [10] W. Hock and K. Schittkowski, "Test Examples for Nonlinear Programming Codes,", Lecture Notes in Economics and Mathematical Systems, (1981). [11] C. Kanzow, An active set-type Newton method for constrained nonlinear systems,, in, (2001), 179. [12] C. Kanzow, N. Yamashita and M. Fukushima, Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints,, Journal of Computational and Applied Mathematics, 173 (2005), 321. [13] C. T. Kelley, "Iterative Methods for Linear and Nonlinear Equations,", SIAM, (1995). [14] D. N. Kozakevich, J. M. Martinez and S. A. Santos, Solving nonlinear systems of equations with simple bounds,, Comput. Appl. Math., 16 (1997), 215. [15] K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164. [16] D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431. [17] K. Meintjes and A. P. Morgan, A methodology for solving chemical equilibrium systems,, Appl. Math. Comput., 22 (1987), 333. [18] K. Meintjes and A. P. Morgan, Chemical equilibrium systems as numerical test problems,, ACM Trans. Math. Software, 16 (1990), 143. [19] R. D. C. Monteiro and J. S. Pang, A potential reduction Newton method for constrained equations,, SIAM J. Optim., 9 (1999), 729. doi: 10.1137/S1052623497318980. [20] J. J. Moré, The LM algorithm: implementation and theory,, in, (1978), 105. [21] J. M. Ortega and W. C. Rheinboldt, "Iterative solution of Nonlinear Equations in Several Variables,", Academic Press, (1970). [22] L. Qi, X. J. Tong and D. H. Li, An active-set projected trust region algorithm for box constrained nonsmooth equations,, Journal of Optimization Theories and Applications, 120 (2004), 601. [23] M. Ulbrich, Nonmonotone trust-region methods for bound-constrained semismooth equations with applications to nonlinear mixed complementarity problems,, SIAM J. Optim., 11 (2001), 889. [24] T. Wang, R. D. C. Monteiro and J. S. Pang, An interior point potential reduction method for constrained equations,, Math. Programming, 74 (1996), 159. [25] A. J. Wood and B. F. Wollenberg, "Power Generation, Operation, and Control,", John Wiley and Sons, (1996). [26] N. Yamashita and M. Fukushima, On the rate of convergence of the LM method,, Computing (Supplement 15), (2001), 237. [27] Y. X. Yuan, Recent advances in numerical methods for nonlinear equations and nonlinear least sqaures,, Numerical Algebra, 1 (2011), 15.

show all references

##### References:
 [1] S. Bellavia, M. Macconi and B. Morini, An affine scaling trust-region approach to bound-constrained nonlinear systems,, Appl. Numer. Math., 44 (2003), 257. doi: 10.1016/S0168-9274(02)00170-8. [2] S. Bellavia and B. Morini, An interior global method for nonlinear systems with simple bounds,, Optim. Methods Software, 20 (2005), 1. [3] H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions,, Optim. Meth. Software, 17 (2002), 605. [4] J. E. Dennis and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations,", Prentice-Hall, (1983). [5] S. P. Dirkse, M. C. Ferris, MCPLIB: a collection of nonlinear mixed complementarity problems,, Optim. Meth. Software, 5 (1995), 319. doi: 10.1080/10556789508805619. [6] M. E. El-Hawary, "Optimal Power Flow: Solutions Techniques, Requirements, and Challenges,", IEEE Service Center, (1996). [7] J. Y. Fan and J. Y. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Computational Optimization and Applications, 34 (2006), 47. [8] J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23. doi: 10.1007/s00607-004-0083-1. [9] C. A. Floudas, P. M. Pardalos, C. S. Adjiman, W. R. Esposito, Z. H. Gumus, S. T. Harding, J. L. Klepeis, C. A. Meyer and C. A. Schweiger, "Handbook of Test Problems in Local and Global Optimization, Nonconvex Optimization and its Applications,", Vol. 33, (1999). [10] W. Hock and K. Schittkowski, "Test Examples for Nonlinear Programming Codes,", Lecture Notes in Economics and Mathematical Systems, (1981). [11] C. Kanzow, An active set-type Newton method for constrained nonlinear systems,, in, (2001), 179. [12] C. Kanzow, N. Yamashita and M. Fukushima, Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints,, Journal of Computational and Applied Mathematics, 173 (2005), 321. [13] C. T. Kelley, "Iterative Methods for Linear and Nonlinear Equations,", SIAM, (1995). [14] D. N. Kozakevich, J. M. Martinez and S. A. Santos, Solving nonlinear systems of equations with simple bounds,, Comput. Appl. Math., 16 (1997), 215. [15] K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164. [16] D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431. [17] K. Meintjes and A. P. Morgan, A methodology for solving chemical equilibrium systems,, Appl. Math. Comput., 22 (1987), 333. [18] K. Meintjes and A. P. Morgan, Chemical equilibrium systems as numerical test problems,, ACM Trans. Math. Software, 16 (1990), 143. [19] R. D. C. Monteiro and J. S. Pang, A potential reduction Newton method for constrained equations,, SIAM J. Optim., 9 (1999), 729. doi: 10.1137/S1052623497318980. [20] J. J. Moré, The LM algorithm: implementation and theory,, in, (1978), 105. [21] J. M. Ortega and W. C. Rheinboldt, "Iterative solution of Nonlinear Equations in Several Variables,", Academic Press, (1970). [22] L. Qi, X. J. Tong and D. H. Li, An active-set projected trust region algorithm for box constrained nonsmooth equations,, Journal of Optimization Theories and Applications, 120 (2004), 601. [23] M. Ulbrich, Nonmonotone trust-region methods for bound-constrained semismooth equations with applications to nonlinear mixed complementarity problems,, SIAM J. Optim., 11 (2001), 889. [24] T. Wang, R. D. C. Monteiro and J. S. Pang, An interior point potential reduction method for constrained equations,, Math. Programming, 74 (1996), 159. [25] A. J. Wood and B. F. Wollenberg, "Power Generation, Operation, and Control,", John Wiley and Sons, (1996). [26] N. Yamashita and M. Fukushima, On the rate of convergence of the LM method,, Computing (Supplement 15), (2001), 237. [27] Y. X. Yuan, Recent advances in numerical methods for nonlinear equations and nonlinear least sqaures,, Numerical Algebra, 1 (2011), 15.
 [1] Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223 [2] Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199 [3] Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25 [4] Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems & Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335 [5] Wen-ling Zhao, Dao-jin Song. A global error bound via the SQP method for constrained optimization problem. Journal of Industrial & Management Optimization, 2007, 3 (4) : 775-781. doi: 10.3934/jimo.2007.3.775 [6] Jinkui Liu, Shengjie Li. Multivariate spectral DY-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization, 2017, 13 (1) : 283-295. doi: 10.3934/jimo.2016017 [7] Xiaojiao Tong, Shuzi Zhou. A smoothing projected Newton-type method for semismooth equations with bound constraints. Journal of Industrial & Management Optimization, 2005, 1 (2) : 235-250. doi: 10.3934/jimo.2005.1.235 [8] Yongjian Yang, Zhiyou Wu, Fusheng Bai. A filled function method for constrained nonlinear integer programming. Journal of Industrial & Management Optimization, 2008, 4 (2) : 353-362. doi: 10.3934/jimo.2008.4.353 [9] Luchuan Ceng, Qamrul Hasan Ansari, Jen-Chih Yao. Extragradient-projection method for solving constrained convex minimization problems. Numerical Algebra, Control & Optimization, 2011, 1 (3) : 341-359. doi: 10.3934/naco.2011.1.341 [10] Chunming Tang, Jinbao Jian, Guoyin Li. A proximal-projection partial bundle method for convex constrained minimax problems. Journal of Industrial & Management Optimization, 2019, 15 (2) : 757-774. doi: 10.3934/jimo.2018069 [11] Yu-Ning Yang, Su Zhang. On linear convergence of projected gradient method for a class of affine rank minimization problems. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1507-1519. doi: 10.3934/jimo.2016.12.1507 [12] Changjun Yu, Kok Lay Teo, Liansheng Zhang, Yanqin Bai. On a refinement of the convergence analysis for the new exact penalty function method for continuous inequality constrained optimization problem. Journal of Industrial & Management Optimization, 2012, 8 (2) : 485-491. doi: 10.3934/jimo.2012.8.485 [13] Zhongwen Chen, Songqiang Qiu, Yujie Jiao. A penalty-free method for equality constrained optimization. Journal of Industrial & Management Optimization, 2013, 9 (2) : 391-409. doi: 10.3934/jimo.2013.9.391 [14] Liejune Shiau, Roland Glowinski. Operator splitting method for friction constrained dynamical systems. Conference Publications, 2005, 2005 (Special) : 806-815. doi: 10.3934/proc.2005.2005.806 [15] Boshi Tian, Xiaoqi Yang, Kaiwen Meng. An interior-point $l_{\frac{1}{2}}$-penalty method for inequality constrained nonlinear optimization. Journal of Industrial & Management Optimization, 2016, 12 (3) : 949-973. doi: 10.3934/jimo.2016.12.949 [16] Ming Yan, Lili Chang, Ningning Yan. Finite element method for constrained optimal control problems governed by nonlinear elliptic PDEs. Mathematical Control & Related Fields, 2012, 2 (2) : 183-194. doi: 10.3934/mcrf.2012.2.183 [17] Gaohang Yu, Shanzhou Niu, Jianhua Ma. Multivariate spectral gradient projection method for nonlinear monotone equations with convex constraints. Journal of Industrial & Management Optimization, 2013, 9 (1) : 117-129. doi: 10.3934/jimo.2013.9.117 [18] A. M. Bagirov, Moumita Ghosh, Dean Webb. A derivative-free method for linearly constrained nonsmooth optimization. Journal of Industrial & Management Optimization, 2006, 2 (3) : 319-338. doi: 10.3934/jimo.2006.2.319 [19] Fengmin Xu, Yanfei Wang. Recovery of seismic wavefields by an lq-norm constrained regularization method. Inverse Problems & Imaging, 2018, 12 (5) : 1157-1172. doi: 10.3934/ipi.2018048 [20] Li Wang, Yang Li, Liwei Zhang. A differential equation method for solving box constrained variational inequality problems. Journal of Industrial & Management Optimization, 2011, 7 (1) : 183-198. doi: 10.3934/jimo.2011.7.183

2018 Impact Factor: 1.025