Recent advances in numerical methods for nonlinear equations and
nonlinear least squares
Pages: 15  34,
Volume 1,
Issue 1,
March
2011
doi:10.3934/naco.2011.1.15 Abstract
References
Full text (445.3K)
Related Articles
YaXiang Yuan  State Key Laboratory of Scientific/Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing,Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Zhong Guan Cun Donglu 55, Beijing, 100190, China (email)
1 
J. Barzilai and J. M. Borwein, Two point step size gradient methods, IMA J. Numer. Anal., 8 (1988), 141148. 

2 
S. Bellavia, C. Cartis, N. I. M. Gould, B. Morini and Ph. L. Toint, Convergence of a regularized Euclidean residual algorithm for nonlinear leastsquares, SIAM J. Numer. Anal., 48 (2010), 129. 

3 
L. Bergamaschi, I. Moret and G. Zilli, Inexact quasiNewton methods for sparse systems of nonlinear equations, Future Generation Computer Systems, 18 (2001), 4153. 

4 
E. G. Birgin, N. Krejic and J. M. Martinez, Globally convergent inexact quasiNewton methods for solving nonlinear systems, Numerical Algorithms, 32 (2003), 249260. 

5 
A. Bouaricha and J. J. Moré, Impact of Partial Separability on LargeScale Optimization, Computational Optimization and Applications, 7 (1997), 2740. 

6 
M. H. Cheng, "QuasiNewton Type Methods for Solving LargeScale Problems,'' Ph.D. thesis, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 2010. 

7 
M. H. Cheng and Y. H. Dai, Sparse twosided rankone updates for nonlinear equations, Science in China, Ser. A., 53 (2010), 110. 

8 
A. R. Conn, N. J. M. Gould and Ph. L. Toint, "TrustRegion Methods," MPSSIAM Series on Optimization, SIAM, Philadelphia, 2000. 

9 
J. E. Dennis Jr. and J. J. Moré, QuasiNewton methods, motivation and theory, SIAM Review, 19 (1977), 4689. 

10 
J. E. Dennis and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations," SIAM, Philadelphia, 1993. 

11 
J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the LevenbergMarquardt method without nonsingularity assumption, Computing, 74 (2005), 2339. 

12 
J. Y. Fan and Y. X. Yuan, Regularized Newton method with correction for monotone nonlinear equations and its application, Report, AMSS, CAS, Beijing, 2010. 

13 
R. Fletcher, A model algorithm for composite NDO problem, Math. Program., 17 (1982), 6776. 

14 
R. Fletcher, "Practical Methods of Optimization," Second Edition, John Wiley and Sons, 1987. 

15 
R. Fletcher and C. Xu, Hybrid methods for nonlinear least squares, IMA J. Numerical Analysis, 7 (1987), 371389. 

16 
G. H. Golub and V. Pereyra, The differentiation of pseudoinverses and nonlinear least squares whose variables separable, SIAM J. Numer. Anal, 10 (1973), 413432. 

17 
G. H. Golub and V. Pereyra, Separable nonlinear least squares: the variable projection method and applications, Inverse Problems, 19 (2003), 126. 

18 
G. H. Golub and C. F. Van Loan, "Matrix Computations,'' 3rd Edition, Johns Hopkins University Press, Baltimore and London, 1996. 

19 
N. I. M. Gould, D. Orban and Ph. L. Toint, Numerical methods for largescale nonlinear optimization, Acta Numerica, 299361, 2005. 

20 
N. I. M. Gould and Ph. L. Toint, FILTRANE, a Fortran 95 filtertrustregion package for solving nonlinear leastsquares and nonlinear feasibility problems, ACM Transactions on Mathematical Software (TOMS), 33 (2007), 325. 

21 
A. Griewank, "Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation," Frontiers in Applied Mathematics, Vol 19, SIAM, Philadelphia, PA, 2000. 

22 
A. Griewank and A. Walther, On constrained optimization by adjoint based quasiNewton methods, Optimizaiton Methods and Software, 17 (2002), 869889. 

23 
L. Kaufman, A variable projection method for solving separable nonlinear least squares problems, BIT, 15 (1975), 4957. 

24 
C.T. Kelley, "Iterative Methods for Linear and Nonlinear Equations," SIAM, Philadelphia, 1995. 

25 
C. T. Kelley, "Solving Nonlinear Equations with Newton's Method," Fundamentals of Algorithms Series, SIAM, Philadelphia, 2003. 

26 
C. Kanzow, N. Yamashita and M. Fukushima, LevenbergMarquardt methods with storng local convergence properties for solving nonlinear equations with convex constraints, J. Comp. Appl. Math., 173 (2005), 321343. 

27 
W. La Cruz and M. Raydan, Nonmonotone spectral methods for largescale nonlinear sysstems, Optimization Methods and Software, 18 (2003), 583599. 

28 
W. La Cruz, J. M. Martinez and M. Raydan, Spectral residual method without gradient information for solving largescale nonlinear systems of equations, Math. Comp., 75 (2006), 14291448. 

29 
D. H. Li and M. Fukushima, A globally and superlinearly convergent GaussNewtonbased BFGS method for symmetric nonlinear equations, SIAM J. Numer. Anal., 37 (1999), 152172. 

30 
D. H. Li and M. Fukushima, A derivativefree line search and global convergence of Broydenlike method for nonlinear equations, Optimizaiton Methods and Software, 13 (2000), 181201. 

31 
X. Liu and Y. X. Yuan, On the separable nonlinear least squares problems, J. Comput. Math., 26 (2008), 390403. 

32 
J. M. Martinez, A quasiNewton method with modification of one column per iteration, Computing, 33 (1984), 353362. 

33 
E. Mizutani and J. W. Demmel, On structureexploiting trustregion regularized nonlinear least squares algorithms for neuralnetwork learning, Neural Networks, 16 (2003), 745753. 

34 
J. J. Moré, The LevenbergMarquardt algorithm: implementation and theory, in "Lecture Notes in Mathematics 630: Numerical Analysis"(eds. G.A. Watson), SpringerVerlag, Berlin, 1978, pp. 105116. 

35 
J. J. Moré, Recent developments in algorithms and software for trust region methods, in "Mathematical Programming: The State of the Art (SpringerVerlag, Berlin, 1983)"(eds. A. Bachem, M. Grötschel and B. Korte), pp. 258287. 

36 
J. J. Moré and D. C. Sorensen, Computing a trust region step, SIAM J. Sci. Statist. Compute., 4 (1983), 553572. 

37 
Yu. Nesterov, Modified GaussNewton scheme with worstcase grarantees for global performance, Optimization Methods and Sofware, 22 (2007), 469483. 

38 
J. Nocedal and S. J. Wright, "Numerical Optimization," Springer, New York, 1999. 

39 
J. M. Ortega and W. C. Rheinboldt, "Iterative Solution of Nonlinear Equations in Several Variables," Academic Press, New York and London, 1970. 

40 
M. J. D. Powell, A new algorithm for unconstrained optimization, in "Nonlinear Programming"(eds. J.B. Rosen, O.L. Mangasarian and K. Ritter), pp. 3165, Academic Press, New York, 1970. 

41 
M. J. D. Powell, A method for minimizing a sum of squares of nonlinear functions without calculating derivatives, The Computer J., 7 (1965), 303307. 

42 
M. J. D. Powell, A hybrid method for nonlinear equations, in "Numerical Methods for Nonlinear Algebraic Equations"(eds. P. Robinowtiz), Gordon and Breach, London, 1970, pp. 87114. 

43 
M. J. D. Powell, A fast algorithm for nonlinearly constrained optimization calculations, in "Numerical Analysis"(eds. G.A. Watson), Springer, Berlin, 1978, pp. 144157. 

44 
M. J. D. Powell and Y. X. Yuan, Conditions for superlinear convergence in $l_1$ and $l_\infty$ solutions of overdetermined nonlinear equations, IMA J. Numerical Analysis, 4 (1984), 241251. 

45 
A. Ruhe and P. Å. Wedin, Algorithms for separable nonlinear least squares problems, SIAM Review, 22 (1980), 318337. 

46 
Y. Saad, "Iterative Methods for Sparse Linear Systems," SIAM, New York, 2003. 

47 
S. Schlenkrich, A. Griewank and A. Walther, On the local convergence of adjoint Broyden methods, Math. Program., 121 (2010), 221247. 

48 
K. Schittkowski, Solving nonlinear least squares problems by a general purpose SQPmethod, in "Trends in Mathematical Optimization"(eds. K.H. Hoffmann, J.B. HiriartUrruty, C. Lemarechal, J. Zowe), International Series of Numerical Mathematics, Vol. 84, Birkhäuser, Boston, 1988, pp. 295309. 

49 
K. Schittkowski, "Numerical Data Fitting in Dynamical Systems," Applied Optimization Vol. 77, Kluwer Academic Publishers, Dordrecht, 2002. 

50 
L. K. Schubert, Modification of a quasiNewton method for nonlinear equations with a sparse Jacobian, Math. Comp., 24 (1970), 2730. 

51 
T. Steihaug, The conjugate gradient method and trust regions in large scale optimization, SIAM J. Numerical Analysis, 20 (1983), 626637. 

52 
J. Stoer and Y. X. Yuan, A subspace study on conjugate gradient algorithms, Z. Angew. Math. Mech., 75 (1995), 6977. 

53 
W. Y. Sun and Y. X. Yuan, "Optimization Theory and Mehtods: Nonlinear Programming," Springer Series on Optimization and Its Application, Vol. 1, Springer, 2006. 

54 
Ph. L. Toint, On sparse and sysmmetric matrix updating subject to a linear equation, Mathematics of Computation, 31 (1977), 954961. 

55 
Ph. L. Toint, Towards an efficient sparsity exploiting Newton method for minimization, in "Sparse Matrices and Their Uses"( eds. I. Duff), Academic Press, 1981, pp. 5788. 

56 
Ph. L. Toint, On large scale nonlinear least squares calculations, SIAM J. Sci. Stat. Comput., 8 (1987), 416435. 

57 
Z. H. Wang and Y. X. Yuan, A subspace implementation of quasiNewton trust region methods for unconstrained optimization, Numerische Mathematik, 104 (2006), 241269. 

58 
N. Yamashita and M. Fukushima, On the rate of convergence of the LevenbergMarquardt method , Computing, (Supplement) 15 (2001), 237249. 

59 
Y. X. Yuan, Trust region algorithms for nonlinear equations, Information, 1 (1998), 720. 

60 
Y. X. Yuan, On the truncated conjugate gradient method, Math. Program., 87 (2000), 561573. 

61 
Y. X. Yuan, Subspace techniques for nonlinear optimization, in "Some Topics in Industrial and Applied Mathematics" (eds. R. Jeltsch, D.Q. Li and I. H. Sloan), (Series in Contemporary Applied Mathematics CAM 8) Higher Education Press. Beijing, 2007, pp. 206218. 

62 
Y. X. Yuan, Subspace methods for large scale nonlinear equations and nonlinear least squares, Optimization and Engineering, 10 (2009), 207218. 

63 
H. C. Zhang, A. R. Conn and K. Scheinberg, A derivativefree algorithm for the leastsquares minimization, http://www.math.lsu.edu/ hozhang/zhcpaper.html. 

64 
H. C. Zhang, A. R. Conn and K. Scheinberg, On the local convergence of a derivativefree algorithm for leastsquares minimization, http://www.math.lsu.edu/ hozhang/zhcpaper.html. 

Go to top
