• Previous Article
    Improved convergence properties of the Lin-Fukushima-Regularization method for mathematical programs with complementarity constraints
  • NACO Home
  • This Issue
  • Next Article
    A modified Fletcher-Reeves-Type derivative-free method for symmetric nonlinear equations
2011, 1(1): 61-69. doi: 10.3934/naco.2011.1.61

Convergence analysis of sparse quasi-Newton updates with positive definite matrix completion for two-dimensional functions

1. 

State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, P.O. Box 2719, Beijing 100080

2. 

Graduate School of Informatics, Kyoto University, Yoshida-honmachi, Sakyo-ku, Kyoto, 606-8501, Japan

Received  September 2010 Revised  September 2010 Published  February 2011

In this paper, we briefly review the extensions of quasi-Newton methods for large-scale optimization. Specially, based on the idea of maximum determinant positive definite matrix completion, Yamashita (2008) proposed a new sparse quasi-Newton update, called MCQN, for unconstrained optimization problems with sparse Hessian structures. In exchange of the relaxation of the secant equation, the MCQN update avoids solving difficult subproblems and overcomes the ill-conditioning of approximate Hessian matrices. A global convergence analysis is given in this paper for the MCQN update with Broyden's convex family assuming that the objective function is uniformly convex and its dimension is only two.
    This paper is dedicated to Professor Masao Fukushima on occasion of his 60th birthday.
Citation: Yuhong Dai, Nobuo Yamashita. Convergence analysis of sparse quasi-Newton updates with positive definite matrix completion for two-dimensional functions. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 61-69. doi: 10.3934/naco.2011.1.61
References:
[1]

R. Byrd and J. Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained optimization,, SIAM Journal on Numerical Analysis, 26 (1989), 727. doi: 10.1137/0726042.

[2]

R. Byrd, J. Nocedal and Y. Yuan, Global convergence of a class of quasi-Newton methods on convex problems,, SIAM Journal on Numerical Analysis, 24 (1987), 1171. doi: 10.1137/0724077.

[3]

M. H. Cheng and Y. H. Dai, A new sparse quasi-Newton update method,, Research report, (2010).

[4]

Y. H. Dai, Convergence properties of the BFGS algorithm,, SIAM Journal on Optimization, 13 (2003), 693. doi: 10.1137/S1052623401383455.

[5]

Y. H. Dai, A perfect example for the BFGS method,, Research report, (2010).

[6]

Y. H. Dai and N. Yamashita, Analysis of sparse quasi-Newton updates with positive definite matrix completion,, Research report, (2007).

[7]

J. E. Dennis, Jr. and J. J. Moré, Quasi-Newton method, motivation and theory,, SIAM Review, 19 (1977), 46. doi: 10.1137/1019005.

[8]

R. Fletcher, An optimal positive definite update for sparse Hessian matrices,, SIAM Journal on Optimization, 5 (1995), 192. doi: 10.1137/0805010.

[9]

M. Fukuda, M. Kojima, K. Murota and K. Nakata, Exploiting sparsity in semidefinite programming via matrix completion I: General frameworks,, SIAM Journal on Optimization, 11 (2000), 647. doi: 10.1137/S1052623400366218.

[10]

A. Griewank and Ph. L. Toint, On the unconstrained optimization of partially separable objective functions,, in, (1982), 301.

[11]

D. H. Li and M. Fukushima, On the global convergence of BFGS method for nonconvex unconstrained optimization problems,, SIAM Journal on Optimization, 11 (2001), 1054. doi: 10.1137/S1052623499354242.

[12]

D. H. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, Journal of Computational and Applied Mathematics, 129 (2001), 15. doi: 10.1016/S0377-0427(00)00540-9.

[13]

D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization,, Mathematical Programming, 45 (1989), 503. doi: 10.1007/BF01589116.

[14]

W. F. Mascarenhas, The BFGS algorithm with exact line searches fails for nonconvex functions,, Mathematical Programming, 99 (2004), 49. doi: 10.1007/s10107-003-0421-7.

[15]

J. Nocedal, Updating quasi-Newton matrices with limited storage,, Mathematics of Computation, 35 (1980), 773. doi: 10.1090/S0025-5718-1980-0572855-7.

[16]

M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches,, in, (1976), 53.

[17]

D. C. Sorensen, Collinear scaling and sequential estimation in sparse optimization algorithm,, Mathematical Programming Study, 18 (1982), 135.

[18]

P. L. Toint, On sparse and symmetric matrix updating subject to a linear equation,, Mathematics of Computation, 31 (1977), 954. doi: 10.1090/S0025-5718-1977-0455338-4.

[19]

N. Yamashita, Sparse quasi-Newton updates with positive definite matrix completion,, Mathematical Programming, 115 (2008), 1. doi: 10.1007/s10107-007-0137-1.

[20]

Y. Yuan, "Numerical Methods for Nonlinear Programming,", Shanghai Scientific & Technical Publishers, (1993).

show all references

References:
[1]

R. Byrd and J. Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained optimization,, SIAM Journal on Numerical Analysis, 26 (1989), 727. doi: 10.1137/0726042.

[2]

R. Byrd, J. Nocedal and Y. Yuan, Global convergence of a class of quasi-Newton methods on convex problems,, SIAM Journal on Numerical Analysis, 24 (1987), 1171. doi: 10.1137/0724077.

[3]

M. H. Cheng and Y. H. Dai, A new sparse quasi-Newton update method,, Research report, (2010).

[4]

Y. H. Dai, Convergence properties of the BFGS algorithm,, SIAM Journal on Optimization, 13 (2003), 693. doi: 10.1137/S1052623401383455.

[5]

Y. H. Dai, A perfect example for the BFGS method,, Research report, (2010).

[6]

Y. H. Dai and N. Yamashita, Analysis of sparse quasi-Newton updates with positive definite matrix completion,, Research report, (2007).

[7]

J. E. Dennis, Jr. and J. J. Moré, Quasi-Newton method, motivation and theory,, SIAM Review, 19 (1977), 46. doi: 10.1137/1019005.

[8]

R. Fletcher, An optimal positive definite update for sparse Hessian matrices,, SIAM Journal on Optimization, 5 (1995), 192. doi: 10.1137/0805010.

[9]

M. Fukuda, M. Kojima, K. Murota and K. Nakata, Exploiting sparsity in semidefinite programming via matrix completion I: General frameworks,, SIAM Journal on Optimization, 11 (2000), 647. doi: 10.1137/S1052623400366218.

[10]

A. Griewank and Ph. L. Toint, On the unconstrained optimization of partially separable objective functions,, in, (1982), 301.

[11]

D. H. Li and M. Fukushima, On the global convergence of BFGS method for nonconvex unconstrained optimization problems,, SIAM Journal on Optimization, 11 (2001), 1054. doi: 10.1137/S1052623499354242.

[12]

D. H. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, Journal of Computational and Applied Mathematics, 129 (2001), 15. doi: 10.1016/S0377-0427(00)00540-9.

[13]

D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization,, Mathematical Programming, 45 (1989), 503. doi: 10.1007/BF01589116.

[14]

W. F. Mascarenhas, The BFGS algorithm with exact line searches fails for nonconvex functions,, Mathematical Programming, 99 (2004), 49. doi: 10.1007/s10107-003-0421-7.

[15]

J. Nocedal, Updating quasi-Newton matrices with limited storage,, Mathematics of Computation, 35 (1980), 773. doi: 10.1090/S0025-5718-1980-0572855-7.

[16]

M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches,, in, (1976), 53.

[17]

D. C. Sorensen, Collinear scaling and sequential estimation in sparse optimization algorithm,, Mathematical Programming Study, 18 (1982), 135.

[18]

P. L. Toint, On sparse and symmetric matrix updating subject to a linear equation,, Mathematics of Computation, 31 (1977), 954. doi: 10.1090/S0025-5718-1977-0455338-4.

[19]

N. Yamashita, Sparse quasi-Newton updates with positive definite matrix completion,, Mathematical Programming, 115 (2008), 1. doi: 10.1007/s10107-007-0137-1.

[20]

Y. Yuan, "Numerical Methods for Nonlinear Programming,", Shanghai Scientific & Technical Publishers, (1993).

[1]

Honglan Zhu, Qin Ni, Meilan Zeng. A quasi-Newton trust region method based on a new fractional model. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 237-249. doi: 10.3934/naco.2015.5.237

[2]

Rouhollah Tavakoli, Hongchao Zhang. A nonmonotone spectral projected gradient method for large-scale topology optimization problems. Numerical Algebra, Control & Optimization, 2012, 2 (2) : 395-412. doi: 10.3934/naco.2012.2.395

[3]

Boris Kramer, John R. Singler. A POD projection method for large-scale algebraic Riccati equations. Numerical Algebra, Control & Optimization, 2016, 6 (4) : 413-435. doi: 10.3934/naco.2016018

[4]

Gaohang Yu. A derivative-free method for solving large-scale nonlinear systems of equations. Journal of Industrial & Management Optimization, 2010, 6 (1) : 149-160. doi: 10.3934/jimo.2010.6.149

[5]

Danuta Gaweł, Krzysztof Fujarewicz. On the sensitivity of feature ranked lists for large-scale biological data. Mathematical Biosciences & Engineering, 2013, 10 (3) : 667-690. doi: 10.3934/mbe.2013.10.667

[6]

Mahmut Çalik, Marcel Oliver. Weak solutions for generalized large-scale semigeostrophic equations. Communications on Pure & Applied Analysis, 2013, 12 (2) : 939-955. doi: 10.3934/cpaa.2013.12.939

[7]

Philippe Bonneton, Nicolas Bruneau, Bruno Castelle, Fabien Marche. Large-scale vorticity generation due to dissipating waves in the surf zone. Discrete & Continuous Dynamical Systems - B, 2010, 13 (4) : 729-738. doi: 10.3934/dcdsb.2010.13.729

[8]

Bo You, Chengkui Zhong, Fang Li. Pullback attractors for three dimensional non-autonomous planetary geostrophic viscous equations of large-scale ocean circulation. Discrete & Continuous Dynamical Systems - B, 2014, 19 (4) : 1213-1226. doi: 10.3934/dcdsb.2014.19.1213

[9]

Masataka Kato, Hiroyuki Masuyama, Shoji Kasahara, Yutaka Takahashi. Effect of energy-saving server scheduling on power consumption for large-scale data centers. Journal of Industrial & Management Optimization, 2016, 12 (2) : 667-685. doi: 10.3934/jimo.2016.12.667

[10]

Tsuguhito Hirai, Hiroyuki Masuyama, Shoji Kasahara, Yutaka Takahashi. Performance analysis of large-scale parallel-distributed processing with backup tasks for cloud computing. Journal of Industrial & Management Optimization, 2014, 10 (1) : 113-129. doi: 10.3934/jimo.2014.10.113

[11]

Suli Zou, Zhongjing Ma, Xiangdong Liu. Auction games for coordination of large-scale elastic loads in deregulated electricity markets. Journal of Industrial & Management Optimization, 2016, 12 (3) : 833-850. doi: 10.3934/jimo.2016.12.833

[12]

Xiantao Xiao, Liwei Zhang, Jianzhong Zhang. On convergence of augmented Lagrangian method for inverse semi-definite quadratic programming problems. Journal of Industrial & Management Optimization, 2009, 5 (2) : 319-339. doi: 10.3934/jimo.2009.5.319

[13]

Jiuping Xu, Pei Wei. Production-distribution planning of construction supply chain management under fuzzy random environment for large-scale construction projects. Journal of Industrial & Management Optimization, 2013, 9 (1) : 31-56. doi: 10.3934/jimo.2013.9.31

[14]

Peter Benner, Ryan Lowe, Matthias Voigt. $\mathcal{L}_{∞}$-norm computation for large-scale descriptor systems using structured iterative eigensolvers. Numerical Algebra, Control & Optimization, 2018, 8 (1) : 119-133. doi: 10.3934/naco.2018007

[15]

Jérémi Dardé. Iterated quasi-reversibility method applied to elliptic and parabolic data completion problems. Inverse Problems & Imaging, 2016, 10 (2) : 379-407. doi: 10.3934/ipi.2016005

[16]

El-Sayed M.E. Mostafa. A nonlinear conjugate gradient method for a special class of matrix optimization problems. Journal of Industrial & Management Optimization, 2014, 10 (3) : 883-903. doi: 10.3934/jimo.2014.10.883

[17]

Yi Yang, Jianwei Ma, Stanley Osher. Seismic data reconstruction via matrix completion. Inverse Problems & Imaging, 2013, 7 (4) : 1379-1392. doi: 10.3934/ipi.2013.7.1379

[18]

Kuo-Hsiung Wang, Chuen-Wen Liao, Tseng-Chang Yen. Cost analysis of the M/M/R machine repair problem with second optional repair: Newton-Quasi method. Journal of Industrial & Management Optimization, 2010, 6 (1) : 197-207. doi: 10.3934/jimo.2010.6.197

[19]

Cheng-Dar Liou. Note on "Cost analysis of the M/M/R machine repair problem with second optional repair: Newton-Quasi method". Journal of Industrial & Management Optimization, 2012, 8 (3) : 727-732. doi: 10.3934/jimo.2012.8.727

[20]

Chien-Wen Chao, Shu-Cherng Fang, Ching-Jong Liao. A tropical cyclone-based method for global optimization. Journal of Industrial & Management Optimization, 2012, 8 (1) : 103-115. doi: 10.3934/jimo.2012.8.103

 Impact Factor: 

Metrics

  • PDF downloads (1)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]