American Institute of Mathematical Sciences

Advanced
2005, 1(2): 181-192. doi: 10.3934/jimo.2005.1.181

Analysis of monotone gradient methods

Yuhong Dai 1, and  Ya-xiang Yuan 2,

1. 

State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, P.O. Box 2719, Beijing 100080, China
2. 

State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering computing, Academy of Mathematics and System Sciences, Chinese Academy of Sciences, P. O. Box 2719, Beijing 100080, P. R., China

Received  August 2004 Revised  December 2004 Published  April 2005

The gradient method is one simple method in nonlinear optimization. In this paper, we give a brief review on monotone gradient methods and study their numerical properties by introducing a new technique of long-term observation. We find that, one monotone gradient algorithm which is proposed by Yuan recently shares with the Barzilai-Borwein (BB) method the property that the gradient components with respect to the eigenvectors of the function Hessian are decreasing together. This might partly explain why this algorithm by Yuan is comparable to the BB method in practice. Some examples are also provided showing that the alternate minimization algorithm and the other algorithm by Yuan may fall into cycles. Some more efficient gradient algorithms are provided. Particularly, one of them is monotone and performs better than the BB method in the quadratic case.
Keywords: monotone, cycle., nonmonotone, gradient method, strictly convex quadratics.
Mathematics Subject Classification: 65k05, 90c0.
Citation: Yuhong Dai, Ya-xiang Yuan. Analysis of monotone gradient methods. Journal of Industrial & Management Optimization, 2005, 1 (2) : 181-192. doi: 10.3934/jimo.2005.1.181
[1]

Gaohang Yu, Shanzhou Niu, Jianhua Ma. Multivariate spectral gradient projection method for nonlinear monotone equations with convex constraints. Journal of Industrial & Management Optimization, 2013, 9 (1) : 117-129. doi: 10.3934/jimo.2013.9.117
[2]

Yigui Ou, Yuanwen Liu. A memory gradient method based on the nonmonotone technique. Journal of Industrial & Management Optimization, 2017, 13 (2) : 857-872. doi: 10.3934/jimo.2016050
[3]

Rouhollah Tavakoli, Hongchao Zhang. A nonmonotone spectral projected gradient method for large-scale topology optimization problems. Numerical Algebra, Control & Optimization, 2012, 2 (2) : 395-412. doi: 10.3934/naco.2012.2.395
[4]

Su-Hong Jiang, Min Li. A modified strictly contractive peaceman-rachford splitting method for multi-block separable convex programming. Journal of Industrial & Management Optimization, 2018, 14 (1) : 397-412. doi: 10.3934/jimo.2017052
[5]

Mickaël Crampon. Entropies of strictly convex projective manifolds. Journal of Modern Dynamics, 2009, 3 (4) : 511-547. doi: 10.3934/jmd.2009.3.511
[6]

Jinkui Liu, Shengjie Li. Multivariate spectral DY-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization, 2017, 13 (1) : 283-295. doi: 10.3934/jimo.2016017
[7]

Lijuan Zhao, Wenyu Sun. Nonmonotone retrospective conic trust region method for unconstrained optimization. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 309-325. doi: 10.3934/naco.2013.3.309
[8]

Joachim Naumann. On the existence of weak solutions of an unsteady p-Laplace thermistor system with strictly monotone electrical conductivities. Discrete & Continuous Dynamical Systems - S, 2017, 10 (4) : 837-852. doi: 10.3934/dcdss.2017042
[9]

Gregorio Díaz, Jesús Ildefonso Díaz. On the free boundary associated with the stationary Monge--Ampère operator on the set of non strictly convex functions. Discrete & Continuous Dynamical Systems - A, 2015, 35 (4) : 1447-1468. doi: 10.3934/dcds.2015.35.1447
[10]

Daniela Saxenhuber, Ronny Ramlau. A gradient-based method for atmospheric tomography. Inverse Problems & Imaging, 2016, 10 (3) : 781-805. doi: 10.3934/ipi.2016021
[11]

Wanyou Cheng, Zixin Chen, Donghui Li. Nomonotone spectral gradient method for sparse recovery. Inverse Problems & Imaging, 2015, 9 (3) : 815-833. doi: 10.3934/ipi.2015.9.815
[12]

Jianjun Zhang, Yunyi Hu, James G. Nagy. A scaled gradient method for digital tomographic image reconstruction. Inverse Problems & Imaging, 2018, 12 (1) : 239-259. doi: 10.3934/ipi.2018010
[13]

Stefan Kindermann. Convergence of the gradient method for ill-posed problems. Inverse Problems & Imaging, 2017, 11 (4) : 703-720. doi: 10.3934/ipi.2017033
[14]

José Antonio Carrillo, Yanghong Huang, Francesco Saverio Patacchini, Gershon Wolansky. Numerical study of a particle method for gradient flows. Kinetic & Related Models, 2017, 10 (3) : 613-641. doi: 10.3934/krm.2017025
[15]

Sanming Liu, Zhijie Wang, Chongyang Liu. On convergence analysis of dual proximal-gradient methods with approximate gradient for a class of nonsmooth convex minimization problems. Journal of Industrial & Management Optimization, 2016, 12 (1) : 389-402. doi: 10.3934/jimo.2016.12.389
[16]

Igor Griva, Roman A. Polyak. Proximal point nonlinear rescaling method for convex optimization. Numerical Algebra, Control & Optimization, 2011, 1 (2) : 283-299. doi: 10.3934/naco.2011.1.283
[17]

Nobuko Sagara, Masao Fukushima. trust region method for nonsmooth convex optimization. Journal of Industrial & Management Optimization, 2005, 1 (2) : 171-180. doi: 10.3934/jimo.2005.1.171
[18]

Jun Chen, Wenyu Sun, Zhenghao Yang. A non-monotone retrospective trust-region method for unconstrained optimization. Journal of Industrial & Management Optimization, 2013, 9 (4) : 919-944. doi: 10.3934/jimo.2013.9.919
[19]

Xiao Ding, Deren Han. A modification of the forward-backward splitting method for maximal monotone mappings. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 295-307. doi: 10.3934/naco.2013.3.295
[20]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013

2017 Impact Factor: 0.994

Tools

Metrics

  • PDF downloads (2)
  • HTML views (0)
  • Cited by (16)

Other articles
by authors

[Back to Top]

Copyright © 2018 American Institute of Mathematical Sciences