March  2015, 5(1): 97-139. doi: 10.3934/mcrf.2015.5.97

A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon

1. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, China

2. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  July 2013 Revised  November 2013 Published  January 2015

A linear-quadratic (LQ, for short) optimal control problem is considered for mean-field stochastic differential equations with constant coefficients in an infinite horizon. The stabilizability of the control system is studied followed by the discussion of the well-posedness of the LQ problem. The optimal control can be expressed as a linear state feedback involving the state and its mean, through the solutions of two algebraic Riccati equations. The solvability of such kind of Riccati equations is investigated by means of semi-definite programming method.
Citation: Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65. doi: 10.1016/0304-4149(95)00050-X. Google Scholar

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183. Google Scholar

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356. doi: 10.1137/050645944. Google Scholar

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131. doi: 10.1109/9.863597. Google Scholar

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434. doi: 10.1137/0117041. Google Scholar

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341. doi: 10.1007/s00245-010-9123-8. Google Scholar

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003). Google Scholar

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007). doi: 10.1007/978-0-8176-4581-6. Google Scholar

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884. doi: 10.1080/07362994.2010.482836. Google Scholar

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994). doi: 10.1137/1.9781611970777. Google Scholar

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197. doi: 10.1007/s00245-011-9136-y. Google Scholar

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524. doi: 10.1214/08-AOP442. Google Scholar

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133. doi: 10.1016/j.spa.2009.05.002. Google Scholar

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431. doi: 10.1214/aop/1176988866. Google Scholar

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507. Google Scholar

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53. doi: 10.1080/17442500902723575. Google Scholar

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29. doi: 10.1007/BF01010922. Google Scholar

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247. doi: 10.1080/17442508708833446. Google Scholar

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015. doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0. Google Scholar

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197. doi: 10.1002/mana.19881370116. Google Scholar

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69. doi: 10.1016/0304-4149(92)90138-G. Google Scholar

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95. doi: 10.1007/978-1-84800-155-8_7. Google Scholar

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221. doi: 10.4310/CIS.2006.v6.n3.a5. Google Scholar

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171. Google Scholar

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937. doi: 10.1080/07362994.2010.515194. Google Scholar

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189. doi: 10.1007/s00440-008-0188-0. Google Scholar

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229. doi: 10.1007/s11537-007-0657-8. Google Scholar

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149. doi: 10.1023/A:1024887007165. Google Scholar

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907. doi: 10.1073/pnas.56.6.1907. Google Scholar

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643. doi: 10.1080/17442508.2011.651619. Google Scholar

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328. doi: 10.1080/01630560802580679. Google Scholar

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406. doi: 10.1017/S0305004100030401. Google Scholar

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246. doi: 10.1017/S1446788700029384. Google Scholar

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189. doi: 10.1007/s10107-002-0347-5. Google Scholar

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49. doi: 10.1137/1038003. Google Scholar

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471. doi: 10.1007/3-540-31186-6_29. Google Scholar

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809. doi: 10.1137/120892477. Google Scholar

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999). doi: 10.1007/978-1-4612-1466-3. Google Scholar

show all references

References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65. doi: 10.1016/0304-4149(95)00050-X. Google Scholar

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183. Google Scholar

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356. doi: 10.1137/050645944. Google Scholar

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131. doi: 10.1109/9.863597. Google Scholar

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434. doi: 10.1137/0117041. Google Scholar

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341. doi: 10.1007/s00245-010-9123-8. Google Scholar

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003). Google Scholar

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007). doi: 10.1007/978-0-8176-4581-6. Google Scholar

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884. doi: 10.1080/07362994.2010.482836. Google Scholar

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994). doi: 10.1137/1.9781611970777. Google Scholar

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197. doi: 10.1007/s00245-011-9136-y. Google Scholar

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524. doi: 10.1214/08-AOP442. Google Scholar

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133. doi: 10.1016/j.spa.2009.05.002. Google Scholar

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431. doi: 10.1214/aop/1176988866. Google Scholar

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507. Google Scholar

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53. doi: 10.1080/17442500902723575. Google Scholar

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29. doi: 10.1007/BF01010922. Google Scholar

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247. doi: 10.1080/17442508708833446. Google Scholar

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015. doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0. Google Scholar

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197. doi: 10.1002/mana.19881370116. Google Scholar

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69. doi: 10.1016/0304-4149(92)90138-G. Google Scholar

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95. doi: 10.1007/978-1-84800-155-8_7. Google Scholar

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221. doi: 10.4310/CIS.2006.v6.n3.a5. Google Scholar

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171. Google Scholar

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937. doi: 10.1080/07362994.2010.515194. Google Scholar

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189. doi: 10.1007/s00440-008-0188-0. Google Scholar

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229. doi: 10.1007/s11537-007-0657-8. Google Scholar

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149. doi: 10.1023/A:1024887007165. Google Scholar

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907. doi: 10.1073/pnas.56.6.1907. Google Scholar

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643. doi: 10.1080/17442508.2011.651619. Google Scholar

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328. doi: 10.1080/01630560802580679. Google Scholar

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406. doi: 10.1017/S0305004100030401. Google Scholar

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246. doi: 10.1017/S1446788700029384. Google Scholar

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189. doi: 10.1007/s10107-002-0347-5. Google Scholar

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49. doi: 10.1137/1038003. Google Scholar

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471. doi: 10.1007/3-540-31186-6_29. Google Scholar

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809. doi: 10.1137/120892477. Google Scholar

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999). doi: 10.1007/978-1-4612-1466-3. Google Scholar

[1]

Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control & Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018

[2]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control & Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028

[3]

Martino Bardi. Explicit solutions of some linear-quadratic mean field games. Networks & Heterogeneous Media, 2012, 7 (2) : 243-261. doi: 10.3934/nhm.2012.7.243

[4]

Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete & Continuous Dynamical Systems - A, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117

[5]

Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017

[6]

Thierry Horsin, Peter I. Kogut. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. I. Existence result. Mathematical Control & Related Fields, 2015, 5 (1) : 73-96. doi: 10.3934/mcrf.2015.5.73

[7]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764

[8]

Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks & Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699

[9]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[10]

Shigeaki Koike, Hiroaki Morimoto, Shigeru Sakaguchi. A linear-quadratic control problem with discretionary stopping. Discrete & Continuous Dynamical Systems - B, 2007, 8 (2) : 261-277. doi: 10.3934/dcdsb.2007.8.261

[11]

Russell Johnson, Carmen Núñez. Remarks on linear-quadratic dissipative control systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (3) : 889-914. doi: 10.3934/dcdsb.2015.20.889

[12]

Haiyan Zhang. A necessary condition for mean-field type stochastic differential equations with correlated state and observation noises. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1287-1301. doi: 10.3934/jimo.2016.12.1287

[13]

Juan Li, Wenqiang Li. Controlled reflected mean-field backward stochastic differential equations coupled with value function and related PDEs. Mathematical Control & Related Fields, 2015, 5 (3) : 501-516. doi: 10.3934/mcrf.2015.5.501

[14]

Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control & Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018

[15]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control & Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026

[16]

Tyrone E. Duncan. Some linear-quadratic stochastic differential games for equations in Hilbert spaces with fractional Brownian motions. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5435-5445. doi: 10.3934/dcds.2015.35.5435

[17]

Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465

[18]

Rong Yang, Li Chen. Mean-field limit for a collision-avoiding flocking system and the time-asymptotic flocking dynamics for the kinetic equation. Kinetic & Related Models, 2014, 7 (2) : 381-400. doi: 10.3934/krm.2014.7.381

[19]

Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Mean-field backward stochastic Volterra integral equations. Discrete & Continuous Dynamical Systems - B, 2013, 18 (7) : 1929-1967. doi: 10.3934/dcdsb.2013.18.1929

[20]

Roberta Fabbri, Russell Johnson, Sylvia Novo, Carmen Núñez. On linear-quadratic dissipative control processes with time-varying coefficients. Discrete & Continuous Dynamical Systems - A, 2013, 33 (1) : 193-210. doi: 10.3934/dcds.2013.33.193

2018 Impact Factor: 1.292

Metrics

  • PDF downloads (29)
  • HTML views (0)
  • Cited by (13)

Other articles
by authors

[Back to Top]