December 2017, 7(4): 585-622. doi: 10.3934/mcrf.2017022

Time-inconsistent optimal control problems with regime-switching

School of Statistics, East China Normal University, 500 Dongchuan Road, Shanghai 200241, China

Received  January 2016 Revised  April 2017 Published  September 2017

In this paper, a time-inconsistent optimal control problem is studied for diffusion processes modulated by a continuous-time Markov chain. In the performance functional, the running cost and terminal cost depend on not only the initial time, but also the initial state of the Markov chain. By modifying the method of multi-person game, we obtain an equilibrium Hamilton-Jacobi-Bellman equation under proper conditions. The well-posedness of this equilibrium HJB Equation is studied in the case where the diffusion term is independent of the control variable. Furthermore, a time-inconsistent linear-quadratic control problem is considered as a special case.

Citation: Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control & Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022
References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Review of Financial Studies, 23 (2010), 2970-3016.

[2]

T. Björk, Finite dimensional optimal filters for a class of ltô-processes with jumping parameters, Stochastics, 4 (1980), 167-183. doi: 10.1080/17442508008833160.

[3]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochastic control problems, 2010, Working Paper, Stockholm School of Economics.

[4]

T. BjörkA. Murgoci and X. Zhou, Mean–variance portfolio optimization with state-dependent risk aversion, Mathematical Finance, 24 (2014), 1-24. doi: 10.1111/j.1467-9965.2011.00515.x.

[5]

J. Buffington and R. Elliott, American options with regime switching, International Journal of Theoretical and Applied Finance, 5 (2002), 497-514. doi: 10.1142/S0219024902001523.

[6]

E. Çanakoğlu and S. Özekici, HARA frontiers of optimal portfolios in stochastic markets, European Journal of Operational Research, 221 (2012), 129-137. doi: 10.1016/j.ejor.2011.10.012.

[7]

I. Ekeland and A. Lazrak, Being serious about non-commitment: Subgame perfect equilibrium in continuous time, 2006, Preprint. University of British Columbia.

[8]

I. Ekeland and T. Pirvu, Investment and consumption without commitment, Mathematics and Financial Economics, 2 (2008), 57-86. doi: 10.1007/s11579-008-0014-6.

[9]

W. Fleming and H. Soner, Controlled Markov Processes and Viscosity Solutions Springer, New York, 2006.

[10]

A. Friedman, Partial Differential Equations of Parabolic Type Prentice Hall, Inc. , Englewood Cliffs, NJ, 1964.

[11]

S. M. Goldman, Consistent plans, Rev. Financ. Stud., 47 (1980), 533-537. doi: 10.2307/2297304.

[12]

D. Laibson, Golden eggs and hyperbolic discounting, Quarterly Journal of Economics, 112 (1997), 443-478. doi: 10.1162/003355397555253.

[13]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Rev. Financ. Stud., 40 (1973), 391-401. doi: 10.2307/2296458.

[14]

R. A. Pollak, Consistent planning, Rev. Financ. Stud., 35 (1968), 201-208. doi: 10.2307/2296548.

[15]

P. Protter, Stochastic Integration and Differential Equations Springer, Berlin, Heidelberg, 2005.

[16]

L. Sotomayor, Stochastic Control with Regime Switching and its Applications to Financial Economics PhD thesis, University of Alberta, 2008.

[17]

L. Sotomayor and A. Cadenillas, Explicit solutions of consumption-investment problems in financial markets with regime switching, Mathematical Finance, 19 (2009), 251-279. doi: 10.1111/j.1467-9965.2009.00366.x.

[18]

R. Strotz, Myopia and inconsistency in dynamic utility maximization, Readings in Welfare Economics, (1973), 128-143. doi: 10.1007/978-1-349-15492-0_10.

[19]

H. Wang and Z. Wu, Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium hjb equation, Mathematical Control and Related Fields, 5 (2015), 651-678. doi: 10.3934/mcrf.2015.5.651.

[20]

J. WeiR. Wang and H. Yang, On the optimal dividend strategy in a regime-switching diffusion model, Advances in Applied Probability, 44 (2012), 886-906. doi: 10.1017/S0001867800005929.

[21]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Mathematical Control and Related Fields, 1 (2011), 83-118. doi: 10.3934/mcrf.2011.1.83.

[22]

J. Yong, Time-inconsistent optimal control problems and the equilibrium HJB equation, Mathematical Control and Related Fields, 2 (2012), 271-329. doi: 10.3934/mcrf.2012.2.271.

[23]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations-time-consistent solutions, SIAM J. Control Optim. , 51 (2013), 2809–2838, arXiv: 1304.3964 [math. OC]. doi: 10.1137/120892477.

[24]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[25]

Q. Zhang and G. Yin, Nearly-optimal asset allocation in hybrid stock investment models, Journal of Optimization Theory and Applications, 121 (2004), 419-444. doi: 10.1023/B:JOTA.0000037412.23243.6c.

[26]

Q. ZhaoY. Shen and J. Wei, Consumption–investment strategies with non-exponential discounting and logarithmic utility, European Journal of Operational Research, 238 (2014), 824-835. doi: 10.1016/j.ejor.2014.04.034.

[27]

X. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482. doi: 10.1137/S0363012902405583.

show all references

References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Review of Financial Studies, 23 (2010), 2970-3016.

[2]

T. Björk, Finite dimensional optimal filters for a class of ltô-processes with jumping parameters, Stochastics, 4 (1980), 167-183. doi: 10.1080/17442508008833160.

[3]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochastic control problems, 2010, Working Paper, Stockholm School of Economics.

[4]

T. BjörkA. Murgoci and X. Zhou, Mean–variance portfolio optimization with state-dependent risk aversion, Mathematical Finance, 24 (2014), 1-24. doi: 10.1111/j.1467-9965.2011.00515.x.

[5]

J. Buffington and R. Elliott, American options with regime switching, International Journal of Theoretical and Applied Finance, 5 (2002), 497-514. doi: 10.1142/S0219024902001523.

[6]

E. Çanakoğlu and S. Özekici, HARA frontiers of optimal portfolios in stochastic markets, European Journal of Operational Research, 221 (2012), 129-137. doi: 10.1016/j.ejor.2011.10.012.

[7]

I. Ekeland and A. Lazrak, Being serious about non-commitment: Subgame perfect equilibrium in continuous time, 2006, Preprint. University of British Columbia.

[8]

I. Ekeland and T. Pirvu, Investment and consumption without commitment, Mathematics and Financial Economics, 2 (2008), 57-86. doi: 10.1007/s11579-008-0014-6.

[9]

W. Fleming and H. Soner, Controlled Markov Processes and Viscosity Solutions Springer, New York, 2006.

[10]

A. Friedman, Partial Differential Equations of Parabolic Type Prentice Hall, Inc. , Englewood Cliffs, NJ, 1964.

[11]

S. M. Goldman, Consistent plans, Rev. Financ. Stud., 47 (1980), 533-537. doi: 10.2307/2297304.

[12]

D. Laibson, Golden eggs and hyperbolic discounting, Quarterly Journal of Economics, 112 (1997), 443-478. doi: 10.1162/003355397555253.

[13]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Rev. Financ. Stud., 40 (1973), 391-401. doi: 10.2307/2296458.

[14]

R. A. Pollak, Consistent planning, Rev. Financ. Stud., 35 (1968), 201-208. doi: 10.2307/2296548.

[15]

P. Protter, Stochastic Integration and Differential Equations Springer, Berlin, Heidelberg, 2005.

[16]

L. Sotomayor, Stochastic Control with Regime Switching and its Applications to Financial Economics PhD thesis, University of Alberta, 2008.

[17]

L. Sotomayor and A. Cadenillas, Explicit solutions of consumption-investment problems in financial markets with regime switching, Mathematical Finance, 19 (2009), 251-279. doi: 10.1111/j.1467-9965.2009.00366.x.

[18]

R. Strotz, Myopia and inconsistency in dynamic utility maximization, Readings in Welfare Economics, (1973), 128-143. doi: 10.1007/978-1-349-15492-0_10.

[19]

H. Wang and Z. Wu, Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium hjb equation, Mathematical Control and Related Fields, 5 (2015), 651-678. doi: 10.3934/mcrf.2015.5.651.

[20]

J. WeiR. Wang and H. Yang, On the optimal dividend strategy in a regime-switching diffusion model, Advances in Applied Probability, 44 (2012), 886-906. doi: 10.1017/S0001867800005929.

[21]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Mathematical Control and Related Fields, 1 (2011), 83-118. doi: 10.3934/mcrf.2011.1.83.

[22]

J. Yong, Time-inconsistent optimal control problems and the equilibrium HJB equation, Mathematical Control and Related Fields, 2 (2012), 271-329. doi: 10.3934/mcrf.2012.2.271.

[23]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations-time-consistent solutions, SIAM J. Control Optim. , 51 (2013), 2809–2838, arXiv: 1304.3964 [math. OC]. doi: 10.1137/120892477.

[24]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[25]

Q. Zhang and G. Yin, Nearly-optimal asset allocation in hybrid stock investment models, Journal of Optimization Theory and Applications, 121 (2004), 419-444. doi: 10.1023/B:JOTA.0000037412.23243.6c.

[26]

Q. ZhaoY. Shen and J. Wei, Consumption–investment strategies with non-exponential discounting and logarithmic utility, European Journal of Operational Research, 238 (2014), 824-835. doi: 10.1016/j.ejor.2014.04.034.

[27]

X. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482. doi: 10.1137/S0363012902405583.

Figure 1.  The solutions for $P(1,t,1)$ and $P(2,t,2)$
[1]

Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369

[2]

Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161

[3]

Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251

[4]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764

[5]

Valery Y. Glizer, Oleg Kelis. Singular infinite horizon zero-sum linear-quadratic differential game: Saddle-point equilibrium sequence. Numerical Algebra, Control & Optimization, 2017, 7 (1) : 1-20. doi: 10.3934/naco.2017001

[6]

Mohamed Assellaou, Olivier Bokanowski, Hasnaa Zidani. Error estimates for second order Hamilton-Jacobi-Bellman equations. Approximation of probabilistic reachable sets. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 3933-3964. doi: 10.3934/dcds.2015.35.3933

[7]

Federica Masiero. Hamilton Jacobi Bellman equations in infinite dimensions with quadratic and superquadratic Hamiltonian. Discrete & Continuous Dynamical Systems - A, 2012, 32 (1) : 223-263. doi: 10.3934/dcds.2012.32.223

[8]

Roberta Fabbri, Russell Johnson, Sylvia Novo, Carmen Núñez. On linear-quadratic dissipative control processes with time-varying coefficients. Discrete & Continuous Dynamical Systems - A, 2013, 33 (1) : 193-210. doi: 10.3934/dcds.2013.33.193

[9]

Shigeaki Koike, Hiroaki Morimoto, Shigeru Sakaguchi. A linear-quadratic control problem with discretionary stopping. Discrete & Continuous Dynamical Systems - B, 2007, 8 (2) : 261-277. doi: 10.3934/dcdsb.2007.8.261

[10]

Russell Johnson, Carmen Núñez. Remarks on linear-quadratic dissipative control systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (3) : 889-914. doi: 10.3934/dcdsb.2015.20.889

[11]

Hongyan Yan, Yun Sun, Yuanguo Zhu. A linear-quadratic control problem of uncertain discrete-time switched systems. Journal of Industrial & Management Optimization, 2017, 13 (1) : 267-282. doi: 10.3934/jimo.2016016

[12]

Henri Bonnel, Ngoc Sang Pham. Nonsmooth optimization over the (weakly or properly) Pareto set of a linear-quadratic multi-objective control problem: Explicit optimality conditions. Journal of Industrial & Management Optimization, 2011, 7 (4) : 789-809. doi: 10.3934/jimo.2011.7.789

[13]

Zhuo Jin, Linyi Qian. Lookback option pricing for regime-switching jump diffusion models. Mathematical Control & Related Fields, 2015, 5 (2) : 237-258. doi: 10.3934/mcrf.2015.5.237

[14]

Fuke Wu, George Yin, Zhuo Jin. Kolmogorov-type systems with regime-switching jump diffusion perturbations. Discrete & Continuous Dynamical Systems - B, 2016, 21 (7) : 2293-2319. doi: 10.3934/dcdsb.2016048

[15]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97

[16]

Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465

[17]

Walter Alt, Robert Baier, Matthias Gerdts, Frank Lempio. Error bounds for Euler approximation of linear-quadratic control problems with bang-bang solutions. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 547-570. doi: 10.3934/naco.2012.2.547

[18]

Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295

[19]

Zhenbo Wang, Wenxun Xing, Shu-Cherng Fang. Two-person knapsack game. Journal of Industrial & Management Optimization, 2010, 6 (4) : 847-860. doi: 10.3934/jimo.2010.6.847

[20]

Manuel de León, Juan Carlos Marrero, David Martín de Diego. Linear almost Poisson structures and Hamilton-Jacobi equation. Applications to nonholonomic mechanics. Journal of Geometric Mechanics, 2010, 2 (2) : 159-198. doi: 10.3934/jgm.2010.2.159

2017 Impact Factor: 0.631

Metrics

  • PDF downloads (33)
  • HTML views (110)
  • Cited by (0)

Other articles
by authors

[Back to Top]