April 2018, 5(2): 109-141. doi: 10.3934/jdg.2018008

Constrained stochastic differential games with additive structure: Average and discount payoffs

1. 

Engineering Faculty. Universidad Veracruzana, Coatzacoalcos, Ver., México

2. 

Mathematics Faculty, Universidad Veracruzana, Xalapa, Ver., México

* Corresponding author: Beatris Adriana Escobedo-Trujillo

Received  July 2017 Revised  December 2017 Published  February 2018

This paper deals with two-person nonzero-sum stochastic differential games (SDGs) with an additive structure, subject to constraints that are additive also. Our main objective is to give conditions for the existence of constrained Nash equilibria for the case of infinite-horizon discounted payoff. This is done by means of the Lagrange multipliers approach combined with dynamic programming arguments. Then, following the vanishing discount approach, the results in the discounted case are used to obtain constrained Nash equilibria in the case of long-run average payoff.

Citation: Beatris Adriana Escobedo-Trujillo, Alejandro Alaffita-Hernández, Raquiel López-Martínez. Constrained stochastic differential games with additive structure: Average and discount payoffs. Journal of Dynamics & Games, 2018, 5 (2) : 109-141. doi: 10.3934/jdg.2018008
References:
[1]

R. Akella and P. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automatic Control., 31 (1986), 116-126. doi: 10.1109/TAC.1986.1104206.

[2]

E. Altman and A. Shwartz A, Constrained Markov games: Nash equilibria, in: J.A. Filar, V. Gaitsgory, K. Mizukami (Eds.), Advances in Dynamic Games and Applications, Birkhäuser, Boston, 5 (2000), 213-221.

[3]

E. AltmanK. AvrachenkovR. Marquez and G. Miller, Zero-sum constrained stochastic games with independent state processes, Math. Meth. Oper. Res., 62 (2005), 375-386. doi: 10.1007/s00186-005-0034-4.

[4]

J. Alvarez-Mena and O. Hernández-Lerma, Existence of Nash equilibria for constrained stochastic games, Math. Methods Oper. Res., 63 (2006), 261-285. doi: 10.1007/s00186-005-0003-y.

[5]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223. doi: 10.1137/090762464.

[6]

V. BogachevN. Krylov and M. Röckner, Regularity and global bounds for the densities of invariant measures of diffusion processe, Dokl. Akad. Nauk., 405 (2005), 583-587.

[7]

V. BogachevM. Röckner and V. Stannat, Uniqueness of solutions of elliptic equations and uniqueness pf invariant measures of diffusions processes, Sb. Math., 193 (2002), 945-976.

[8]

V. Borkar, A topology for Markov controls, Appl. Math. Optim., 20 (1989), 55-62. doi: 10.1007/BF01447645.

[9]

V. Borkar and M. Ghosh, Controlled diffusions with constraints, J. Math. Anal. Appl., 152 (1990), 88-108. doi: 10.1016/0022-247X(90)90094-V.

[10]

V. Borkar and M. Ghosh, Controlled diffusions with constraints Ⅱ, J. Math. Anal. Appl., 176 (1993), 310-321. doi: 10.1006/jmaa.1993.1216.

[11]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385.

[12]

A. Calderón and J. Rosenblueth, Minimizing approximate original solutions for commensurate delayed controls, Appl. Math. Lett., 7 (1994), 5-10. doi: 10.1016/0893-9659(94)90063-9.

[13]

A. DvoretzkyA. Wald and J. Wolfowitz, Elimination of randomization in certain statistical decision procedure and zero-sum two person games, Ann. Math. Statist., 22 (1951), 1-21. doi: 10.1214/aoms/1177729689.

[14]

B. Escobedo-TrujilloJ. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687. doi: 10.1007/s10957-011-9974-4.

[15]

G. Folland, Real Analysis. Modern Techniques and Their Applications, 2$^{nd}$ edition, John Wiley and Sons, New York, 1999.

[16]

M. GhoshA. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1962-1988. doi: 10.1137/S0363012996299302.

[17]

M. Ghosh and A. Bagchi, Stochastic game with average payoff criterion, Appl. Math. Optim., 38 (1998), 283-301. doi: 10.1007/s002459900092.

[18]

D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order. Reprinted version, Springer-Verlag, Berlin, 2001.

[19]

P. Glynn and S. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Prob., 24 (1996), 916-931. doi: 10.1214/aop/1039639370.

[20]

H. Hernández-Hernández, Existence of nash equilibria in discounted nonzero-sum stochastic games with additive structure, Morfismos, 6 (2002), 43-65.

[21]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369. doi: 10.1007/s00245-007-9025-6.

[22]

H. Jasso-Fuentes and G. Yin, Advanced Criteria for Controlled Markov-codulated Diffusions in an Infinite Horizon: Overtaking, Bias, and Blackwell Optimality, Science Press, Beijing China, 2013.

[23]

H. Jasso-FuentesB. Escobedo-Trujilo and A. Mendoza-Pérez, The Lagrange and the vanishing discount techniques to controlled diffusions with cost constraints, J. Math. Anal. Appl., 437 (2016), 999-1035. doi: 10.1016/j.jmaa.2016.01.036.

[24]

H. Jasso-FuentesJ. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform., 34 (2017), 283-309. doi: 10.1093/imamci/dnv045.

[25]

F. Klebaner, Introduction to Stochastic Calculus with Applications, 2$^{nd}$ edition, Imperial College Press, London, 2005.

[26]

H. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Internat. Game Theory Rev., 1 (1999), 131-147. doi: 10.1142/S0219198999000098.

[27]

A. Mendoza-PérezH. Jasso-Fuentes and O. Hernández, The Lagrange approach to ergodic control of diffusions with cost constraints, Optimization, 64 (2015), 179-196. doi: 10.1080/02331934.2012.736992.

[28]

S. Meyn and R. Tweedie, Stability of Markovian processes, Ⅲ. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548.

[29]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494. doi: 10.1007/s00186-006-0090-4.

[30]

B. /Oksendal, Stochastic Differential Equations: An Introduction with Applications, 4$^{th}$ edition, Springer-Verlag, New York, 1994.

[31]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454. doi: 10.1007/s001860400392.

[32]

T. RaghavanS. Tijs and O. Vrieze, On stochastic games with additive reward and transition structure, J. Optim. Theory Appl., 47 (1985), 451-464. doi: 10.1007/BF00942191.

[33]

V. Singh and N. Hemachandra, A characterization of stationary Nash equilibria of constrained stochastic games with independent state processes, Operations Research Letters, 42 (2014), 48-52. doi: 10.1016/j.orl.2013.11.007.

[34]

J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1972.

[35]

Q. Wei and X. Chen, Constrained stochastic games with the average payoff criteria, Operations Research Letters, 43 (2015), 83-88. doi: 10.1016/j.orl.2014.12.003.

show all references

References:
[1]

R. Akella and P. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automatic Control., 31 (1986), 116-126. doi: 10.1109/TAC.1986.1104206.

[2]

E. Altman and A. Shwartz A, Constrained Markov games: Nash equilibria, in: J.A. Filar, V. Gaitsgory, K. Mizukami (Eds.), Advances in Dynamic Games and Applications, Birkhäuser, Boston, 5 (2000), 213-221.

[3]

E. AltmanK. AvrachenkovR. Marquez and G. Miller, Zero-sum constrained stochastic games with independent state processes, Math. Meth. Oper. Res., 62 (2005), 375-386. doi: 10.1007/s00186-005-0034-4.

[4]

J. Alvarez-Mena and O. Hernández-Lerma, Existence of Nash equilibria for constrained stochastic games, Math. Methods Oper. Res., 63 (2006), 261-285. doi: 10.1007/s00186-005-0003-y.

[5]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223. doi: 10.1137/090762464.

[6]

V. BogachevN. Krylov and M. Röckner, Regularity and global bounds for the densities of invariant measures of diffusion processe, Dokl. Akad. Nauk., 405 (2005), 583-587.

[7]

V. BogachevM. Röckner and V. Stannat, Uniqueness of solutions of elliptic equations and uniqueness pf invariant measures of diffusions processes, Sb. Math., 193 (2002), 945-976.

[8]

V. Borkar, A topology for Markov controls, Appl. Math. Optim., 20 (1989), 55-62. doi: 10.1007/BF01447645.

[9]

V. Borkar and M. Ghosh, Controlled diffusions with constraints, J. Math. Anal. Appl., 152 (1990), 88-108. doi: 10.1016/0022-247X(90)90094-V.

[10]

V. Borkar and M. Ghosh, Controlled diffusions with constraints Ⅱ, J. Math. Anal. Appl., 176 (1993), 310-321. doi: 10.1006/jmaa.1993.1216.

[11]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385.

[12]

A. Calderón and J. Rosenblueth, Minimizing approximate original solutions for commensurate delayed controls, Appl. Math. Lett., 7 (1994), 5-10. doi: 10.1016/0893-9659(94)90063-9.

[13]

A. DvoretzkyA. Wald and J. Wolfowitz, Elimination of randomization in certain statistical decision procedure and zero-sum two person games, Ann. Math. Statist., 22 (1951), 1-21. doi: 10.1214/aoms/1177729689.

[14]

B. Escobedo-TrujilloJ. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687. doi: 10.1007/s10957-011-9974-4.

[15]

G. Folland, Real Analysis. Modern Techniques and Their Applications, 2$^{nd}$ edition, John Wiley and Sons, New York, 1999.

[16]

M. GhoshA. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1962-1988. doi: 10.1137/S0363012996299302.

[17]

M. Ghosh and A. Bagchi, Stochastic game with average payoff criterion, Appl. Math. Optim., 38 (1998), 283-301. doi: 10.1007/s002459900092.

[18]

D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order. Reprinted version, Springer-Verlag, Berlin, 2001.

[19]

P. Glynn and S. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Prob., 24 (1996), 916-931. doi: 10.1214/aop/1039639370.

[20]

H. Hernández-Hernández, Existence of nash equilibria in discounted nonzero-sum stochastic games with additive structure, Morfismos, 6 (2002), 43-65.

[21]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369. doi: 10.1007/s00245-007-9025-6.

[22]

H. Jasso-Fuentes and G. Yin, Advanced Criteria for Controlled Markov-codulated Diffusions in an Infinite Horizon: Overtaking, Bias, and Blackwell Optimality, Science Press, Beijing China, 2013.

[23]

H. Jasso-FuentesB. Escobedo-Trujilo and A. Mendoza-Pérez, The Lagrange and the vanishing discount techniques to controlled diffusions with cost constraints, J. Math. Anal. Appl., 437 (2016), 999-1035. doi: 10.1016/j.jmaa.2016.01.036.

[24]

H. Jasso-FuentesJ. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform., 34 (2017), 283-309. doi: 10.1093/imamci/dnv045.

[25]

F. Klebaner, Introduction to Stochastic Calculus with Applications, 2$^{nd}$ edition, Imperial College Press, London, 2005.

[26]

H. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Internat. Game Theory Rev., 1 (1999), 131-147. doi: 10.1142/S0219198999000098.

[27]

A. Mendoza-PérezH. Jasso-Fuentes and O. Hernández, The Lagrange approach to ergodic control of diffusions with cost constraints, Optimization, 64 (2015), 179-196. doi: 10.1080/02331934.2012.736992.

[28]

S. Meyn and R. Tweedie, Stability of Markovian processes, Ⅲ. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548.

[29]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494. doi: 10.1007/s00186-006-0090-4.

[30]

B. /Oksendal, Stochastic Differential Equations: An Introduction with Applications, 4$^{th}$ edition, Springer-Verlag, New York, 1994.

[31]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454. doi: 10.1007/s001860400392.

[32]

T. RaghavanS. Tijs and O. Vrieze, On stochastic games with additive reward and transition structure, J. Optim. Theory Appl., 47 (1985), 451-464. doi: 10.1007/BF00942191.

[33]

V. Singh and N. Hemachandra, A characterization of stationary Nash equilibria of constrained stochastic games with independent state processes, Operations Research Letters, 42 (2014), 48-52. doi: 10.1016/j.orl.2013.11.007.

[34]

J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1972.

[35]

Q. Wei and X. Chen, Constrained stochastic games with the average payoff criteria, Operations Research Letters, 43 (2015), 83-88. doi: 10.1016/j.orl.2014.12.003.

[1]

Takeshi Fukao, Nobuyuki Kenmochi. Abstract theory of variational inequalities and Lagrange multipliers. Conference Publications, 2013, 2013 (special) : 237-246. doi: 10.3934/proc.2013.2013.237

[2]

Karla L. Cortez, Javier F. Rosenblueth. Normality and uniqueness of Lagrange multipliers. Discrete & Continuous Dynamical Systems - A, 2018, 38 (6) : 3169-3188. doi: 10.3934/dcds.2018138

[3]

Ying Gao, Xinmin Yang, Jin Yang, Hong Yan. Scalarizations and Lagrange multipliers for approximate solutions in the vector optimization problems with set-valued maps. Journal of Industrial & Management Optimization, 2015, 11 (2) : 673-683. doi: 10.3934/jimo.2015.11.673

[4]

Maria do Rosário de Pinho, Ilya Shvartsman. Lipschitz continuity of optimal control and Lagrange multipliers in a problem with mixed and pure state constraints. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 505-522. doi: 10.3934/dcds.2011.29.505

[5]

Beatris Adriana Escobedo-Trujillo, José Daniel López-Barrientos. Nonzero-sum stochastic differential games with additive structure and average payoffs. Journal of Dynamics & Games, 2014, 1 (4) : 555-578. doi: 10.3934/jdg.2014.1.555

[6]

Liming Wang. A passivity-based stability criterion for reaction diffusion systems with interconnected structure. Discrete & Continuous Dynamical Systems - B, 2012, 17 (1) : 303-323. doi: 10.3934/dcdsb.2012.17.303

[7]

Henry Jacobs, Joris Vankerschaver. Fluid-structure interaction in the Lagrange-Poincaré formalism: The Navier-Stokes and inviscid regimes. Journal of Geometric Mechanics, 2014, 6 (1) : 39-66. doi: 10.3934/jgm.2014.6.39

[8]

Georg Ostrovski, Sebastian van Strien. Payoff performance of fictitious play. Journal of Dynamics & Games, 2014, 1 (4) : 621-638. doi: 10.3934/jdg.2014.1.621

[9]

Lasse Kliemann, Elmira Shirazi Sheykhdarabadi, Anand Srivastav. Price of anarchy for graph coloring games with concave payoff. Journal of Dynamics & Games, 2017, 4 (1) : 41-58. doi: 10.3934/jdg.2017003

[10]

D. Warren, K Najarian. Learning theory applied to Sigmoid network classification of protein biological function using primary protein structure. Conference Publications, 2003, 2003 (Special) : 898-904. doi: 10.3934/proc.2003.2003.898

[11]

Serhii Dyshko. On extendability of additive code isometries. Advances in Mathematics of Communications, 2016, 10 (1) : 45-52. doi: 10.3934/amc.2016.10.45

[12]

Juan Pablo Rincón-Zapatero. Hopf-Lax formula for variational problems with non-constant discount. Journal of Geometric Mechanics, 2009, 1 (3) : 357-367. doi: 10.3934/jgm.2009.1.357

[13]

Beatris A. Escobedo-Trujillo. Discount-sensitive equilibria in zero-sum stochastic differential games. Journal of Dynamics & Games, 2016, 3 (1) : 25-50. doi: 10.3934/jdg.2016002

[14]

Pedro L. García, Antonio Fernández, César Rodrigo. Variational integrators for discrete Lagrange problems. Journal of Geometric Mechanics, 2010, 2 (4) : 343-374. doi: 10.3934/jgm.2010.2.343

[15]

Giovanni Bonfanti, Arrigo Cellina. The validity of the Euler-Lagrange equation. Discrete & Continuous Dynamical Systems - A, 2010, 28 (2) : 511-517. doi: 10.3934/dcds.2010.28.511

[16]

Menita Carozza, Jan Kristensen, Antonia Passarelli di Napoli. On the validity of the Euler-Lagrange system. Communications on Pure & Applied Analysis, 2015, 14 (1) : 51-62. doi: 10.3934/cpaa.2015.14.51

[17]

Cem Güneri, Ferruh Özbudak, Funda ÖzdemIr. On complementary dual additive cyclic codes. Advances in Mathematics of Communications, 2017, 11 (2) : 353-357. doi: 10.3934/amc.2017028

[18]

Constantine M. Dafermos. Hyperbolic balance laws with relaxation. Discrete & Continuous Dynamical Systems - A, 2016, 36 (8) : 4271-4285. doi: 10.3934/dcds.2016.36.4271

[19]

Jiecheng Chen, Dashan Fan, Lijing Sun. Asymptotic estimates for unimodular Fourier multipliers on modulation spaces. Discrete & Continuous Dynamical Systems - A, 2012, 32 (2) : 467-485. doi: 10.3934/dcds.2012.32.467

[20]

Russell E. Warren, Stanley J. Osher. Hyperspectral unmixing by the alternating direction method of multipliers. Inverse Problems & Imaging, 2015, 9 (3) : 917-933. doi: 10.3934/ipi.2015.9.917

 Impact Factor: 

Article outline

[Back to Top]