April  2018, 5(2): 109-141. doi: 10.3934/jdg.2018008

Constrained stochastic differential games with additive structure: Average and discount payoffs

1. 

Engineering Faculty. Universidad Veracruzana, Coatzacoalcos, Ver., México

2. 

Mathematics Faculty, Universidad Veracruzana, Xalapa, Ver., México

* Corresponding author: Beatris Adriana Escobedo-Trujillo

Received  July 2017 Revised  December 2017 Published  February 2018

This paper deals with two-person nonzero-sum stochastic differential games (SDGs) with an additive structure, subject to constraints that are additive also. Our main objective is to give conditions for the existence of constrained Nash equilibria for the case of infinite-horizon discounted payoff. This is done by means of the Lagrange multipliers approach combined with dynamic programming arguments. Then, following the vanishing discount approach, the results in the discounted case are used to obtain constrained Nash equilibria in the case of long-run average payoff.

Citation: Beatris Adriana Escobedo-Trujillo, Alejandro Alaffita-Hernández, Raquiel López-Martínez. Constrained stochastic differential games with additive structure: Average and discount payoffs. Journal of Dynamics & Games, 2018, 5 (2) : 109-141. doi: 10.3934/jdg.2018008
References:
[1]

R. Akella and P. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automatic Control., 31 (1986), 116-126. doi: 10.1109/TAC.1986.1104206. Google Scholar

[2]

E. Altman and A. Shwartz A, Constrained Markov games: Nash equilibria, in: J.A. Filar, V. Gaitsgory, K. Mizukami (Eds.), Advances in Dynamic Games and Applications, Birkhäuser, Boston, 5 (2000), 213-221. Google Scholar

[3]

E. AltmanK. AvrachenkovR. Marquez and G. Miller, Zero-sum constrained stochastic games with independent state processes, Math. Meth. Oper. Res., 62 (2005), 375-386. doi: 10.1007/s00186-005-0034-4. Google Scholar

[4]

J. Alvarez-Mena and O. Hernández-Lerma, Existence of Nash equilibria for constrained stochastic games, Math. Methods Oper. Res., 63 (2006), 261-285. doi: 10.1007/s00186-005-0003-y. Google Scholar

[5]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223. doi: 10.1137/090762464. Google Scholar

[6]

V. BogachevN. Krylov and M. Röckner, Regularity and global bounds for the densities of invariant measures of diffusion processe, Dokl. Akad. Nauk., 405 (2005), 583-587. Google Scholar

[7]

V. BogachevM. Röckner and V. Stannat, Uniqueness of solutions of elliptic equations and uniqueness pf invariant measures of diffusions processes, Sb. Math., 193 (2002), 945-976. Google Scholar

[8]

V. Borkar, A topology for Markov controls, Appl. Math. Optim., 20 (1989), 55-62. doi: 10.1007/BF01447645. Google Scholar

[9]

V. Borkar and M. Ghosh, Controlled diffusions with constraints, J. Math. Anal. Appl., 152 (1990), 88-108. doi: 10.1016/0022-247X(90)90094-V. Google Scholar

[10]

V. Borkar and M. Ghosh, Controlled diffusions with constraints Ⅱ, J. Math. Anal. Appl., 176 (1993), 310-321. doi: 10.1006/jmaa.1993.1216. Google Scholar

[11]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385. Google Scholar

[12]

A. Calderón and J. Rosenblueth, Minimizing approximate original solutions for commensurate delayed controls, Appl. Math. Lett., 7 (1994), 5-10. doi: 10.1016/0893-9659(94)90063-9. Google Scholar

[13]

A. DvoretzkyA. Wald and J. Wolfowitz, Elimination of randomization in certain statistical decision procedure and zero-sum two person games, Ann. Math. Statist., 22 (1951), 1-21. doi: 10.1214/aoms/1177729689. Google Scholar

[14]

B. Escobedo-TrujilloJ. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687. doi: 10.1007/s10957-011-9974-4. Google Scholar

[15]

G. Folland, Real Analysis. Modern Techniques and Their Applications, 2$^{nd}$ edition, John Wiley and Sons, New York, 1999. Google Scholar

[16]

M. GhoshA. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1962-1988. doi: 10.1137/S0363012996299302. Google Scholar

[17]

M. Ghosh and A. Bagchi, Stochastic game with average payoff criterion, Appl. Math. Optim., 38 (1998), 283-301. doi: 10.1007/s002459900092. Google Scholar

[18]

D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order. Reprinted version, Springer-Verlag, Berlin, 2001. Google Scholar

[19]

P. Glynn and S. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Prob., 24 (1996), 916-931. doi: 10.1214/aop/1039639370. Google Scholar

[20]

H. Hernández-Hernández, Existence of nash equilibria in discounted nonzero-sum stochastic games with additive structure, Morfismos, 6 (2002), 43-65. Google Scholar

[21]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369. doi: 10.1007/s00245-007-9025-6. Google Scholar

[22]

H. Jasso-Fuentes and G. Yin, Advanced Criteria for Controlled Markov-codulated Diffusions in an Infinite Horizon: Overtaking, Bias, and Blackwell Optimality, Science Press, Beijing China, 2013.Google Scholar

[23]

H. Jasso-FuentesB. Escobedo-Trujilo and A. Mendoza-Pérez, The Lagrange and the vanishing discount techniques to controlled diffusions with cost constraints, J. Math. Anal. Appl., 437 (2016), 999-1035. doi: 10.1016/j.jmaa.2016.01.036. Google Scholar

[24]

H. Jasso-FuentesJ. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform., 34 (2017), 283-309. doi: 10.1093/imamci/dnv045. Google Scholar

[25]

F. Klebaner, Introduction to Stochastic Calculus with Applications, 2$^{nd}$ edition, Imperial College Press, London, 2005. Google Scholar

[26]

H. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Internat. Game Theory Rev., 1 (1999), 131-147. doi: 10.1142/S0219198999000098. Google Scholar

[27]

A. Mendoza-PérezH. Jasso-Fuentes and O. Hernández, The Lagrange approach to ergodic control of diffusions with cost constraints, Optimization, 64 (2015), 179-196. doi: 10.1080/02331934.2012.736992. Google Scholar

[28]

S. Meyn and R. Tweedie, Stability of Markovian processes, Ⅲ. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548. Google Scholar

[29]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494. doi: 10.1007/s00186-006-0090-4. Google Scholar

[30]

B. /Oksendal, Stochastic Differential Equations: An Introduction with Applications, 4$^{th}$ edition, Springer-Verlag, New York, 1994. Google Scholar

[31]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454. doi: 10.1007/s001860400392. Google Scholar

[32]

T. RaghavanS. Tijs and O. Vrieze, On stochastic games with additive reward and transition structure, J. Optim. Theory Appl., 47 (1985), 451-464. doi: 10.1007/BF00942191. Google Scholar

[33]

V. Singh and N. Hemachandra, A characterization of stationary Nash equilibria of constrained stochastic games with independent state processes, Operations Research Letters, 42 (2014), 48-52. doi: 10.1016/j.orl.2013.11.007. Google Scholar

[34]

J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1972. Google Scholar

[35]

Q. Wei and X. Chen, Constrained stochastic games with the average payoff criteria, Operations Research Letters, 43 (2015), 83-88. doi: 10.1016/j.orl.2014.12.003. Google Scholar

show all references

References:
[1]

R. Akella and P. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automatic Control., 31 (1986), 116-126. doi: 10.1109/TAC.1986.1104206. Google Scholar

[2]

E. Altman and A. Shwartz A, Constrained Markov games: Nash equilibria, in: J.A. Filar, V. Gaitsgory, K. Mizukami (Eds.), Advances in Dynamic Games and Applications, Birkhäuser, Boston, 5 (2000), 213-221. Google Scholar

[3]

E. AltmanK. AvrachenkovR. Marquez and G. Miller, Zero-sum constrained stochastic games with independent state processes, Math. Meth. Oper. Res., 62 (2005), 375-386. doi: 10.1007/s00186-005-0034-4. Google Scholar

[4]

J. Alvarez-Mena and O. Hernández-Lerma, Existence of Nash equilibria for constrained stochastic games, Math. Methods Oper. Res., 63 (2006), 261-285. doi: 10.1007/s00186-005-0003-y. Google Scholar

[5]

A. Arapostathis and V. Borkar, Uniform recurrence properties of controlled diffusions and applications to optimal control, SIAM J. Control Optim., 48 (2010), 4181-4223. doi: 10.1137/090762464. Google Scholar

[6]

V. BogachevN. Krylov and M. Röckner, Regularity and global bounds for the densities of invariant measures of diffusion processe, Dokl. Akad. Nauk., 405 (2005), 583-587. Google Scholar

[7]

V. BogachevM. Röckner and V. Stannat, Uniqueness of solutions of elliptic equations and uniqueness pf invariant measures of diffusions processes, Sb. Math., 193 (2002), 945-976. Google Scholar

[8]

V. Borkar, A topology for Markov controls, Appl. Math. Optim., 20 (1989), 55-62. doi: 10.1007/BF01447645. Google Scholar

[9]

V. Borkar and M. Ghosh, Controlled diffusions with constraints, J. Math. Anal. Appl., 152 (1990), 88-108. doi: 10.1016/0022-247X(90)90094-V. Google Scholar

[10]

V. Borkar and M. Ghosh, Controlled diffusions with constraints Ⅱ, J. Math. Anal. Appl., 176 (1993), 310-321. doi: 10.1006/jmaa.1993.1216. Google Scholar

[11]

V. Borkar and M. Ghosh, Stochastic differential games: Occupation measure based approach, J. Optim. Theory Appl., 73 (1992), 359-385. Google Scholar

[12]

A. Calderón and J. Rosenblueth, Minimizing approximate original solutions for commensurate delayed controls, Appl. Math. Lett., 7 (1994), 5-10. doi: 10.1016/0893-9659(94)90063-9. Google Scholar

[13]

A. DvoretzkyA. Wald and J. Wolfowitz, Elimination of randomization in certain statistical decision procedure and zero-sum two person games, Ann. Math. Statist., 22 (1951), 1-21. doi: 10.1214/aoms/1177729689. Google Scholar

[14]

B. Escobedo-TrujilloJ. López-Barrientos and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum stochastic differential games, J. Optim. Theory Appl., 153 (2012), 662-687. doi: 10.1007/s10957-011-9974-4. Google Scholar

[15]

G. Folland, Real Analysis. Modern Techniques and Their Applications, 2$^{nd}$ edition, John Wiley and Sons, New York, 1999. Google Scholar

[16]

M. GhoshA. Arapostathis and S. Marcus, Ergodic control of switching diffusions, SIAM J. Control Optim., 35 (1997), 1962-1988. doi: 10.1137/S0363012996299302. Google Scholar

[17]

M. Ghosh and A. Bagchi, Stochastic game with average payoff criterion, Appl. Math. Optim., 38 (1998), 283-301. doi: 10.1007/s002459900092. Google Scholar

[18]

D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order. Reprinted version, Springer-Verlag, Berlin, 2001. Google Scholar

[19]

P. Glynn and S. Meyn, A Liapounov bound for solutions of the Poisson equation, Ann. Prob., 24 (1996), 916-931. doi: 10.1214/aop/1039639370. Google Scholar

[20]

H. Hernández-Hernández, Existence of nash equilibria in discounted nonzero-sum stochastic games with additive structure, Morfismos, 6 (2002), 43-65. Google Scholar

[21]

H. Jasso-Fuentes and O. Hernández-Lerma, Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim., 57 (2008), 349-369. doi: 10.1007/s00245-007-9025-6. Google Scholar

[22]

H. Jasso-Fuentes and G. Yin, Advanced Criteria for Controlled Markov-codulated Diffusions in an Infinite Horizon: Overtaking, Bias, and Blackwell Optimality, Science Press, Beijing China, 2013.Google Scholar

[23]

H. Jasso-FuentesB. Escobedo-Trujilo and A. Mendoza-Pérez, The Lagrange and the vanishing discount techniques to controlled diffusions with cost constraints, J. Math. Anal. Appl., 437 (2016), 999-1035. doi: 10.1016/j.jmaa.2016.01.036. Google Scholar

[24]

H. Jasso-FuentesJ. López-Barrientos and B. Escobedo-Trujilo, Infinite horizon nonzero-sum stochastic differential games with additive structure, IMA J. Math. Control Inform., 34 (2017), 283-309. doi: 10.1093/imamci/dnv045. Google Scholar

[25]

F. Klebaner, Introduction to Stochastic Calculus with Applications, 2$^{nd}$ edition, Imperial College Press, London, 2005. Google Scholar

[26]

H. Küenle, Equilibrium strategies in stochastic games with additive cost and transition structure, Internat. Game Theory Rev., 1 (1999), 131-147. doi: 10.1142/S0219198999000098. Google Scholar

[27]

A. Mendoza-PérezH. Jasso-Fuentes and O. Hernández, The Lagrange approach to ergodic control of diffusions with cost constraints, Optimization, 64 (2015), 179-196. doi: 10.1080/02331934.2012.736992. Google Scholar

[28]

S. Meyn and R. Tweedie, Stability of Markovian processes, Ⅲ. Foster-Lyapunov criteria for continuous-time precesses, Adv. Appl. Prob., 25 (1993), 518-548. Google Scholar

[29]

A. Nowak, Remark on sensitive equilibria in stochastic games with additive reward and transition structure, Math. Meth. Oper. Res., 64 (2006), 481-494. doi: 10.1007/s00186-006-0090-4. Google Scholar

[30]

B. /Oksendal, Stochastic Differential Equations: An Introduction with Applications, 4$^{th}$ edition, Springer-Verlag, New York, 1994. Google Scholar

[31]

T. Prieto-Rumeau and O. Hernández-Lerma, Bias and overtaking equilibria for zero-sum continuous-time Markov games, Math. Meth. Oper. Res., 61 (2005), 437-454. doi: 10.1007/s001860400392. Google Scholar

[32]

T. RaghavanS. Tijs and O. Vrieze, On stochastic games with additive reward and transition structure, J. Optim. Theory Appl., 47 (1985), 451-464. doi: 10.1007/BF00942191. Google Scholar

[33]

V. Singh and N. Hemachandra, A characterization of stationary Nash equilibria of constrained stochastic games with independent state processes, Operations Research Letters, 42 (2014), 48-52. doi: 10.1016/j.orl.2013.11.007. Google Scholar

[34]

J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, New York, 1972. Google Scholar

[35]

Q. Wei and X. Chen, Constrained stochastic games with the average payoff criteria, Operations Research Letters, 43 (2015), 83-88. doi: 10.1016/j.orl.2014.12.003. Google Scholar

[1]

Takeshi Fukao, Nobuyuki Kenmochi. Abstract theory of variational inequalities and Lagrange multipliers. Conference Publications, 2013, 2013 (special) : 237-246. doi: 10.3934/proc.2013.2013.237

[2]

Karla L. Cortez, Javier F. Rosenblueth. Normality and uniqueness of Lagrange multipliers. Discrete & Continuous Dynamical Systems - A, 2018, 38 (6) : 3169-3188. doi: 10.3934/dcds.2018138

[3]

Maria do Rosário de Pinho, Ilya Shvartsman. Lipschitz continuity of optimal control and Lagrange multipliers in a problem with mixed and pure state constraints. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 505-522. doi: 10.3934/dcds.2011.29.505

[4]

Ying Gao, Xinmin Yang, Jin Yang, Hong Yan. Scalarizations and Lagrange multipliers for approximate solutions in the vector optimization problems with set-valued maps. Journal of Industrial & Management Optimization, 2015, 11 (2) : 673-683. doi: 10.3934/jimo.2015.11.673

[5]

Beatris Adriana Escobedo-Trujillo, José Daniel López-Barrientos. Nonzero-sum stochastic differential games with additive structure and average payoffs. Journal of Dynamics & Games, 2014, 1 (4) : 555-578. doi: 10.3934/jdg.2014.1.555

[6]

Manfred Deistler. Singular arma systems: A structure theory. Numerical Algebra, Control & Optimization, 2019, 9 (3) : 383-391. doi: 10.3934/naco.2019025

[7]

Liming Wang. A passivity-based stability criterion for reaction diffusion systems with interconnected structure. Discrete & Continuous Dynamical Systems - B, 2012, 17 (1) : 303-323. doi: 10.3934/dcdsb.2012.17.303

[8]

Ting-Hao Hsu, Gail S. K. Wolkowicz. A criterion for the existence of relaxation oscillations with applications to predator-prey systems and an epidemic model. Discrete & Continuous Dynamical Systems - B, 2017, 22 (11) : 0-0. doi: 10.3934/dcdsb.2019219

[9]

Henry Jacobs, Joris Vankerschaver. Fluid-structure interaction in the Lagrange-Poincaré formalism: The Navier-Stokes and inviscid regimes. Journal of Geometric Mechanics, 2014, 6 (1) : 39-66. doi: 10.3934/jgm.2014.6.39

[10]

Georg Ostrovski, Sebastian van Strien. Payoff performance of fictitious play. Journal of Dynamics & Games, 2014, 1 (4) : 621-638. doi: 10.3934/jdg.2014.1.621

[11]

Lasse Kliemann, Elmira Shirazi Sheykhdarabadi, Anand Srivastav. Price of anarchy for graph coloring games with concave payoff. Journal of Dynamics & Games, 2017, 4 (1) : 41-58. doi: 10.3934/jdg.2017003

[12]

D. Warren, K Najarian. Learning theory applied to Sigmoid network classification of protein biological function using primary protein structure. Conference Publications, 2003, 2003 (Special) : 898-904. doi: 10.3934/proc.2003.2003.898

[13]

Serhii Dyshko. On extendability of additive code isometries. Advances in Mathematics of Communications, 2016, 10 (1) : 45-52. doi: 10.3934/amc.2016.10.45

[14]

Simon Hoof. Cooperative dynamic advertising via state-dependent payoff weights. Journal of Dynamics & Games, 2019, 6 (3) : 195-209. doi: 10.3934/jdg.2019014

[15]

Pedro L. García, Antonio Fernández, César Rodrigo. Variational integrators for discrete Lagrange problems. Journal of Geometric Mechanics, 2010, 2 (4) : 343-374. doi: 10.3934/jgm.2010.2.343

[16]

Giovanni Bonfanti, Arrigo Cellina. The validity of the Euler-Lagrange equation. Discrete & Continuous Dynamical Systems - A, 2010, 28 (2) : 511-517. doi: 10.3934/dcds.2010.28.511

[17]

Menita Carozza, Jan Kristensen, Antonia Passarelli di Napoli. On the validity of the Euler-Lagrange system. Communications on Pure & Applied Analysis, 2015, 14 (1) : 51-62. doi: 10.3934/cpaa.2015.14.51

[18]

Juan Pablo Rincón-Zapatero. Hopf-Lax formula for variational problems with non-constant discount. Journal of Geometric Mechanics, 2009, 1 (3) : 357-367. doi: 10.3934/jgm.2009.1.357

[19]

Beatris A. Escobedo-Trujillo. Discount-sensitive equilibria in zero-sum stochastic differential games. Journal of Dynamics & Games, 2016, 3 (1) : 25-50. doi: 10.3934/jdg.2016002

[20]

Cem Güneri, Ferruh Özbudak, Funda ÖzdemIr. On complementary dual additive cyclic codes. Advances in Mathematics of Communications, 2017, 11 (2) : 353-357. doi: 10.3934/amc.2017028

[Back to Top]