2014, 1(1): 57-78. doi: 10.3934/jdg.2014.1.57

On the Euler equation approach to discrete--time nonstationary optimal control problems

1. 

Departamento de Matemáticas, Instituto Tecnológico Autónomo de México (ITAM), Río Hondo 1, México D.F. 01000, Mexico

2. 

Mathematics Department, CINVESTAV-IPN, A. Postal 14-740, México D.F. 07000, Mexico

Received  April 2012 Revised  March 2013 Published  June 2013

We are concerned with deterministic and stochastic nonstationary discrete--time optimal control problems in infinite horizon. We show, using Gâteaux differentials, that the so--called Euler equation and a transversality condition are necessary conditions for optimality. In particular, the transversality condition is obtained in a more general form and under milder hypotheses than in previous works. Sufficient conditions are also provided. We also find closed--form solutions to several (discounted) stationary and nonstationary control problems.
Citation: David González-Sánchez, Onésimo Hernández-Lerma. On the Euler equation approach to discrete--time nonstationary optimal control problems. Journal of Dynamics & Games, 2014, 1 (1) : 57-78. doi: 10.3934/jdg.2014.1.57
References:
[1]

D. Acemoglu, "Introduction to Modern Economic Growth,", Princeton University Press, (2009).

[2]

J. Adda and R. Cooper, "Dynamic Economics. Quantitative Methods and Applications,", MIT Press, (2003).

[3]

V. I. Arkin and I. V. Evstigneev, "Stochastic Models of Control and Economic Dynamics,", Academic Press, (1987).

[4]

Y. Bar-Ness, The discrete Euler equation on the normed linear space $l_n^1$,, Int. J. Control, 21 (1975), 625. doi: 10.1080/00207177508922017.

[5]

W. A. Brock and L. Mirman, Optimal economic growth and uncertainty: The discounted case,, J. Econ. Theory, 4 (1972), 479. doi: 10.1016/0022-0531(72)90135-4.

[6]

J. A. Cadzow, Discrete calculus of variations,, Int. J. Control, 11 (1970), 393. doi: 10.1080/00207177008905922.

[7]

G. C. Chow, "Dynamic Economics: Optimization by the Lagrange Method,", Oxford University Press, (1997).

[8]

I. Ekeland and J. A. Scheinkman, Transversality conditions for some infinite horizon discrete time optimization problems,, Math. Oper. Res., 11 (1986), 216. doi: 10.1287/moor.11.2.216.

[9]

S. Elaydi, "An Introduction to Difference Equations,", Third edition, (2005).

[10]

J. Engwerda, "LQ Dynamic Optimization and Differential Games,", John Wiley & Sons, (2005).

[11]

S. Flåm and A. Fougères, Infinite horizon programs; Convergence of approximate solutions,, Ann. Oper. Res., 29 (1991), 333. doi: 10.1007/BF02283604.

[12]

W. H. Fleming and R. W. Rishel, "Deterministic and Stochastic Optimal Control,", Applications of Mathematics, (1975).

[13]

X. Guo, A. Hernández-del-Valle and O. Hernández-Lerma, Nonstationary discrete-time deterministic and stochastic control systems: Bounded and unbounded cases,, Systems Control Lett., 60 (2011), 503. doi: 10.1016/j.sysconle.2011.04.006.

[14]

O. Hernández-Lerma and J. B. Lasserre, "Discrete-Time Markov Control Processes: Basic Optimality Criteria,", Applications of Mathematics (New York), 30 (1996).

[15]

T. Kamihigashi, A simple proof of the necessity of the transversality condition,, Econ. Theory, 20 (2002), 427. doi: 10.1007/s001990100198.

[16]

T. Kamihigashi, Transversality conditions and dynamic economic behaviour,, in, (2008), 384. doi: 10.1057/9780230226203.1737.

[17]

W. G. Kelley and A. C. Peterson, "Difference Equations. An Introduction with Applications,", Academic Press, (1991).

[18]

C. Le Van and R.-A. Dana, "Dynamic Programming in Economics,", Dynamic Modeling and Econometrics in Economics and Finance, 5 (2003).

[19]

D. Levhari and L. D. Mirman, The great fish war: An example using dynamic Cournot-Nash solution,, Bell J. Econom., 11 (1980), 322. doi: 10.2307/3003416.

[20]

L. Ljungqvist and T. J. Sargent, "Recursive Macroeconomic Theory,", Second edition, (2004).

[21]

D. G. Luenberger, "Optimization by Vector Space Methods,", John Wiley & Sons, (1969).

[22]

K. Okuguchi, A dynamic Cournot-Nash equilibrium in fishery: The effects of entry,, Riv. Mat. Sci. Econom. Social., 4 (1981), 59. doi: 10.1007/BF02123580.

[23]

W. Rudin, "Principles of Mathematical Analysis,", Third edition, (1976).

[24]

I. Schochetman and R. L. Smith, Finite dimensional approximation in infinite-dimensional mathematical programming,, Math. Programming, 54 (1992), 307. doi: 10.1007/BF01586057.

[25]

N. L. Stokey, R. E. Lucas and E. C. Prescott, Jr., "Recursive Methods in Economic Dynamics,", With the collaboration of Edward C. Prescott, (1989).

[26]

K. Sydsæter, P. J. Hammond, A. Seierstad and A. Strøm, "Further Mathematics for Economic Analysis,", Second edition, (2008).

show all references

References:
[1]

D. Acemoglu, "Introduction to Modern Economic Growth,", Princeton University Press, (2009).

[2]

J. Adda and R. Cooper, "Dynamic Economics. Quantitative Methods and Applications,", MIT Press, (2003).

[3]

V. I. Arkin and I. V. Evstigneev, "Stochastic Models of Control and Economic Dynamics,", Academic Press, (1987).

[4]

Y. Bar-Ness, The discrete Euler equation on the normed linear space $l_n^1$,, Int. J. Control, 21 (1975), 625. doi: 10.1080/00207177508922017.

[5]

W. A. Brock and L. Mirman, Optimal economic growth and uncertainty: The discounted case,, J. Econ. Theory, 4 (1972), 479. doi: 10.1016/0022-0531(72)90135-4.

[6]

J. A. Cadzow, Discrete calculus of variations,, Int. J. Control, 11 (1970), 393. doi: 10.1080/00207177008905922.

[7]

G. C. Chow, "Dynamic Economics: Optimization by the Lagrange Method,", Oxford University Press, (1997).

[8]

I. Ekeland and J. A. Scheinkman, Transversality conditions for some infinite horizon discrete time optimization problems,, Math. Oper. Res., 11 (1986), 216. doi: 10.1287/moor.11.2.216.

[9]

S. Elaydi, "An Introduction to Difference Equations,", Third edition, (2005).

[10]

J. Engwerda, "LQ Dynamic Optimization and Differential Games,", John Wiley & Sons, (2005).

[11]

S. Flåm and A. Fougères, Infinite horizon programs; Convergence of approximate solutions,, Ann. Oper. Res., 29 (1991), 333. doi: 10.1007/BF02283604.

[12]

W. H. Fleming and R. W. Rishel, "Deterministic and Stochastic Optimal Control,", Applications of Mathematics, (1975).

[13]

X. Guo, A. Hernández-del-Valle and O. Hernández-Lerma, Nonstationary discrete-time deterministic and stochastic control systems: Bounded and unbounded cases,, Systems Control Lett., 60 (2011), 503. doi: 10.1016/j.sysconle.2011.04.006.

[14]

O. Hernández-Lerma and J. B. Lasserre, "Discrete-Time Markov Control Processes: Basic Optimality Criteria,", Applications of Mathematics (New York), 30 (1996).

[15]

T. Kamihigashi, A simple proof of the necessity of the transversality condition,, Econ. Theory, 20 (2002), 427. doi: 10.1007/s001990100198.

[16]

T. Kamihigashi, Transversality conditions and dynamic economic behaviour,, in, (2008), 384. doi: 10.1057/9780230226203.1737.

[17]

W. G. Kelley and A. C. Peterson, "Difference Equations. An Introduction with Applications,", Academic Press, (1991).

[18]

C. Le Van and R.-A. Dana, "Dynamic Programming in Economics,", Dynamic Modeling and Econometrics in Economics and Finance, 5 (2003).

[19]

D. Levhari and L. D. Mirman, The great fish war: An example using dynamic Cournot-Nash solution,, Bell J. Econom., 11 (1980), 322. doi: 10.2307/3003416.

[20]

L. Ljungqvist and T. J. Sargent, "Recursive Macroeconomic Theory,", Second edition, (2004).

[21]

D. G. Luenberger, "Optimization by Vector Space Methods,", John Wiley & Sons, (1969).

[22]

K. Okuguchi, A dynamic Cournot-Nash equilibrium in fishery: The effects of entry,, Riv. Mat. Sci. Econom. Social., 4 (1981), 59. doi: 10.1007/BF02123580.

[23]

W. Rudin, "Principles of Mathematical Analysis,", Third edition, (1976).

[24]

I. Schochetman and R. L. Smith, Finite dimensional approximation in infinite-dimensional mathematical programming,, Math. Programming, 54 (1992), 307. doi: 10.1007/BF01586057.

[25]

N. L. Stokey, R. E. Lucas and E. C. Prescott, Jr., "Recursive Methods in Economic Dynamics,", With the collaboration of Edward C. Prescott, (1989).

[26]

K. Sydsæter, P. J. Hammond, A. Seierstad and A. Strøm, "Further Mathematics for Economic Analysis,", Second edition, (2008).

[1]

Senda Ounaies, Jean-Marc Bonnisseau, Souhail Chebbi, Halil Mete Soner. Merton problem in an infinite horizon and a discrete time with frictions. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1323-1331. doi: 10.3934/jimo.2016.12.1323

[2]

Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming formulations of deterministic infinite horizon optimal control problems in discrete time. Discrete & Continuous Dynamical Systems - B, 2017, 22 (10) : 3821-3838. doi: 10.3934/dcdsb.2017192

[3]

Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time. Discrete & Continuous Dynamical Systems - B, 2017, 22 (11) : 1-25. doi: 10.3934/dcdsb.2018235

[4]

Yi Wang, Dun Zhou. Transversality for time-periodic competitive-cooperative tridiagonal systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (6) : 1821-1830. doi: 10.3934/dcdsb.2015.20.1821

[5]

Fabio Bagagiolo. An infinite horizon optimal control problem for some switching systems. Discrete & Continuous Dynamical Systems - B, 2001, 1 (4) : 443-462. doi: 10.3934/dcdsb.2001.1.443

[6]

Hongbiao Fan, Jun-E Feng, Min Meng. Piecewise observers of rectangular discrete fuzzy descriptor systems with multiple time-varying delays. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1535-1556. doi: 10.3934/jimo.2016.12.1535

[7]

Tao Pang, Azmat Hussain. An infinite time horizon portfolio optimization model with delays. Mathematical Control & Related Fields, 2016, 6 (4) : 629-651. doi: 10.3934/mcrf.2016018

[8]

Chuandong Li, Fali Ma, Tingwen Huang. 2-D analysis based iterative learning control for linear discrete-time systems with time delay. Journal of Industrial & Management Optimization, 2011, 7 (1) : 175-181. doi: 10.3934/jimo.2011.7.175

[9]

Valery Y. Glizer, Oleg Kelis. Asymptotic properties of an infinite horizon partial cheap control problem for linear systems with known disturbances. Numerical Algebra, Control & Optimization, 2018, 8 (2) : 211-235. doi: 10.3934/naco.2018013

[10]

Tingwen Huang, Guanrong Chen, Juergen Kurths. Synchronization of chaotic systems with time-varying coupling delays. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1071-1082. doi: 10.3934/dcdsb.2011.16.1071

[11]

Karl P. Hadeler. Quiescent phases and stability in discrete time dynamical systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (1) : 129-152. doi: 10.3934/dcdsb.2015.20.129

[12]

Piotr Oprocha. Chain recurrence in multidimensional time discrete dynamical systems. Discrete & Continuous Dynamical Systems - A, 2008, 20 (4) : 1039-1056. doi: 10.3934/dcds.2008.20.1039

[13]

Byungik Kahng, Miguel Mendes. The characterization of maximal invariant sets of non-linear discrete-time control dynamical systems. Conference Publications, 2013, 2013 (special) : 393-406. doi: 10.3934/proc.2013.2013.393

[14]

Elena K. Kostousova. On polyhedral control synthesis for dynamical discrete-time systems under uncertainties and state constraints. Discrete & Continuous Dynamical Systems - A, 2018, 0 (0) : 1-14. doi: 10.3934/dcds.2018153

[15]

Hongyan Yan, Yun Sun, Yuanguo Zhu. A linear-quadratic control problem of uncertain discrete-time switched systems. Journal of Industrial & Management Optimization, 2017, 13 (1) : 267-282. doi: 10.3934/jimo.2016016

[16]

Elena K. Kostousova. On control synthesis for uncertain dynamical discrete-time systems through polyhedral techniques. Conference Publications, 2015, 2015 (special) : 723-732. doi: 10.3934/proc.2015.0723

[17]

Yuefen Chen, Yuanguo Zhu. Indefinite LQ optimal control with process state inequality constraints for discrete-time uncertain systems. Journal of Industrial & Management Optimization, 2018, 14 (3) : 913-930. doi: 10.3934/jimo.2017082

[18]

Monika Dryl, Delfim F. M. Torres. Necessary optimality conditions for infinite horizon variational problems on time scales. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 145-160. doi: 10.3934/naco.2013.3.145

[19]

S. Mohamad, K. Gopalsamy. Neuronal dynamics in time varying enviroments: Continuous and discrete time models. Discrete & Continuous Dynamical Systems - A, 2000, 6 (4) : 841-860. doi: 10.3934/dcds.2000.6.841

[20]

M. Motta, C. Sartori. Exit time problems for nonlinear unbounded control systems. Discrete & Continuous Dynamical Systems - A, 1999, 5 (1) : 137-156. doi: 10.3934/dcds.1999.5.137

 Impact Factor: 

Metrics

  • PDF downloads (3)
  • HTML views (0)
  • Cited by (2)

[Back to Top]