# American Institute of Mathematical Sciences

## Stochastic dynamics Ⅱ: Finite random dynamical systems, linear representation, and entropy production

 1 Department of Applied Mathematics & Statistics, Johns Hopkins University, Baltimore, MD 21218-2608, USA 2 Department of Applied Mathematics, University of Washington, Seattle, WA 98195-3925, USA

* Corresponding author: Felix X.-F. Ye

Received  April 2018 Revised  January 2019 Published  June 2019

We study finite state random dynamical systems (RDS) and their induced Markov chains (MC) as stochastic models for complex dynamics. The linear representation of deterministic maps in RDS is a matrix-valued random variable whose expectation corresponds to the transition matrix of the MC. The instantaneous Gibbs entropy, Shannon-Khinchin entropy of a step, and the entropy production rate of the MC are discussed. These three concepts, as key anchoring points in applications of stochastic dynamics, characterize respectively the uncertainties of a system at instant time $t$, the randomness generated in a step in the dynamics, and the dynamical asymmetry with respect to time reversal. The stationary entropy production rate, expressed in terms of the cycle distributions, has found an expression in terms of the probability of the deterministic maps with single attractor in the maximum entropy RDS. For finite RDS with invertible transformations, the non-negative entropy production rate of its MC is bounded above by the Kullback-Leibler divergence of the probability of the deterministic maps with respect to its time-reversal dual probability.

Citation: Felix X.-F. Ye, Hong Qian. Stochastic dynamics Ⅱ: Finite random dynamical systems, linear representation, and entropy production. Discrete & Continuous Dynamical Systems - B, doi: 10.3934/dcdsb.2019122
##### References:
 [1] H. Arbabi and I. Mezic, Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the Koopman operator, SIAM J. Applied Dynamical Systems, 16 (2017), 2096-2126. doi: 10.1137/17M1125236. [2] L. Arnold, Trends and open problems in the theory of random dynamical systems, in Probability Towards 2000 (eds. L. Accardi and C. C. Heyde), Springer New York, New York, NY, 1998, 34–46. doi: 10.1007/978-1-4612-2224-8_2. [3] L. Arnold, Random Dynamical Systems, Springer Monographs in Mathematics, Springer-Verlag, Berlin, 1998. doi: 10.1007/978-3-662-12878-7. [4] L. Arnold and H. Crauel, Random dynamical systems, in Lyapunov Exponents (Oberwolfach, 1990), vol. 1486 of Lecture Notes in Math., Springer, Berlin, 1991, 1–22. doi: 10.1007/BFb0086654. [5] V. Baladi, Positive Transfer Operators and Decay of Correlations, Advanced series in nonlinear dynamics, World Scientific Publishing Company, 2000. doi: 10.1142/9789812813633. [6] A. Ben-Naim, Entropy and the Second Law: Interpretation and Misss-interpretationsss, World Scientific, 2012. doi: 10.1142/8333. [7] P. G. Bergmann and J. L. Lebowitz, New approach to nonequilibrium processes, Phys. Rev., 99 (1955), 578-587. doi: 10.1103/PhysRev.99.578. [8] P. Bollobas, Modern Graph Theory, Graduate Texts in Mathematics, Springer New York, 1998. doi: 10.1007/978-1-4612-0619-4. [9] S. R. Caplan and D. Zeilberger, T. L. Hill's graphical method for solving linear equations, Advances in Applied Mathematics, 3 (1982), 377-383. doi: 10.1016/S0196-8858(82)80011-0. [10] R. T. Cox, The statistical method of gibbs in irreversible change, Rev. Mod. Phys., 22 (1950), 238-248. doi: 10.1103/RevModPhys.22.238. [11] P. Diaconis and D. Freedman, Iterated random functions, SIAM Review, 41 (1999), 45-76. doi: 10.1137/S0036144598338446. [12] P. Gaspard and X.-J. Wang, Noise, chaos, and ($\epsilon$, $\tau$)-entropy per unit time, Phys. Rep., 235 (1993), 291-343. doi: 10.1016/0370-1573(93)90012-3. [13] T. Hill, Free Energy Transduction and Biochemical Cycle Kinetics, Dover Books on Chemistry, Dover Publications, 2004. doi: 10.1007/978-1-4612-3558-3. [14] A. Hobson, A new theorem of information theory, Journal of Statistical Physics, 1 (1969), 383-391. doi: 10.1007/BF01106578. [15] W. Huang, H. Qian, S. Wang, F. X.-F. Ye and Y. Yi, Synchronization in discrete-time, discrete-state random dynamical systems, In preparation. [16] D.-Q. Jiang, M. Qian and M.-P. Qian, Mathematical Theory of Nonequilibrium Steady States: On the Frontier of Probability and Dynamical Systems, Lecture Notes in Mathematics, 1833. Springer-Verlag, Berlin, 2004. doi: 10.1007/b94615. [17] S. Kalpazidou, Cycle Representations of Markov Processes, Stochastic Modelling and Applied Probability, Springer, New York, 2006. [18] A. Khinchin, Mathematical Foundations of Information Theory, Translated by R. A. Silverman and M. D. Friedman. Dover Publications, Inc., New York, N. Y., 1957. [19] Y. Kifer, Random Perturbations of Dynamical Systems, Progress in Probability, Birkhäuser Boston, Inc., Boston, MA, 1988. doi: 10.1007/978-1-4615-8181-9. [20] E. L. King and C. Altman, A schematic method of deriving the rate laws for enzyme-catalyzed reactions, The Journal of Physical Chemistry, 60 (1956), 1375-1378. doi: 10.1021/j150544a010. [21] A. N. Kolmogorov, Three approaches to the quantitative definition of information, International Journal of Computer Mathematics, 2 (1968), 157-168. doi: 10.1080/00207166808803030. [22] A. Lasota and M. Mackey, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, Second edition. Applied Mathematical Sciences, 97. Springer-Verlag, New York, 1994. doi: 10.1007/978-1-4612-4286-4. [23] K. K. Lin, Stimulus-response reliability of biological networks, in Nonautonomous Dynamical Systems in the Life Sciences (eds. P. E. Kloeden and C. Pötzsche), Springer International Publishing, Cham, 2013,135–161. doi: 10.1007/978-3-319-03080-7_4. [24] J.-L. Luo, C. Van den Broeck and G. Nicolis, Stability criteria and fluctuations around nonequilibrium states, Zeitschrift für Physik B Condensed Matter, 56 (1984), 165–170. doi: 10.1007/BF01469698. [25] Y.-A. Ma, H. Qian and F. X.-F. Ye, Stochastic dynamics: Models for intrinsic and extrinsic noises and their applications (in Chinese), Scientia Sinica Mathematica, 47 (2017), 1693-1702. [26] D. Mumford, The dawning age of stochasticity, in Mathematics: Frontiers and Perspectives (eds. V. Arnold, M. Atiyah, P. Lax and B. Mazur), 197–218, Amer. Math. Soc., Providence, RI, 2000. [27] J. G. Propp and D. B. Wilson, Exact sampling with coupled Markov chains and applications to statistical mechanics, Random Structures & Algorithms, 9 (1996), 223-252. doi: 10.1002/(SICI)1098-2418(199608/09)9:1/2<223::AID-RSA14>3.0.CO;2-O. [28] H. Qian, A decomposition of irreversible diffusion processes without detailed balance, Journal of Mathematical Physics, 54 (2013), 053302, 10pp. doi: 10.1063/1.4803847. [29] H. Qian, Nonlinear stochastic dynamics of complex systems, i: a chemical reaction kinetic perspective with mesoscopic nonequilibrium thermodynamics, arXiv: 1605.08070. [30] A. N. Quas, On representations of Markov chains by random smooth maps, Bulletin of the London Mathematical Society, 23 (1991), 487-492. doi: 10.1112/blms/23.5.487. [31] H. Risken, The Fokker-Planck Equation: Methods of Solution and Applications, Second edition. Springer Series in Synergetics, 18. Springer-Verlag, Berlin, 1989. doi: 10.1007/978-3-642-96807-5. [32] D. Ruelle, Positivity of entropy production in nonequilibrium statistical mechanics, Journal of Statistical Physics, 85 (1996), 1-23. doi: 10.1007/BF02175553. [33] M. Scheutzow, Attractors for ergodic and monotone random dynamical systems, in Seminar on Stochastic Analysis, Random Fields and Applications V (eds. R. C. Dalang, F. Russo and M. Dozzi), Birkhäuser Basel, Basel, 59 (2008), 331–344. doi: 10.1007/978-3-7643-8458-6_18. [34] J. Schnakenberg, Network theory of microscopic and macroscopic behavior of master equation systems, Rev. Mod. Phys., 48 (1976), 571-585. doi: 10.1103/RevModPhys.48.571. [35] U. Seifert, Entropy production along a stochastic trajectory and an integral fluctuation theorem, Phys. Rev. Lett., 95 (2005), 040602. doi: 10.1103/PhysRevLett.95.040602. [36] A. Swishchuk and S. Islam, Random Dynamical Systems in Finance, Chapman and Hall/CRC, New York, 2013. doi: 10.1201/b14989. [37] M. Tribus, Thermostatics and Thermodynamics: An Introduction to Energy, Information and States of Matter, with Engineering Applications, University series in basic engineering, Van Nostrand, 1961. [38] N. Van Kampen, Stochastic Processes in Physics and Chemistry, Lecture Notes in Mathematics, 888. North-Holland Publishing Co., Amsterdam-New York, 1981. [39] J. Voigt, Stochastic operators, information, and entropy, Communications in Mathematical Physics, 81 (1981), 31-38. doi: 10.1007/BF01941799. [40] F. X.-F. Ye, Y.-A. Ma and H. Qian, Estimate exponential memory decay in hidden Markov model and its applications, arXiv: 1710.06078. [41] F. X.-F. Ye, Y. Wang and H. Qian, Stochastic dynamics: Markov chains and random transformations, Discrete and Continuous Dynamical Systems - B, 21 (2016), 2337-2361. doi: 10.3934/dcdsb.2016050.

show all references

##### References:
 [1] H. Arbabi and I. Mezic, Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the Koopman operator, SIAM J. Applied Dynamical Systems, 16 (2017), 2096-2126. doi: 10.1137/17M1125236. [2] L. Arnold, Trends and open problems in the theory of random dynamical systems, in Probability Towards 2000 (eds. L. Accardi and C. C. Heyde), Springer New York, New York, NY, 1998, 34–46. doi: 10.1007/978-1-4612-2224-8_2. [3] L. Arnold, Random Dynamical Systems, Springer Monographs in Mathematics, Springer-Verlag, Berlin, 1998. doi: 10.1007/978-3-662-12878-7. [4] L. Arnold and H. Crauel, Random dynamical systems, in Lyapunov Exponents (Oberwolfach, 1990), vol. 1486 of Lecture Notes in Math., Springer, Berlin, 1991, 1–22. doi: 10.1007/BFb0086654. [5] V. Baladi, Positive Transfer Operators and Decay of Correlations, Advanced series in nonlinear dynamics, World Scientific Publishing Company, 2000. doi: 10.1142/9789812813633. [6] A. Ben-Naim, Entropy and the Second Law: Interpretation and Misss-interpretationsss, World Scientific, 2012. doi: 10.1142/8333. [7] P. G. Bergmann and J. L. Lebowitz, New approach to nonequilibrium processes, Phys. Rev., 99 (1955), 578-587. doi: 10.1103/PhysRev.99.578. [8] P. Bollobas, Modern Graph Theory, Graduate Texts in Mathematics, Springer New York, 1998. doi: 10.1007/978-1-4612-0619-4. [9] S. R. Caplan and D. Zeilberger, T. L. Hill's graphical method for solving linear equations, Advances in Applied Mathematics, 3 (1982), 377-383. doi: 10.1016/S0196-8858(82)80011-0. [10] R. T. Cox, The statistical method of gibbs in irreversible change, Rev. Mod. Phys., 22 (1950), 238-248. doi: 10.1103/RevModPhys.22.238. [11] P. Diaconis and D. Freedman, Iterated random functions, SIAM Review, 41 (1999), 45-76. doi: 10.1137/S0036144598338446. [12] P. Gaspard and X.-J. Wang, Noise, chaos, and ($\epsilon$, $\tau$)-entropy per unit time, Phys. Rep., 235 (1993), 291-343. doi: 10.1016/0370-1573(93)90012-3. [13] T. Hill, Free Energy Transduction and Biochemical Cycle Kinetics, Dover Books on Chemistry, Dover Publications, 2004. doi: 10.1007/978-1-4612-3558-3. [14] A. Hobson, A new theorem of information theory, Journal of Statistical Physics, 1 (1969), 383-391. doi: 10.1007/BF01106578. [15] W. Huang, H. Qian, S. Wang, F. X.-F. Ye and Y. Yi, Synchronization in discrete-time, discrete-state random dynamical systems, In preparation. [16] D.-Q. Jiang, M. Qian and M.-P. Qian, Mathematical Theory of Nonequilibrium Steady States: On the Frontier of Probability and Dynamical Systems, Lecture Notes in Mathematics, 1833. Springer-Verlag, Berlin, 2004. doi: 10.1007/b94615. [17] S. Kalpazidou, Cycle Representations of Markov Processes, Stochastic Modelling and Applied Probability, Springer, New York, 2006. [18] A. Khinchin, Mathematical Foundations of Information Theory, Translated by R. A. Silverman and M. D. Friedman. Dover Publications, Inc., New York, N. Y., 1957. [19] Y. Kifer, Random Perturbations of Dynamical Systems, Progress in Probability, Birkhäuser Boston, Inc., Boston, MA, 1988. doi: 10.1007/978-1-4615-8181-9. [20] E. L. King and C. Altman, A schematic method of deriving the rate laws for enzyme-catalyzed reactions, The Journal of Physical Chemistry, 60 (1956), 1375-1378. doi: 10.1021/j150544a010. [21] A. N. Kolmogorov, Three approaches to the quantitative definition of information, International Journal of Computer Mathematics, 2 (1968), 157-168. doi: 10.1080/00207166808803030. [22] A. Lasota and M. Mackey, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, Second edition. Applied Mathematical Sciences, 97. Springer-Verlag, New York, 1994. doi: 10.1007/978-1-4612-4286-4. [23] K. K. Lin, Stimulus-response reliability of biological networks, in Nonautonomous Dynamical Systems in the Life Sciences (eds. P. E. Kloeden and C. Pötzsche), Springer International Publishing, Cham, 2013,135–161. doi: 10.1007/978-3-319-03080-7_4. [24] J.-L. Luo, C. Van den Broeck and G. Nicolis, Stability criteria and fluctuations around nonequilibrium states, Zeitschrift für Physik B Condensed Matter, 56 (1984), 165–170. doi: 10.1007/BF01469698. [25] Y.-A. Ma, H. Qian and F. X.-F. Ye, Stochastic dynamics: Models for intrinsic and extrinsic noises and their applications (in Chinese), Scientia Sinica Mathematica, 47 (2017), 1693-1702. [26] D. Mumford, The dawning age of stochasticity, in Mathematics: Frontiers and Perspectives (eds. V. Arnold, M. Atiyah, P. Lax and B. Mazur), 197–218, Amer. Math. Soc., Providence, RI, 2000. [27] J. G. Propp and D. B. Wilson, Exact sampling with coupled Markov chains and applications to statistical mechanics, Random Structures & Algorithms, 9 (1996), 223-252. doi: 10.1002/(SICI)1098-2418(199608/09)9:1/2<223::AID-RSA14>3.0.CO;2-O. [28] H. Qian, A decomposition of irreversible diffusion processes without detailed balance, Journal of Mathematical Physics, 54 (2013), 053302, 10pp. doi: 10.1063/1.4803847. [29] H. Qian, Nonlinear stochastic dynamics of complex systems, i: a chemical reaction kinetic perspective with mesoscopic nonequilibrium thermodynamics, arXiv: 1605.08070. [30] A. N. Quas, On representations of Markov chains by random smooth maps, Bulletin of the London Mathematical Society, 23 (1991), 487-492. doi: 10.1112/blms/23.5.487. [31] H. Risken, The Fokker-Planck Equation: Methods of Solution and Applications, Second edition. Springer Series in Synergetics, 18. Springer-Verlag, Berlin, 1989. doi: 10.1007/978-3-642-96807-5. [32] D. Ruelle, Positivity of entropy production in nonequilibrium statistical mechanics, Journal of Statistical Physics, 85 (1996), 1-23. doi: 10.1007/BF02175553. [33] M. Scheutzow, Attractors for ergodic and monotone random dynamical systems, in Seminar on Stochastic Analysis, Random Fields and Applications V (eds. R. C. Dalang, F. Russo and M. Dozzi), Birkhäuser Basel, Basel, 59 (2008), 331–344. doi: 10.1007/978-3-7643-8458-6_18. [34] J. Schnakenberg, Network theory of microscopic and macroscopic behavior of master equation systems, Rev. Mod. Phys., 48 (1976), 571-585. doi: 10.1103/RevModPhys.48.571. [35] U. Seifert, Entropy production along a stochastic trajectory and an integral fluctuation theorem, Phys. Rev. Lett., 95 (2005), 040602. doi: 10.1103/PhysRevLett.95.040602. [36] A. Swishchuk and S. Islam, Random Dynamical Systems in Finance, Chapman and Hall/CRC, New York, 2013. doi: 10.1201/b14989. [37] M. Tribus, Thermostatics and Thermodynamics: An Introduction to Energy, Information and States of Matter, with Engineering Applications, University series in basic engineering, Van Nostrand, 1961. [38] N. Van Kampen, Stochastic Processes in Physics and Chemistry, Lecture Notes in Mathematics, 888. North-Holland Publishing Co., Amsterdam-New York, 1981. [39] J. Voigt, Stochastic operators, information, and entropy, Communications in Mathematical Physics, 81 (1981), 31-38. doi: 10.1007/BF01941799. [40] F. X.-F. Ye, Y.-A. Ma and H. Qian, Estimate exponential memory decay in hidden Markov model and its applications, arXiv: 1710.06078. [41] F. X.-F. Ye, Y. Wang and H. Qian, Stochastic dynamics: Markov chains and random transformations, Discrete and Continuous Dynamical Systems - B, 21 (2016), 2337-2361. doi: 10.3934/dcdsb.2016050.
Random maps are generated via this Markov chain. This is the illustration of state transition diagram for the Markov chain
A 3-state completely connected MC. The transition matrix of derived chain dynamics is in (34)
A 4-state MC with initial state 1
$e(\mathcal{T}_1) = M_{21}M_{31}, e(\mathcal{T}_2) = M_{12}M_{31}, e(\mathcal{T}_3) = M_{21}M_{13}$
The derived chain $\eta_t$ and the cycles formed for this sample trajectory $X_t$ [16]
 $t$ 0 1 2 3 4 5 6 7 8 $X_t(\omega)$ 2 1 3 1 3 1 1 2 2 $\eta_t(\omega)$ [2] [2, 1] [2, 1, 3] [2, 1] [2, 1, 3] [2, 1] [2, 1] [2] [2] $cycles$ (1, 3) (3, 1) (1) (1, 2) (2)
 $t$ 0 1 2 3 4 5 6 7 8 $X_t(\omega)$ 2 1 3 1 3 1 1 2 2 $\eta_t(\omega)$ [2] [2, 1] [2, 1, 3] [2, 1] [2, 1, 3] [2, 1] [2, 1] [2] [2] $cycles$ (1, 3) (3, 1) (1) (1, 2) (2)
 [1] Yujun Zhu. Preimage entropy for random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2007, 18 (4) : 829-851. doi: 10.3934/dcds.2007.18.829 [2] Deena Schmidt, Janet Best, Mark S. Blumberg. Random graph and stochastic process contributions to network dynamics. Conference Publications, 2011, 2011 (Special) : 1279-1288. doi: 10.3934/proc.2011.2011.1279 [3] Felix X.-F. Ye, Yue Wang, Hong Qian. Stochastic dynamics: Markov chains and random transformations. Discrete & Continuous Dynamical Systems - B, 2016, 21 (7) : 2337-2361. doi: 10.3934/dcdsb.2016050 [4] Daoyi Xu, Yumei Huang, Zhiguo Yang. Existence theorems for periodic Markov process and stochastic functional differential equations. Discrete & Continuous Dynamical Systems - A, 2009, 24 (3) : 1005-1023. doi: 10.3934/dcds.2009.24.1005 [5] Ralf Banisch, Carsten Hartmann. A sparse Markov chain approximation of LQ-type stochastic control problems. Mathematical Control & Related Fields, 2016, 6 (3) : 363-389. doi: 10.3934/mcrf.2016007 [6] Xiangnan He, Wenlian Lu, Tianping Chen. On transverse stability of random dynamical system. Discrete & Continuous Dynamical Systems - A, 2013, 33 (2) : 701-721. doi: 10.3934/dcds.2013.33.701 [7] Karsten Keller, Sergiy Maksymenko, Inga Stolz. Entropy determination based on the ordinal structure of a dynamical system. Discrete & Continuous Dynamical Systems - B, 2015, 20 (10) : 3507-3524. doi: 10.3934/dcdsb.2015.20.3507 [8] Ralf Banisch, Carsten Hartmann. Addendum to "A sparse Markov chain approximation of LQ-type stochastic control problems". Mathematical Control & Related Fields, 2017, 7 (4) : 623-623. doi: 10.3934/mcrf.2017023 [9] Zhiming Li, Lin Shu. The metric entropy of random dynamical systems in a Hilbert space: Characterization of invariant measures satisfying Pesin's entropy formula. Discrete & Continuous Dynamical Systems - A, 2013, 33 (9) : 4123-4155. doi: 10.3934/dcds.2013.33.4123 [10] Junyi Tu, Yuncheng You. Random attractor of stochastic Brusselator system with multiplicative noise. Discrete & Continuous Dynamical Systems - A, 2016, 36 (5) : 2757-2779. doi: 10.3934/dcds.2016.36.2757 [11] Samuel N. Cohen, Lukasz Szpruch. On Markovian solutions to Markov Chain BSDEs. Numerical Algebra, Control & Optimization, 2012, 2 (2) : 257-269. doi: 10.3934/naco.2012.2.257 [12] Bixiang Wang. Stochastic bifurcation of pathwise random almost periodic and almost automorphic solutions for random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2015, 35 (8) : 3745-3769. doi: 10.3934/dcds.2015.35.3745 [13] Mario Roy, Mariusz Urbański. Random graph directed Markov systems. Discrete & Continuous Dynamical Systems - A, 2011, 30 (1) : 261-298. doi: 10.3934/dcds.2011.30.261 [14] María J. Garrido–Atienza, Kening Lu, Björn Schmalfuss. Random dynamical systems for stochastic partial differential equations driven by a fractional Brownian motion. Discrete & Continuous Dynamical Systems - B, 2010, 14 (2) : 473-493. doi: 10.3934/dcdsb.2010.14.473 [15] Paulina Grzegorek, Michal Kupsa. Exponential return times in a zero-entropy process. Communications on Pure & Applied Analysis, 2012, 11 (3) : 1339-1361. doi: 10.3934/cpaa.2012.11.1339 [16] Lianfa He, Hongwen Zheng, Yujun Zhu. Shadowing in random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2005, 12 (2) : 355-362. doi: 10.3934/dcds.2005.12.355 [17] Philippe Marie, Jérôme Rousseau. Recurrence for random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2011, 30 (1) : 1-16. doi: 10.3934/dcds.2011.30.1 [18] Johnathan M. Bardsley. Gaussian Markov random field priors for inverse problems. Inverse Problems & Imaging, 2013, 7 (2) : 397-416. doi: 10.3934/ipi.2013.7.397 [19] Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Thermodynamic formalism for random countable Markov shifts. Discrete & Continuous Dynamical Systems - A, 2008, 22 (1&2) : 131-164. doi: 10.3934/dcds.2008.22.131 [20] Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Corrigendum to: Thermodynamic formalism for random countable Markov shifts. Discrete & Continuous Dynamical Systems - A, 2015, 35 (1) : 593-594. doi: 10.3934/dcds.2015.35.593

2017 Impact Factor: 0.972