2015, 20(10): 3301-3343. doi: 10.3934/dcdsb.2015.20.3301

On entropy, entropy-like quantities, and applications

1. 

Universidad Miguel Hernández, Centro de Investigación Operativa, Avda. Universidad s/n, Elche (Alicante), 03202, Spain

2. 

Universität zu Lübeck, Institut für Mathematik, Ratzeburger Allee 160, 23562 Lübeck

3. 

Graduate School for Computing in Medicine and Life Science, Universität zu Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany

Received  January 2015 Revised  March 2015 Published  September 2015

This is a review on entropy in various fields of mathematics and science. Its scope is to convey a unified vision of the classical entropies and some newer, related notions to a broad audience with an intermediate background in dynamical systems and ergodic theory. Due to the breadth and depth of the subject, we have opted for a compact exposition whose contents are a compromise between conceptual import and instrumental relevance. The intended technical level and the space limitation born furthermore upon the final selection of the topics, which cover the three items named in the title. Specifically, the first part is devoted to the avatars of entropy in the traditional contexts: many particle physics, information theory, and dynamical systems. This chronological order helps present the materials in a didactic manner. The axiomatic approach will be also considered at this stage to show that, quite remarkably, the essence of entropy can be encapsulated in a few basic properties. Inspired by the classical entropies, further akin quantities have been proposed in the course of time, mostly aimed at specific needs. A common denominator of those addressed in the second part of this review is their major impact on research. The final part shows that, along with its profound role in the theory, entropy has interesting practical applications beyond information theory and communications technology. For this sake we preferred examples from applied mathematics, although there are certainly nice applications in, say, physics, computer science and even social sciences. This review concludes with a representative list of references.
Citation: José M. Amigó, Karsten Keller, Valentina A. Unakafova. On entropy, entropy-like quantities, and applications. Discrete & Continuous Dynamical Systems - B, 2015, 20 (10) : 3301-3343. doi: 10.3934/dcdsb.2015.20.3301
References:
[1]

S. Abe, Tsallis entropy: How unique?,, Contin. Mech. Thermodyn., 16 (2004), 237. doi: 10.1007/s00161-003-0153-1.

[2]

U. R. Acharya, O. Faust, N. Kannathal, T. Chua and S. Laxminarayan, Non-linear analysis of EEG signals at various sleep stages,, Comput. Meth. Prog. Bio., 80 (2005), 37. doi: 10.1016/j.cmpb.2005.06.011.

[3]

U. R. Acharya, K. P. Joseph, N. Kannathal, C. M. Lim and J. S. Suri, Heart rate variability: A review,, Advances in Cardiac Signal Processing, (2007), 121. doi: 10.1007/978-3-540-36675-1_5.

[4]

J. Aczél and Z. Daróczy, Charakterisierung der Entropien positiver Ordnung und der Shannonschen Entropie,, Acta Math. Acad. Sci. Hung., 14 (1963), 95. doi: 10.1007/BF01901932.

[5]

R. Adler, A. Konheim and M. McAndrew, Topological entropy,, Trans. Amer. Mat. Soc., 114 (1965), 309. doi: 10.1090/S0002-9947-1965-0175106-9.

[6]

R. L. Adler and B. Marcus, Topological entropy and the equivalence of dynamical systems,, Mem. Amer. Math. Soc., 20 (1979). doi: 10.1090/memo/0219.

[7]

V. Afraimovich, M. Courbage and L. Glebsky, Directional complexity and entropy for lift mappings,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3385.

[8]

L. Alsedà, J. Llibre and M. Misiurewicz, Combinatorial Dynamics and Entropy in Dimension One,, World Scientific, (2000). doi: 10.1142/4205.

[9]

J. M. Amigó, L. Kocarev and J. Szczepanski, Order patterns and chaos,, Phys. Lett. A, 355 (2006), 27.

[10]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, True and false forbidden patterns in deterministic and random dynamics,, Eur. Phys. Lett., 79 (2007). doi: 10.1209/0295-5075/79/50001.

[11]

J. M. Amigó and M. B. Kennel, Forbidden ordinal patterns in higher dimensional dynamics,, Physica D, 237 (2008), 2893. doi: 10.1016/j.physd.2008.05.003.

[12]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, Combinatorial detection of determinism in noisy time series,, Europhys. Lett., 82 (2008).

[13]

J. M. Amigó, Permutation Complexity in Dynamical Systems-Ordinal Patterns, Permutation Entropy, and All That,, Springer Series in Synergetics, (2010). doi: 10.1007/978-3-642-04084-9.

[14]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, Detecting determinism in time series with ordinal patterns: A comparative study,, Int. J. Bif. Chaos, 20 (2010), 2915. doi: 10.1142/S0218127410027453.

[15]

J. M. Amigó, R. Monetti, T. Aschenbrenner and W. Bunk, Transcripts: An algebraic approach to coupled time series,, Chaos, 22 (2012). doi: 10.1063/1.3673238.

[16]

J. M. Amigó, The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized,, Physica D, 241 (2012), 789. doi: 10.1016/j.physd.2012.01.004.

[17]

J. M. Amigó and K. Keller, Permutation entropy: One concept, two approaches,, Eur. Phys. J. Special Topics, 222 (2013), 263.

[18]

J. M. Amigó, P. Kloeden and A. Giménez, Switching systems and entropy,, J. Diff. Eq. Appl., 19 (2013), 1872. doi: 10.1080/10236198.2013.788166.

[19]

J. M. Amigó, P. E. Kloeden and A. Giménez, Entropy increase in switching systems,, Entropy, 15 (2013), 2363. doi: 10.3390/e15062363.

[20]

J. M. Amigó, T. Aschenbrenner, W. Bunk and R. Monetti, Dimensional reduction of conditional algebraic multi-information via transcripts,, Inform. Sciences, 278 (2014), 298. doi: 10.1016/j.ins.2014.03.054.

[21]

J. M. Amigó and A. Giménez, A simplified algorithm for the topological entropy of multimodal maps,, Entropy, 16 (2014), 627. doi: 10.3390/e16020627.

[22]

J. M. Amigó, K. Keller and V. A. Unakafova, Ordinal symbolic analysis and its application to biomedical recordings},, Phil. Trans. R. Soc. A, 373 (2015). doi: 10.1098/rsta.2014.0091.

[23]

J. M. Amigó and A. Giménez, Formulas for the topological entropy of multimodal maps based on min-max symbols,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3415.

[24]

R. G. Andrzejak, K. Lehnertz, F. Mormann, C. Rieke, P. David and C. E. Elger, Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state,, Phys. Rev. E, 64 (2001). doi: 10.1103/PhysRevE.64.061907.

[25]

A. Antoniouk, K. Keller and S. Maksymenko, Kolmogorov-Sinai entropy via separation properties of order-generated sigma-algebras,, Discr. Contin. Dyn. Syst. A, 34 (2014), 1793.

[26]

R. B. Ash, Information Theory,, Dover Publications, (1990).

[27]

C. Bandt, G. Keller and B. Pompe, Entropy of interval maps via permutations,, Nonlinearity, 15 (2002), 1595. doi: 10.1088/0951-7715/15/5/312.

[28]

E. Beadle, J. Schroeder, B Moran and S. Suvorova, An overview of Rényi Entropy and some potential applications,, in 42nd Asilomar Conference on Signals, (2008), 1698. doi: 10.1109/ACSSC.2008.5074715.

[29]

C. H. Bennet, Notes on Landauer's principle, reversible computation and Maxwell's demon,, Stud. Hist. Philos. M. P., 34 (2003), 501. doi: 10.1016/S1355-2198(03)00039-X.

[30]

G. D. Birkhoff, Proof of a recurrence theorem for strongly transitive systems,, PNAS, 17 (1931), 650. doi: 10.1073/pnas.17.12.650.

[31]

S. A. Borovkova, Estimation and Prediction for Nonlinear Time Series,, Ph.D thesis, (1998).

[32]

M. Boyle and D. Lind, Expansive subdynamics,, Trans. Amer. Math. Soc., 349 (1997), 55. doi: 10.1090/S0002-9947-97-01634-6.

[33]

R. Bowen, Entropy for group endomorphisms and homogeneous spaces,, Trans. Amer. Math. Soc., 153 (1971), 401. doi: 10.1090/S0002-9947-1971-0274707-X.

[34]

L. Bowen, A measure-conjugacy invariant for free group actions,, Ann. of Math., 171 (2010), 1387. doi: 10.4007/annals.2010.171.1387.

[35]

L. Bowen, Measure-conjugacy invariants for actions of countable sofic groups,, J. Amer. Math. Soc., 23 (2010), 217. doi: 10.1090/S0894-0347-09-00637-7.

[36]

H. W. Broer and F. Takens, Dynamical Systems and Chaos,, Applied Mathematical Sciences, (2011). doi: 10.1007/978-1-4419-6870-8.

[37]

M. Brin and A. Katok, On local entropy,, in Geometric dynamics (ed. J. Palis), (1007), 30. doi: 10.1007/BFb0061408.

[38]

A. A. Bruzzo, B. Gesierich, M. Santi, C. A. Tassinari, N. Birbaumer and G. Rubboli, Permutation entropy to detect vigilance changes and preictal states from scalp EEG in epileptic patients. A preliminary study,, Neurol. Sci., 29 (2008), 3. doi: 10.1007/s10072-008-0851-3.

[39]

N. Burioka, M. Miyata, G. Cornélissen, F. Halberg, T. Takeshima, D. T. Kaplan, H. Suyama, M. Endo, Y. Maegaki and T. Nomura, et. al, Approximate entropy in the electroencephalogram during wake and sleep,, Clin. EEG Neurosci., 36 (2005), 21. doi: 10.1177/155005940503600106.

[40]

C. Cafaro, W. M. Lord, J. Sun and E. M. Bollt, Causation entropy from symbolic representations of dynamical systems,, Chaos, 25 (2015).

[41]

J. S. Cánovas and A. Guillamón, Permutations and time series analysis,, Chaos, 19 (2009). doi: 10.1063/1.3238256.

[42]

Y. Cao, W.-W. Tung, J. B. Gao, V. A. Protopopescu and L. M. Hively, Detecting dynamical changes in time series using the permutation entropy,, Phys. Rev. E, 70 (2004). doi: 10.1103/PhysRevE.70.046217.

[43]

A. Capurro, L. Diambra, D. Lorenzo, O. Macadar, M. T. Martin, C. Mostaccio, A. Plastino, E. Rofman, M. E. Torres and J. Velluti, Tsallis entropy and cortical dynamics: The analysis of EEG signals,, Physica A, 257 (1998), 149. doi: 10.1016/S0378-4371(98)00137-X.

[44]

A. Capurro, L. Diambra, D. Lorenzo, O. Macadar, M. T. Martin, C. Mostaccio, A. Plastino, J. Perez, E. Rofman and M. E. Torres, Human brain dynamics: The analysis of EEG signals with Tsallis information measure,, Physica A, 265 (1999), 235. doi: 10.1016/S0378-4371(98)00471-3.

[45]

D. Carrasco-Olivera, R. Metzger and C. A. Morales, Topological entropy for set-valued maps,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3461.

[46]

H. C. Choe, Computational Ergodic Theory,, Algorithms and Computation in Mathematics, (2005).

[47]

R. Clausius, The Mechanical Theory of Heat,, MacMillan and Co., (1865).

[48]

M. Courbage and B. Kamiński, Space-time directional Lyapunov exponents for cellular automata,, J. Statis. Phys., 124 (2006), 1499. doi: 10.1007/s10955-006-9172-1.

[49]

T. M. Cover and J. A. Thomas, Elements of Information Theory,, John Wiley & Sons, (2006).

[50]

B. Dai and B. Hu, Minimum conditional entropy clustering: A discriminative framework for clustering,, in JMLR: Workshop and Conference Proceedings, (2010), 47.

[51]

K. Denbigh, How subjective is entropy,, in Maxwell's Demon, (1990), 109.

[52]

M. Denker, Finite generators for ergodic, measure-preserving transformation,, Zeit. Wahr. ver. Geb., 29 (1974), 45. doi: 10.1007/BF00533186.

[53]

M. Denker, C. Grillenberger and K. Sigmund, Ergodic Theory on Compact Spaces,, Lecture Notes in Mathematics, (1976).

[54]

M. Denker and G. Keller, Rigorous statistical procedures for data from dynamical systems,, J. Stat. Phys., 44 (1986), 67. doi: 10.1007/BF01010905.

[55]

E. I. Dinaburg, The relation between topological entropy and metric entropy,, Soviet Math., 11 (1970), 13.

[56]

R. J. V. dos Santos, Generalization of Shannon's theorem for Tsallis entropy,, J. Math. Phys., 38 (1997), 4104. doi: 10.1063/1.532107.

[57]

J.-P. Eckmann and D. Ruelle, Ergodic theory of chaos and strange attractors,, Rev. Mod. Phys., 57 (1985), 617. doi: 10.1103/RevModPhys.57.617.

[58]

M. Einsiedler and T. Ward, Ergodic Theory with a View Towards Number Theory,, Graduate Texts in Mathematics, (2011). doi: 10.1007/978-0-85729-021-2.

[59]

D. K. Faddeev, On the notion of entropy of finite probability distributions,, Usp. Mat. Nauk (in Russian), 11 (1956), 227.

[60]

F. Falniowski, On the connections of the generalized entropies and Kolmogorov-Sinai entropies,, Entropy, 16 (2014), 3732. doi: 10.3390/e16073732.

[61]

B. Frank, B. Pompe, U. Schneider and D. Hoyer, Permutation entropy improves fetal behavioural state classification based on heart rate analysis from biomagnetic recordings in near term fetuses,, Med. Biol. Eng. Comput., 44 (2006), 179. doi: 10.1007/s11517-005-0015-z.

[62]

S. Furuichi, On uniqueness Theorems for Tsallis entropy and Tsallis relative entropy,, IEEE Trans. Inf. Theory, 51 (2005), 3638. doi: 10.1109/TIT.2005.855606.

[63]

L. G. Gamero, A. Plastino and M. E. Torres, Wavelet analysis and nonlinear dynamics in a nonextensive setting,, Physica A, 246 (1997), 487. doi: 10.1016/S0378-4371(97)00367-1.

[64]

P. G. Gaspard and X. J. Wang, Noise, chaos, and $(\varepsilon,\tau)$-entropy per unit time,, Phys. Rep., 235 (1993), 291. doi: 10.1016/0370-1573(93)90012-3.

[65]

T. N. T. Goodman, Relating topological entropy and measure entropy,, Bull. London Math. Soc., 3 (1971), 176. doi: 10.1112/blms/3.2.176.

[66]

N. Gradojevic and R. Genccay, Financial applications of nonextensive entropy,, IEEE Signal Process. Mag., 28 (2011).

[67]

B. Graff, G. Graff and A. Kaczkowska, Entropy Measures of Heart Rate Variability for Short ECG Datasets in Patients with Congestive Heart Failure,, Acta Phys. Pol. B Proc. Suppl., 5 (2012), 153.

[68]

C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods,, Econometrica, 37 (1969), 424.

[69]

P. Grassberger and I. Procaccia, Dimensions and entropies of strange attractors from a fluctuating dynamics approach,, Physica D, 13 (1984), 34. doi: 10.1016/0167-2789(84)90269-0.

[70]

P. Grassberger and I. Procaccia, Measuring the strangeness of strange attractors,, Physica D, 9 (1983), 189. doi: 10.1016/0167-2789(83)90298-1.

[71]

P. Grassberger and I. Procaccia, Characterization of strange attractors,, Phys. Rev. Lett., 50 (1983), 346. doi: 10.1103/PhysRevLett.50.346.

[72]

D. W. Hahs and S. D. Pethel, Distinguishing anticipation from causality: Anticipatory bias in the estimation of information flow,, Phys. Rev. Lett., 107 (2011).

[73]

R. Hanel and S. Thurner, A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and equidistribution functions,, EPL, 93 (2011).

[74]

R. Hanel, S. Thurner and M. Gell-Mann, Generalized entropies and logarithms and their duality relations,, Proceed. Nat. Acad. Scie., 109 (2012), 19151. doi: 10.1073/pnas.1216885109.

[75]

B. Hasselblatt and A. Katok, Principal structures,, in Handbook of Dynamical Systems (eds. B. Hasselblatt and A. Katok), (2002), 1. doi: 10.1016/S1874-575X(02)80003-0.

[76]

J. Havrda and F. Charvát, Quantification method of classification processes. Concept of structural $\alpha$-entropy,, Kybernetika, 3 (1967), 30.

[77]

G. A. Hedlund, Endomorphisms and automorphisms of the shift dynamical system,, Math. Syst. Theory, 3 (1969), 320. doi: 10.1007/BF01691062.

[78]

H. G. E. Hentschel and I. Procaccia, The infinite number of generalized dimensions of fractals and strange attractors,, Physica D, 8 (1983), 435. doi: 10.1016/0167-2789(83)90235-X.

[79]

R. Hornero, D. Abasolo, J. Escuredo and C. Gomez, Nonlinear analysis of electroencephalogram and magnetoencephalogram recordings in patients with Alzheimer's disease,, Phil. Trans. R. Soc. A, 367 (2009), 317. doi: 10.1098/rsta.2008.0197.

[80]

V. M. Ilić, M. S. Stanković and E. H. Mulalić, Comments on "Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy'',, IEEE Trans. Inf. Theory, 59 (2013), 6950.

[81]

E. T. Jaynes, Information theory and statistical mechanics,, Phys. Rev., 106 (1957), 620. doi: 10.1103/PhysRev.106.620.

[82]

P. Jizba and T. Arimitsu, The world according to Rényi: Thermodynamics of multifractal systems,, Ann. Phys., 312 (2004), 17. doi: 10.1016/j.aop.2004.01.002.

[83]

C. C. Jouny and G. K. Bergey, Characterization of early partial seizure onset: Frequency, complexity and entropy,, Clin. Neurophysiol., 123 (2012), 658. doi: 10.1016/j.clinph.2011.08.003.

[84]

N. Kannathal, M. L. Choo, U. R. Acharya and P. K. Sadasivan, Entropies for detection of epilepsy in EEG,, Comput. Meth. Prog. Bio., 80 (2005), 187. doi: 10.1016/j.cmpb.2005.06.012.

[85]

H. Kantz and T. Schreiber, Nonlinear Time Series Analysis,, Cambridge University Press, (2004).

[86]

A. Katok, Fifty years of entropy in dynamics: 1978-2007,, J. Mod. Dyn., 1 (2007), 545. doi: 10.3934/jmd.2007.1.545.

[87]

K. Keller and H. Lauffer, Symbolic analysis of high-dimensional time series,, Int. J. Bif. Chaos, 13 (2003), 2657. doi: 10.1142/S0218127403008168.

[88]

K. Keller, Permutations and the Kolmogorov-Sinai entropy,, Discr. Contin. Dyn. Syst. A, 32 (2012), 891. doi: 10.3934/dcds.2012.32.891.

[89]

K. Keller, A. M. Unakafov and V. A. Unakafova, Ordinal Patterns, Entropy, and EEG,, Entropy, 16 (2014), 6212. doi: 10.3390/e16126212.

[90]

K. Keller, S. Maksymenko and I. Stolz, Entropy determination based on the ordinal structure of a dynamical system,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3507.

[91]

A. I. Khinchin, Mathematical Foundations of Information Theory,, Dover, (1957).

[92]

A. N. Kolmogorov, A new metric invariant of transitive dynamical systems and Lebesgue space endomorphisms,, Dokl. Acad. Sci. USSR, 119 (1958), 861.

[93]

S. Kolyada and L. Snoha, Topological entropy of nonautonomous dynamical systems,, Random Comput. Dynam., 4 (1996), 205.

[94]

Z. S. Kowalski, Finite generators of ergodic endomorphisms,, Colloq. Math., 49 (1984), 87.

[95]

Z. S. Kowalski, Minimal generators for aperiodic endomorphisms,, Commentat. Math. Univ. Carol., 36 (1995), 721.

[96]

W. Krieger, On entropy and generators of measure-preserving transformations,, Trans. Amer. Math. Soc., 149 (1970), 453. doi: 10.1090/S0002-9947-1970-0259068-3.

[97]

J. Kurths, A. Voss, P. Saparin, A. Witt, H. J. Kleiner and N. Wessel, Quantitative analysis of heart rate variability,, Chaos, 5 (1995), 88. doi: 10.1063/1.166090.

[98]

D. E. Lake, J. S. Richman, M. P. Griffin and J. R. Moorman, Sample entropy analysis of neonatal heart rate variability,, Am. J. Physiol.-Reg. I., 283 (2002), 789. doi: 10.1152/ajpregu.00069.2002.

[99]

A. M. Law and W. D. Kelton, Simulation, Modeling, and Analysis,, McGraw-Hill, (2000).

[100]

F. Ledrappier and L.-S. Young, The metric entropy of diffeomorphisms. II. Relations betweenentropy, exponents and dimension,, Ann. Math., 122 (1985), 540. doi: 10.2307/1971329.

[101]

X. Li, G. Ouyang and D. A. Richards, Predictability analysis of absence seizures with permutation entropy,, Epilepsy Res., 77 (2007), 70. doi: 10.1016/j.eplepsyres.2007.08.002.

[102]

J. Li, J. Yan, X. Liu and G. Ouyang, Using permutation entropy to measure the changes in eeg signals during absence seizures,, Entropy, 16 (2014), 3049. doi: 10.3390/e16063049.

[103]

H. Li, K. Zhang and T. Jiang, Minimum entropy clustering and applications to gene expression analysis,, in Proc. IEEE Comput. Syst. Bioinform. Conf., (2004), 142.

[104]

Z. Liang, Y. Wang, X. Sun, D. Li, L. J. Voss, J. W. Sleigh, S. Hagihira and X. Li, EEG entropy measures in anesthesia,, Front. Comput. Neurosci., 9 (2015). doi: 10.3389/fncom.2015.00016.

[105]

D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding,, Cambridge University Press, (1995). doi: 10.1017/CBO9780511626302.

[106]

J. Llibre, Brief survey on the topological entropy,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3363.

[107]

N. Mammone, F. La Foresta and F. C. Morabito, Automatic artifact rejection from multichannel scalp EEG by wavelet ICA,, IEEE Sens. J., 12 (2012), 533. doi: 10.1109/JSEN.2011.2115236.

[108]

M. Matilla-García, A non-parametric test for independence based on symbolic dynamics,, J. Econ. Dyn. Control, 31 (2007), 3889. doi: 10.1016/j.jedc.2007.01.018.

[109]

M. Matilla-García and M. Ruiz Marín, A non-parametric independence test using permutation entropy,, J. Econ., 144 (2008), 139. doi: 10.1016/j.jeconom.2007.12.005.

[110]

A. M. Mesón and F. Vericat, On the Kolmogorov-like generalization of Tsallis entropy, correlation entropies and multifractal analysis,, J. Math. Phys., 43 (2002), 904. doi: 10.1063/1.1429323.

[111]

R. Miles and T. Ward, Directional uniformities, periodic points, and entropy,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3525.

[112]

J. Milnor, On the entropy geometry of cellular automata,, Complex Systems, 2 (1988), 357.

[113]

J. Milnor and W. Thurston, On iterated maps of the interval,, in Dynamical Systems (ed. J. C. Alexander), (1342), 465. doi: 10.1007/BFb0082847.

[114]

M. Misiurewicz and W. Szlenk, Entropy of piecewise monotone mappings,, Studia Math., 67 (1980), 45.

[115]

M. Misiurewicz, Permutations and topological entropy for interval maps,, Nonlinearity, 16 (2003), 971. doi: 10.1088/0951-7715/16/3/310.

[116]

R. Monetti, J. M. Amigó, T. Aschenbrenner and W. Bunk, Permutation complexity of interacting dynamical systems,, Eur. Phys. J. Special Topics, 222 (2013), 421. doi: 10.1140/epjst/e2013-01850-y.

[117]

R. Monetti, W. Bunk, T. Aschenbrenner, S. Springer and J. M. Amigó, Information directionality in coupled time series using transcripts,, Phys. Rev. E, 88 (2013). doi: 10.1103/PhysRevE.88.022911.

[118]

F. C. Morabito, D. Labate, F. La Foresta, A. Bramanti, G. Morabito and I. Palamara, Multivariate multi-scale permutation entropy for complexity analysis of Alzheimer's disease EEG,, Entropy, 14 (2012), 1186. doi: 10.3390/e14071186.

[119]

N. Nicolaou and J. Georgiou, The use of permutation entropy to characterize sleep electroencephalograms,, Clin. EEG Neurosci., 42 (2011), 24. doi: 10.1177/155005941104200107.

[120]

D. Ornstein, Two Bernoulli shifts with the same entropy are isomorphic,, Adv. Math., 4 (1970), 337. doi: 10.1016/0001-8708(70)90029-0.

[121]

D. Ornstein, Two Bernoulli shifts with infinite entropy are isomorphic,, Adv. Math., 5 (1970), 339. doi: 10.1016/0001-8708(70)90008-3.

[122]

D. Ornstein and B. Weiss, Entropy and isomorphism theorem for actions of amenable groups,, J. Analyse Math., 48 (1987), 1. doi: 10.1007/BF02790325.

[123]

D. Ornstein, Newton's law and coin tossing,, Notices Amer. Math. Soc., 60 (2013), 450. doi: 10.1090/noti974.

[124]

D. Ornstein and B. Weiss, Entropy is the only finitely observable invariant,, J. Mod. Dyn., 1 (2007), 93.

[125]

G. Ouyang, C. Dang, D. A. Richards and X. Li, Ordinal pattern based similarity analysis for EEG recordings,, Clin. Neurophysiol, 121 (2010), 694. doi: 10.1016/j.clinph.2009.12.030.

[126]

S. Y. Park and A. K. Bera, Maximum entropy autoregressive conditional heretoskedasticity model,, J. Econometrics, 150 (2009), 219. doi: 10.1016/j.jeconom.2008.12.014.

[127]

U. Parlitz, S. Berg, S. Luther, A. Schirdewan, J. Kurths and N. Wessel, Classifying cardiac biosignals using ordinal pattern statistics and symbolic dynamics,, Comput. Biol. Med., 42 (2012), 319. doi: 10.1016/j.compbiomed.2011.03.017.

[128]

W. Parry, Intrinsic Markov chains,, Trans. Amer. Math. Soc., 112 (1964), 55. doi: 10.1090/S0002-9947-1964-0161372-1.

[129]

Y. Pesin, Characteristic Lyapunov exponents and smooth ergodic theory,, Russian Math. Surveys, 32 (1977), 55.

[130]

S. M. Pincus, Approximate entropy as a measure of system complexity,, PNAS, 88 (1991), 2297. doi: 10.1073/pnas.88.6.2297.

[131]

S. M. Pincus and R. R. Viscarello, Approximate entropy: A regularity measure for fetal heart rate analysis,, Obstet. Gynecol., 79 (1992), 249.

[132]

S. M. Pincus, Approximate entropy as a measure of irregularity for psychiatric serial metrics,, Bipolar Disord., 8 (2006), 430. doi: 10.1111/j.1399-5618.2006.00375.x.

[133]

B. Pompe and J. Runge, Momentary information transfer as a coupling measure of time series,, Phys. Rev. E, 83 (2011). doi: 10.1103/PhysRevE.83.051122.

[134]

J. Poza, R. Hornero, J. Escudero, A. Fernández and C. A. Sánchez, Regional analysis of spontaneous MEG rhythms in patients with Alzheimer's disease using spectral entropies,, Ann. Biomed. Eng., 36 (2008), 141. doi: 10.1007/s10439-007-9402-y.

[135]

J. C. Principe, Information Theoretic Learning. Renyi's Entropy and Kernel Perspectives,, Springer, (2010). doi: 10.1007/978-1-4419-1570-2.

[136]

A. Rényi, On measures of entropy and information,, in Proceedings of the 4th Berkeley Symposium on Mathematics, (1961), 547.

[137]

J. S. Richman and J. R. Moorman, Physiological time-series analysis using approximate entropy and sample entropy,, Am. J. Physiol.-Heart C., 278 (2000), 2039.

[138]

S. J. Roberts, R. Everson and I. Rezek, Minimum entropy data partitioning,, in Proceedings of International Conference on Artificial Neural Networks, (1999), 844.

[139]

E. A. Robinson and A. Şahin, Rank-one $\mathbbZ^d$ actions and directional entropy,, Ergod. Theor. Dynam. Syst., 31 (2011), 285. doi: 10.1017/S0143385709000911.

[140]

O. A. Rosso, M. T. Martin and A. Plastino, Brain electrical activity analysis using wavelet-based informational tools,, Physica A, 313 (2002), 587. doi: 10.1016/S0378-4371(02)00958-5.

[141]

D. Rudolph and B. Weiss, Entropy and mixing for amenable group actions,, Ann. of Math., 151 (2000), 1119. doi: 10.2307/121130.

[142]

D. Ruelle, Statistical mechanics on a compact set with $Z^{\nu}$ action satisfying expansiveness and specification,, Trans. Amer. Math. Soc., 187 (1973), 237.

[143]

J. Runge, J. Heitzig, N. Marwan and J. Kurths, Quantifying causal coupling strength: A lag-specific measure for multivariate time series related to transfer entropy,, Phys. Rev. E, 86 (2012).

[144]

X. San Liang, Unraveling the cause-effect relation between time series,, Phys. Rev. E, 90 (2014).

[145]

T. Schreiber, Measuring information transfer,, Phys. Rev. Lett., 85 (2000), 461. doi: 10.1103/PhysRevLett.85.461.

[146]

C. E. Shannon, A mathematical theory of communication,, Bell Syst. Tech. J., 27 (1948), 379. doi: 10.1002/j.1538-7305.1948.tb01338.x.

[147]

Y. G. Sinai, On the notion of entropy of dynamical systems,, Dokl. Acad. Sci. USSR, 125 (1959), 768.

[148]

Y. G. Sinai, Flows with finite entropy,, Dokl. Acad. Sci. USSR, 125 (1959), 1200.

[149]

Y. G. Sinai, Gibbs measures in ergodic theory,, Uspehi Mat. Nauk, 27 (1972), 21.

[150]

D. Smirnov, Spurious causalities with transfer entropy,, Phys. Rev. E, 87 (2013).

[151]

M. Smorodinsky, Ergodic Theory, Entropy,, Lectures Notes in Mathematics, (1971).

[152]

R. Sneddon, The Tsallis entropy of natural information,, Physica A, 386 (2007), 101. doi: 10.1016/j.physa.2007.05.065.

[153]

V. Srinivasan, C. Eswaran and N. Sriraam, Approximate entropy-based epileptic EEG detection using artificial neural networks,, IEEE T. Inf. Technol. B., 11 (2007), 288. doi: 10.1109/TITB.2006.884369.

[154]

H. Suyari, Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy,, IEEE T. Inform. Theory, 50 (2004), 1783. doi: 10.1109/TIT.2004.831749.

[155]

F. Takens, Detecting strange attractors in turbulence,, in Dynamical Systems and Turbulence (eds. D. A. Rand and L. S. Young), (1981), 366.

[156]

F. Takens and E. Verbitskiy, Rényi entropies of aperiodic dynamical systems,, Israel J. Math., 127 (2002), 279. doi: 10.1007/BF02784535.

[157]

F. Takens and E. Verbitskiy, Generalized entropies: Rényi and correlation integral approach,, Nonlinearity, 11 (1998), 771. doi: 10.1088/0951-7715/11/4/001.

[158]

S. Tong, A. Bezerianos, A. Malhotra, Y. Zhu and N. Thakor, Parameterized entropy analysis of EEG following hypoxicischemic brain injury,, Phys. Lett. A, 314 (2003), 354. doi: 10.1016/S0375-9601(03)00949-6.

[159]

M. E. Torres and L. G. Gamero, Relative complexity changes in time series using information measures,, Physica A, 286 (2000), 457. doi: 10.1016/S0378-4371(00)00309-5.

[160]

B. Tóthmérész, Comparison of different methods for diversity ordering,, J. Veg. Sci., 6 (1995), 283.

[161]

C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics,, J. Stat. Phys., 52 (1988), 479. doi: 10.1007/BF01016429.

[162]

C. Tsallis, The nonadditive entropy Sq and its applications in physics and elsewhere: Some remarks,, Entropy, 13 (2011), 1765. doi: 10.3390/e13101765.

[163]

A. M. Unakafov and K. Keller, Conditional entropy of ordinal patterns,, Physica D, 269 (2014), 94. doi: 10.1016/j.physd.2013.11.015.

[164]

V. A. Unakafova, Investigating Measures of Complexity for Dynamical Systems and for Time Series,, Ph.D thesis, (2015).

[165]

E. A. Verbitskiy, Generalized Entropies in Dynamical Systems,, Ph.D thesis, (2000).

[166]

A. Voss, S. Schulz, R. Schroeder, M. Baumert and P. Caminal, Methods derived from nonlinear dynamics for analysing heart rate variability,, Phil. Trans. R. Soc. A, 367 (2009), 277. doi: 10.1098/rsta.2008.0232.

[167]

P. Walters, An Introduction to Ergodic Theory,, Springer Verlag, (1982).

[168]

B. Weiss, Subshifts of finite type and sofic systems,, Monats. Math., 77 (1973), 462. doi: 10.1007/BF01295322.

[169]

B. Weiss, Entropy and actions of sofic groups,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3375.

[170]

N. Wiener, The theory of prediction,, in Modern Mathematics for the Engineer (ed. E. F. Beckenbach), (1956).

[171]

A. D. Wissner-Gross and C. E. Freer, Causal entropic forces,, Phys. Rev. Lett., 110 (2013). doi: 10.1103/PhysRevLett.110.168702.

[172]

M. Zanin, L. Zunino, O. A. Rosso and D. Papo, Permutation entropy and its main biomedical and econophysics applications: A review,, Entropy, 14 (2012), 1553. doi: 10.3390/e14081553.

[173]

D. Zhang, X. Jia, H. Ding, D. Ye and N. V. Thakor, Application of Tsallis entropy to EEG: Quantifying the presence of burst suppression after asphyxial cardiac arrest in rats,, IEEE Trans. Bio.-Med. Eng., 57 (2010), 867.

show all references

References:
[1]

S. Abe, Tsallis entropy: How unique?,, Contin. Mech. Thermodyn., 16 (2004), 237. doi: 10.1007/s00161-003-0153-1.

[2]

U. R. Acharya, O. Faust, N. Kannathal, T. Chua and S. Laxminarayan, Non-linear analysis of EEG signals at various sleep stages,, Comput. Meth. Prog. Bio., 80 (2005), 37. doi: 10.1016/j.cmpb.2005.06.011.

[3]

U. R. Acharya, K. P. Joseph, N. Kannathal, C. M. Lim and J. S. Suri, Heart rate variability: A review,, Advances in Cardiac Signal Processing, (2007), 121. doi: 10.1007/978-3-540-36675-1_5.

[4]

J. Aczél and Z. Daróczy, Charakterisierung der Entropien positiver Ordnung und der Shannonschen Entropie,, Acta Math. Acad. Sci. Hung., 14 (1963), 95. doi: 10.1007/BF01901932.

[5]

R. Adler, A. Konheim and M. McAndrew, Topological entropy,, Trans. Amer. Mat. Soc., 114 (1965), 309. doi: 10.1090/S0002-9947-1965-0175106-9.

[6]

R. L. Adler and B. Marcus, Topological entropy and the equivalence of dynamical systems,, Mem. Amer. Math. Soc., 20 (1979). doi: 10.1090/memo/0219.

[7]

V. Afraimovich, M. Courbage and L. Glebsky, Directional complexity and entropy for lift mappings,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3385.

[8]

L. Alsedà, J. Llibre and M. Misiurewicz, Combinatorial Dynamics and Entropy in Dimension One,, World Scientific, (2000). doi: 10.1142/4205.

[9]

J. M. Amigó, L. Kocarev and J. Szczepanski, Order patterns and chaos,, Phys. Lett. A, 355 (2006), 27.

[10]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, True and false forbidden patterns in deterministic and random dynamics,, Eur. Phys. Lett., 79 (2007). doi: 10.1209/0295-5075/79/50001.

[11]

J. M. Amigó and M. B. Kennel, Forbidden ordinal patterns in higher dimensional dynamics,, Physica D, 237 (2008), 2893. doi: 10.1016/j.physd.2008.05.003.

[12]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, Combinatorial detection of determinism in noisy time series,, Europhys. Lett., 82 (2008).

[13]

J. M. Amigó, Permutation Complexity in Dynamical Systems-Ordinal Patterns, Permutation Entropy, and All That,, Springer Series in Synergetics, (2010). doi: 10.1007/978-3-642-04084-9.

[14]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, Detecting determinism in time series with ordinal patterns: A comparative study,, Int. J. Bif. Chaos, 20 (2010), 2915. doi: 10.1142/S0218127410027453.

[15]

J. M. Amigó, R. Monetti, T. Aschenbrenner and W. Bunk, Transcripts: An algebraic approach to coupled time series,, Chaos, 22 (2012). doi: 10.1063/1.3673238.

[16]

J. M. Amigó, The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized,, Physica D, 241 (2012), 789. doi: 10.1016/j.physd.2012.01.004.

[17]

J. M. Amigó and K. Keller, Permutation entropy: One concept, two approaches,, Eur. Phys. J. Special Topics, 222 (2013), 263.

[18]

J. M. Amigó, P. Kloeden and A. Giménez, Switching systems and entropy,, J. Diff. Eq. Appl., 19 (2013), 1872. doi: 10.1080/10236198.2013.788166.

[19]

J. M. Amigó, P. E. Kloeden and A. Giménez, Entropy increase in switching systems,, Entropy, 15 (2013), 2363. doi: 10.3390/e15062363.

[20]

J. M. Amigó, T. Aschenbrenner, W. Bunk and R. Monetti, Dimensional reduction of conditional algebraic multi-information via transcripts,, Inform. Sciences, 278 (2014), 298. doi: 10.1016/j.ins.2014.03.054.

[21]

J. M. Amigó and A. Giménez, A simplified algorithm for the topological entropy of multimodal maps,, Entropy, 16 (2014), 627. doi: 10.3390/e16020627.

[22]

J. M. Amigó, K. Keller and V. A. Unakafova, Ordinal symbolic analysis and its application to biomedical recordings},, Phil. Trans. R. Soc. A, 373 (2015). doi: 10.1098/rsta.2014.0091.

[23]

J. M. Amigó and A. Giménez, Formulas for the topological entropy of multimodal maps based on min-max symbols,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3415.

[24]

R. G. Andrzejak, K. Lehnertz, F. Mormann, C. Rieke, P. David and C. E. Elger, Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state,, Phys. Rev. E, 64 (2001). doi: 10.1103/PhysRevE.64.061907.

[25]

A. Antoniouk, K. Keller and S. Maksymenko, Kolmogorov-Sinai entropy via separation properties of order-generated sigma-algebras,, Discr. Contin. Dyn. Syst. A, 34 (2014), 1793.

[26]

R. B. Ash, Information Theory,, Dover Publications, (1990).

[27]

C. Bandt, G. Keller and B. Pompe, Entropy of interval maps via permutations,, Nonlinearity, 15 (2002), 1595. doi: 10.1088/0951-7715/15/5/312.

[28]

E. Beadle, J. Schroeder, B Moran and S. Suvorova, An overview of Rényi Entropy and some potential applications,, in 42nd Asilomar Conference on Signals, (2008), 1698. doi: 10.1109/ACSSC.2008.5074715.

[29]

C. H. Bennet, Notes on Landauer's principle, reversible computation and Maxwell's demon,, Stud. Hist. Philos. M. P., 34 (2003), 501. doi: 10.1016/S1355-2198(03)00039-X.

[30]

G. D. Birkhoff, Proof of a recurrence theorem for strongly transitive systems,, PNAS, 17 (1931), 650. doi: 10.1073/pnas.17.12.650.

[31]

S. A. Borovkova, Estimation and Prediction for Nonlinear Time Series,, Ph.D thesis, (1998).

[32]

M. Boyle and D. Lind, Expansive subdynamics,, Trans. Amer. Math. Soc., 349 (1997), 55. doi: 10.1090/S0002-9947-97-01634-6.

[33]

R. Bowen, Entropy for group endomorphisms and homogeneous spaces,, Trans. Amer. Math. Soc., 153 (1971), 401. doi: 10.1090/S0002-9947-1971-0274707-X.

[34]

L. Bowen, A measure-conjugacy invariant for free group actions,, Ann. of Math., 171 (2010), 1387. doi: 10.4007/annals.2010.171.1387.

[35]

L. Bowen, Measure-conjugacy invariants for actions of countable sofic groups,, J. Amer. Math. Soc., 23 (2010), 217. doi: 10.1090/S0894-0347-09-00637-7.

[36]

H. W. Broer and F. Takens, Dynamical Systems and Chaos,, Applied Mathematical Sciences, (2011). doi: 10.1007/978-1-4419-6870-8.

[37]

M. Brin and A. Katok, On local entropy,, in Geometric dynamics (ed. J. Palis), (1007), 30. doi: 10.1007/BFb0061408.

[38]

A. A. Bruzzo, B. Gesierich, M. Santi, C. A. Tassinari, N. Birbaumer and G. Rubboli, Permutation entropy to detect vigilance changes and preictal states from scalp EEG in epileptic patients. A preliminary study,, Neurol. Sci., 29 (2008), 3. doi: 10.1007/s10072-008-0851-3.

[39]

N. Burioka, M. Miyata, G. Cornélissen, F. Halberg, T. Takeshima, D. T. Kaplan, H. Suyama, M. Endo, Y. Maegaki and T. Nomura, et. al, Approximate entropy in the electroencephalogram during wake and sleep,, Clin. EEG Neurosci., 36 (2005), 21. doi: 10.1177/155005940503600106.

[40]

C. Cafaro, W. M. Lord, J. Sun and E. M. Bollt, Causation entropy from symbolic representations of dynamical systems,, Chaos, 25 (2015).

[41]

J. S. Cánovas and A. Guillamón, Permutations and time series analysis,, Chaos, 19 (2009). doi: 10.1063/1.3238256.

[42]

Y. Cao, W.-W. Tung, J. B. Gao, V. A. Protopopescu and L. M. Hively, Detecting dynamical changes in time series using the permutation entropy,, Phys. Rev. E, 70 (2004). doi: 10.1103/PhysRevE.70.046217.

[43]

A. Capurro, L. Diambra, D. Lorenzo, O. Macadar, M. T. Martin, C. Mostaccio, A. Plastino, E. Rofman, M. E. Torres and J. Velluti, Tsallis entropy and cortical dynamics: The analysis of EEG signals,, Physica A, 257 (1998), 149. doi: 10.1016/S0378-4371(98)00137-X.

[44]

A. Capurro, L. Diambra, D. Lorenzo, O. Macadar, M. T. Martin, C. Mostaccio, A. Plastino, J. Perez, E. Rofman and M. E. Torres, Human brain dynamics: The analysis of EEG signals with Tsallis information measure,, Physica A, 265 (1999), 235. doi: 10.1016/S0378-4371(98)00471-3.

[45]

D. Carrasco-Olivera, R. Metzger and C. A. Morales, Topological entropy for set-valued maps,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3461.

[46]

H. C. Choe, Computational Ergodic Theory,, Algorithms and Computation in Mathematics, (2005).

[47]

R. Clausius, The Mechanical Theory of Heat,, MacMillan and Co., (1865).

[48]

M. Courbage and B. Kamiński, Space-time directional Lyapunov exponents for cellular automata,, J. Statis. Phys., 124 (2006), 1499. doi: 10.1007/s10955-006-9172-1.

[49]

T. M. Cover and J. A. Thomas, Elements of Information Theory,, John Wiley & Sons, (2006).

[50]

B. Dai and B. Hu, Minimum conditional entropy clustering: A discriminative framework for clustering,, in JMLR: Workshop and Conference Proceedings, (2010), 47.

[51]

K. Denbigh, How subjective is entropy,, in Maxwell's Demon, (1990), 109.

[52]

M. Denker, Finite generators for ergodic, measure-preserving transformation,, Zeit. Wahr. ver. Geb., 29 (1974), 45. doi: 10.1007/BF00533186.

[53]

M. Denker, C. Grillenberger and K. Sigmund, Ergodic Theory on Compact Spaces,, Lecture Notes in Mathematics, (1976).

[54]

M. Denker and G. Keller, Rigorous statistical procedures for data from dynamical systems,, J. Stat. Phys., 44 (1986), 67. doi: 10.1007/BF01010905.

[55]

E. I. Dinaburg, The relation between topological entropy and metric entropy,, Soviet Math., 11 (1970), 13.

[56]

R. J. V. dos Santos, Generalization of Shannon's theorem for Tsallis entropy,, J. Math. Phys., 38 (1997), 4104. doi: 10.1063/1.532107.

[57]

J.-P. Eckmann and D. Ruelle, Ergodic theory of chaos and strange attractors,, Rev. Mod. Phys., 57 (1985), 617. doi: 10.1103/RevModPhys.57.617.

[58]

M. Einsiedler and T. Ward, Ergodic Theory with a View Towards Number Theory,, Graduate Texts in Mathematics, (2011). doi: 10.1007/978-0-85729-021-2.

[59]

D. K. Faddeev, On the notion of entropy of finite probability distributions,, Usp. Mat. Nauk (in Russian), 11 (1956), 227.

[60]

F. Falniowski, On the connections of the generalized entropies and Kolmogorov-Sinai entropies,, Entropy, 16 (2014), 3732. doi: 10.3390/e16073732.

[61]

B. Frank, B. Pompe, U. Schneider and D. Hoyer, Permutation entropy improves fetal behavioural state classification based on heart rate analysis from biomagnetic recordings in near term fetuses,, Med. Biol. Eng. Comput., 44 (2006), 179. doi: 10.1007/s11517-005-0015-z.

[62]

S. Furuichi, On uniqueness Theorems for Tsallis entropy and Tsallis relative entropy,, IEEE Trans. Inf. Theory, 51 (2005), 3638. doi: 10.1109/TIT.2005.855606.

[63]

L. G. Gamero, A. Plastino and M. E. Torres, Wavelet analysis and nonlinear dynamics in a nonextensive setting,, Physica A, 246 (1997), 487. doi: 10.1016/S0378-4371(97)00367-1.

[64]

P. G. Gaspard and X. J. Wang, Noise, chaos, and $(\varepsilon,\tau)$-entropy per unit time,, Phys. Rep., 235 (1993), 291. doi: 10.1016/0370-1573(93)90012-3.

[65]

T. N. T. Goodman, Relating topological entropy and measure entropy,, Bull. London Math. Soc., 3 (1971), 176. doi: 10.1112/blms/3.2.176.

[66]

N. Gradojevic and R. Genccay, Financial applications of nonextensive entropy,, IEEE Signal Process. Mag., 28 (2011).

[67]

B. Graff, G. Graff and A. Kaczkowska, Entropy Measures of Heart Rate Variability for Short ECG Datasets in Patients with Congestive Heart Failure,, Acta Phys. Pol. B Proc. Suppl., 5 (2012), 153.

[68]

C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods,, Econometrica, 37 (1969), 424.

[69]

P. Grassberger and I. Procaccia, Dimensions and entropies of strange attractors from a fluctuating dynamics approach,, Physica D, 13 (1984), 34. doi: 10.1016/0167-2789(84)90269-0.

[70]

P. Grassberger and I. Procaccia, Measuring the strangeness of strange attractors,, Physica D, 9 (1983), 189. doi: 10.1016/0167-2789(83)90298-1.

[71]

P. Grassberger and I. Procaccia, Characterization of strange attractors,, Phys. Rev. Lett., 50 (1983), 346. doi: 10.1103/PhysRevLett.50.346.

[72]

D. W. Hahs and S. D. Pethel, Distinguishing anticipation from causality: Anticipatory bias in the estimation of information flow,, Phys. Rev. Lett., 107 (2011).

[73]

R. Hanel and S. Thurner, A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and equidistribution functions,, EPL, 93 (2011).

[74]

R. Hanel, S. Thurner and M. Gell-Mann, Generalized entropies and logarithms and their duality relations,, Proceed. Nat. Acad. Scie., 109 (2012), 19151. doi: 10.1073/pnas.1216885109.

[75]

B. Hasselblatt and A. Katok, Principal structures,, in Handbook of Dynamical Systems (eds. B. Hasselblatt and A. Katok), (2002), 1. doi: 10.1016/S1874-575X(02)80003-0.

[76]

J. Havrda and F. Charvát, Quantification method of classification processes. Concept of structural $\alpha$-entropy,, Kybernetika, 3 (1967), 30.

[77]

G. A. Hedlund, Endomorphisms and automorphisms of the shift dynamical system,, Math. Syst. Theory, 3 (1969), 320. doi: 10.1007/BF01691062.

[78]

H. G. E. Hentschel and I. Procaccia, The infinite number of generalized dimensions of fractals and strange attractors,, Physica D, 8 (1983), 435. doi: 10.1016/0167-2789(83)90235-X.

[79]

R. Hornero, D. Abasolo, J. Escuredo and C. Gomez, Nonlinear analysis of electroencephalogram and magnetoencephalogram recordings in patients with Alzheimer's disease,, Phil. Trans. R. Soc. A, 367 (2009), 317. doi: 10.1098/rsta.2008.0197.

[80]

V. M. Ilić, M. S. Stanković and E. H. Mulalić, Comments on "Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy'',, IEEE Trans. Inf. Theory, 59 (2013), 6950.

[81]

E. T. Jaynes, Information theory and statistical mechanics,, Phys. Rev., 106 (1957), 620. doi: 10.1103/PhysRev.106.620.

[82]

P. Jizba and T. Arimitsu, The world according to Rényi: Thermodynamics of multifractal systems,, Ann. Phys., 312 (2004), 17. doi: 10.1016/j.aop.2004.01.002.

[83]

C. C. Jouny and G. K. Bergey, Characterization of early partial seizure onset: Frequency, complexity and entropy,, Clin. Neurophysiol., 123 (2012), 658. doi: 10.1016/j.clinph.2011.08.003.

[84]

N. Kannathal, M. L. Choo, U. R. Acharya and P. K. Sadasivan, Entropies for detection of epilepsy in EEG,, Comput. Meth. Prog. Bio., 80 (2005), 187. doi: 10.1016/j.cmpb.2005.06.012.

[85]

H. Kantz and T. Schreiber, Nonlinear Time Series Analysis,, Cambridge University Press, (2004).

[86]

A. Katok, Fifty years of entropy in dynamics: 1978-2007,, J. Mod. Dyn., 1 (2007), 545. doi: 10.3934/jmd.2007.1.545.

[87]

K. Keller and H. Lauffer, Symbolic analysis of high-dimensional time series,, Int. J. Bif. Chaos, 13 (2003), 2657. doi: 10.1142/S0218127403008168.

[88]

K. Keller, Permutations and the Kolmogorov-Sinai entropy,, Discr. Contin. Dyn. Syst. A, 32 (2012), 891. doi: 10.3934/dcds.2012.32.891.

[89]

K. Keller, A. M. Unakafov and V. A. Unakafova, Ordinal Patterns, Entropy, and EEG,, Entropy, 16 (2014), 6212. doi: 10.3390/e16126212.

[90]

K. Keller, S. Maksymenko and I. Stolz, Entropy determination based on the ordinal structure of a dynamical system,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3507.

[91]

A. I. Khinchin, Mathematical Foundations of Information Theory,, Dover, (1957).

[92]

A. N. Kolmogorov, A new metric invariant of transitive dynamical systems and Lebesgue space endomorphisms,, Dokl. Acad. Sci. USSR, 119 (1958), 861.

[93]

S. Kolyada and L. Snoha, Topological entropy of nonautonomous dynamical systems,, Random Comput. Dynam., 4 (1996), 205.

[94]

Z. S. Kowalski, Finite generators of ergodic endomorphisms,, Colloq. Math., 49 (1984), 87.

[95]

Z. S. Kowalski, Minimal generators for aperiodic endomorphisms,, Commentat. Math. Univ. Carol., 36 (1995), 721.

[96]

W. Krieger, On entropy and generators of measure-preserving transformations,, Trans. Amer. Math. Soc., 149 (1970), 453. doi: 10.1090/S0002-9947-1970-0259068-3.

[97]

J. Kurths, A. Voss, P. Saparin, A. Witt, H. J. Kleiner and N. Wessel, Quantitative analysis of heart rate variability,, Chaos, 5 (1995), 88. doi: 10.1063/1.166090.

[98]

D. E. Lake, J. S. Richman, M. P. Griffin and J. R. Moorman, Sample entropy analysis of neonatal heart rate variability,, Am. J. Physiol.-Reg. I., 283 (2002), 789. doi: 10.1152/ajpregu.00069.2002.

[99]

A. M. Law and W. D. Kelton, Simulation, Modeling, and Analysis,, McGraw-Hill, (2000).

[100]

F. Ledrappier and L.-S. Young, The metric entropy of diffeomorphisms. II. Relations betweenentropy, exponents and dimension,, Ann. Math., 122 (1985), 540. doi: 10.2307/1971329.

[101]

X. Li, G. Ouyang and D. A. Richards, Predictability analysis of absence seizures with permutation entropy,, Epilepsy Res., 77 (2007), 70. doi: 10.1016/j.eplepsyres.2007.08.002.

[102]

J. Li, J. Yan, X. Liu and G. Ouyang, Using permutation entropy to measure the changes in eeg signals during absence seizures,, Entropy, 16 (2014), 3049. doi: 10.3390/e16063049.

[103]

H. Li, K. Zhang and T. Jiang, Minimum entropy clustering and applications to gene expression analysis,, in Proc. IEEE Comput. Syst. Bioinform. Conf., (2004), 142.

[104]

Z. Liang, Y. Wang, X. Sun, D. Li, L. J. Voss, J. W. Sleigh, S. Hagihira and X. Li, EEG entropy measures in anesthesia,, Front. Comput. Neurosci., 9 (2015). doi: 10.3389/fncom.2015.00016.

[105]

D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding,, Cambridge University Press, (1995). doi: 10.1017/CBO9780511626302.

[106]

J. Llibre, Brief survey on the topological entropy,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3363.

[107]

N. Mammone, F. La Foresta and F. C. Morabito, Automatic artifact rejection from multichannel scalp EEG by wavelet ICA,, IEEE Sens. J., 12 (2012), 533. doi: 10.1109/JSEN.2011.2115236.

[108]

M. Matilla-García, A non-parametric test for independence based on symbolic dynamics,, J. Econ. Dyn. Control, 31 (2007), 3889. doi: 10.1016/j.jedc.2007.01.018.

[109]

M. Matilla-García and M. Ruiz Marín, A non-parametric independence test using permutation entropy,, J. Econ., 144 (2008), 139. doi: 10.1016/j.jeconom.2007.12.005.

[110]

A. M. Mesón and F. Vericat, On the Kolmogorov-like generalization of Tsallis entropy, correlation entropies and multifractal analysis,, J. Math. Phys., 43 (2002), 904. doi: 10.1063/1.1429323.

[111]

R. Miles and T. Ward, Directional uniformities, periodic points, and entropy,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3525.

[112]

J. Milnor, On the entropy geometry of cellular automata,, Complex Systems, 2 (1988), 357.

[113]

J. Milnor and W. Thurston, On iterated maps of the interval,, in Dynamical Systems (ed. J. C. Alexander), (1342), 465. doi: 10.1007/BFb0082847.

[114]

M. Misiurewicz and W. Szlenk, Entropy of piecewise monotone mappings,, Studia Math., 67 (1980), 45.

[115]

M. Misiurewicz, Permutations and topological entropy for interval maps,, Nonlinearity, 16 (2003), 971. doi: 10.1088/0951-7715/16/3/310.

[116]

R. Monetti, J. M. Amigó, T. Aschenbrenner and W. Bunk, Permutation complexity of interacting dynamical systems,, Eur. Phys. J. Special Topics, 222 (2013), 421. doi: 10.1140/epjst/e2013-01850-y.

[117]

R. Monetti, W. Bunk, T. Aschenbrenner, S. Springer and J. M. Amigó, Information directionality in coupled time series using transcripts,, Phys. Rev. E, 88 (2013). doi: 10.1103/PhysRevE.88.022911.

[118]

F. C. Morabito, D. Labate, F. La Foresta, A. Bramanti, G. Morabito and I. Palamara, Multivariate multi-scale permutation entropy for complexity analysis of Alzheimer's disease EEG,, Entropy, 14 (2012), 1186. doi: 10.3390/e14071186.

[119]

N. Nicolaou and J. Georgiou, The use of permutation entropy to characterize sleep electroencephalograms,, Clin. EEG Neurosci., 42 (2011), 24. doi: 10.1177/155005941104200107.

[120]

D. Ornstein, Two Bernoulli shifts with the same entropy are isomorphic,, Adv. Math., 4 (1970), 337. doi: 10.1016/0001-8708(70)90029-0.

[121]

D. Ornstein, Two Bernoulli shifts with infinite entropy are isomorphic,, Adv. Math., 5 (1970), 339. doi: 10.1016/0001-8708(70)90008-3.

[122]

D. Ornstein and B. Weiss, Entropy and isomorphism theorem for actions of amenable groups,, J. Analyse Math., 48 (1987), 1. doi: 10.1007/BF02790325.

[123]

D. Ornstein, Newton's law and coin tossing,, Notices Amer. Math. Soc., 60 (2013), 450. doi: 10.1090/noti974.

[124]

D. Ornstein and B. Weiss, Entropy is the only finitely observable invariant,, J. Mod. Dyn., 1 (2007), 93.

[125]

G. Ouyang, C. Dang, D. A. Richards and X. Li, Ordinal pattern based similarity analysis for EEG recordings,, Clin. Neurophysiol, 121 (2010), 694. doi: 10.1016/j.clinph.2009.12.030.

[126]

S. Y. Park and A. K. Bera, Maximum entropy autoregressive conditional heretoskedasticity model,, J. Econometrics, 150 (2009), 219. doi: 10.1016/j.jeconom.2008.12.014.

[127]

U. Parlitz, S. Berg, S. Luther, A. Schirdewan, J. Kurths and N. Wessel, Classifying cardiac biosignals using ordinal pattern statistics and symbolic dynamics,, Comput. Biol. Med., 42 (2012), 319. doi: 10.1016/j.compbiomed.2011.03.017.

[128]

W. Parry, Intrinsic Markov chains,, Trans. Amer. Math. Soc., 112 (1964), 55. doi: 10.1090/S0002-9947-1964-0161372-1.

[129]

Y. Pesin, Characteristic Lyapunov exponents and smooth ergodic theory,, Russian Math. Surveys, 32 (1977), 55.

[130]

S. M. Pincus, Approximate entropy as a measure of system complexity,, PNAS, 88 (1991), 2297. doi: 10.1073/pnas.88.6.2297.

[131]

S. M. Pincus and R. R. Viscarello, Approximate entropy: A regularity measure for fetal heart rate analysis,, Obstet. Gynecol., 79 (1992), 249.

[132]

S. M. Pincus, Approximate entropy as a measure of irregularity for psychiatric serial metrics,, Bipolar Disord., 8 (2006), 430. doi: 10.1111/j.1399-5618.2006.00375.x.

[133]

B. Pompe and J. Runge, Momentary information transfer as a coupling measure of time series,, Phys. Rev. E, 83 (2011). doi: 10.1103/PhysRevE.83.051122.

[134]

J. Poza, R. Hornero, J. Escudero, A. Fernández and C. A. Sánchez, Regional analysis of spontaneous MEG rhythms in patients with Alzheimer's disease using spectral entropies,, Ann. Biomed. Eng., 36 (2008), 141. doi: 10.1007/s10439-007-9402-y.

[135]

J. C. Principe, Information Theoretic Learning. Renyi's Entropy and Kernel Perspectives,, Springer, (2010). doi: 10.1007/978-1-4419-1570-2.

[136]

A. Rényi, On measures of entropy and information,, in Proceedings of the 4th Berkeley Symposium on Mathematics, (1961), 547.

[137]

J. S. Richman and J. R. Moorman, Physiological time-series analysis using approximate entropy and sample entropy,, Am. J. Physiol.-Heart C., 278 (2000), 2039.

[138]

S. J. Roberts, R. Everson and I. Rezek, Minimum entropy data partitioning,, in Proceedings of International Conference on Artificial Neural Networks, (1999), 844.

[139]

E. A. Robinson and A. Şahin, Rank-one $\mathbbZ^d$ actions and directional entropy,, Ergod. Theor. Dynam. Syst., 31 (2011), 285. doi: 10.1017/S0143385709000911.

[140]

O. A. Rosso, M. T. Martin and A. Plastino, Brain electrical activity analysis using wavelet-based informational tools,, Physica A, 313 (2002), 587. doi: 10.1016/S0378-4371(02)00958-5.

[141]

D. Rudolph and B. Weiss, Entropy and mixing for amenable group actions,, Ann. of Math., 151 (2000), 1119. doi: 10.2307/121130.

[142]

D. Ruelle, Statistical mechanics on a compact set with $Z^{\nu}$ action satisfying expansiveness and specification,, Trans. Amer. Math. Soc., 187 (1973), 237.

[143]

J. Runge, J. Heitzig, N. Marwan and J. Kurths, Quantifying causal coupling strength: A lag-specific measure for multivariate time series related to transfer entropy,, Phys. Rev. E, 86 (2012).

[144]

X. San Liang, Unraveling the cause-effect relation between time series,, Phys. Rev. E, 90 (2014).

[145]

T. Schreiber, Measuring information transfer,, Phys. Rev. Lett., 85 (2000), 461. doi: 10.1103/PhysRevLett.85.461.

[146]

C. E. Shannon, A mathematical theory of communication,, Bell Syst. Tech. J., 27 (1948), 379. doi: 10.1002/j.1538-7305.1948.tb01338.x.

[147]

Y. G. Sinai, On the notion of entropy of dynamical systems,, Dokl. Acad. Sci. USSR, 125 (1959), 768.

[148]

Y. G. Sinai, Flows with finite entropy,, Dokl. Acad. Sci. USSR, 125 (1959), 1200.

[149]

Y. G. Sinai, Gibbs measures in ergodic theory,, Uspehi Mat. Nauk, 27 (1972), 21.

[150]

D. Smirnov, Spurious causalities with transfer entropy,, Phys. Rev. E, 87 (2013).

[151]

M. Smorodinsky, Ergodic Theory, Entropy,, Lectures Notes in Mathematics, (1971).

[152]

R. Sneddon, The Tsallis entropy of natural information,, Physica A, 386 (2007), 101. doi: 10.1016/j.physa.2007.05.065.

[153]

V. Srinivasan, C. Eswaran and N. Sriraam, Approximate entropy-based epileptic EEG detection using artificial neural networks,, IEEE T. Inf. Technol. B., 11 (2007), 288. doi: 10.1109/TITB.2006.884369.

[154]

H. Suyari, Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy,, IEEE T. Inform. Theory, 50 (2004), 1783. doi: 10.1109/TIT.2004.831749.

[155]

F. Takens, Detecting strange attractors in turbulence,, in Dynamical Systems and Turbulence (eds. D. A. Rand and L. S. Young), (1981), 366.

[156]

F. Takens and E. Verbitskiy, Rényi entropies of aperiodic dynamical systems,, Israel J. Math., 127 (2002), 279. doi: 10.1007/BF02784535.

[157]

F. Takens and E. Verbitskiy, Generalized entropies: Rényi and correlation integral approach,, Nonlinearity, 11 (1998), 771. doi: 10.1088/0951-7715/11/4/001.

[158]

S. Tong, A. Bezerianos, A. Malhotra, Y. Zhu and N. Thakor, Parameterized entropy analysis of EEG following hypoxicischemic brain injury,, Phys. Lett. A, 314 (2003), 354. doi: 10.1016/S0375-9601(03)00949-6.

[159]

M. E. Torres and L. G. Gamero, Relative complexity changes in time series using information measures,, Physica A, 286 (2000), 457. doi: 10.1016/S0378-4371(00)00309-5.

[160]

B. Tóthmérész, Comparison of different methods for diversity ordering,, J. Veg. Sci., 6 (1995), 283.

[161]

C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics,, J. Stat. Phys., 52 (1988), 479. doi: 10.1007/BF01016429.

[162]

C. Tsallis, The nonadditive entropy Sq and its applications in physics and elsewhere: Some remarks,, Entropy, 13 (2011), 1765. doi: 10.3390/e13101765.

[163]

A. M. Unakafov and K. Keller, Conditional entropy of ordinal patterns,, Physica D, 269 (2014), 94. doi: 10.1016/j.physd.2013.11.015.

[164]

V. A. Unakafova, Investigating Measures of Complexity for Dynamical Systems and for Time Series,, Ph.D thesis, (2015).

[165]

E. A. Verbitskiy, Generalized Entropies in Dynamical Systems,, Ph.D thesis, (2000).

[166]

A. Voss, S. Schulz, R. Schroeder, M. Baumert and P. Caminal, Methods derived from nonlinear dynamics for analysing heart rate variability,, Phil. Trans. R. Soc. A, 367 (2009), 277. doi: 10.1098/rsta.2008.0232.

[167]

P. Walters, An Introduction to Ergodic Theory,, Springer Verlag, (1982).

[168]

B. Weiss, Subshifts of finite type and sofic systems,, Monats. Math., 77 (1973), 462. doi: 10.1007/BF01295322.

[169]

B. Weiss, Entropy and actions of sofic groups,, Discr. Contin. Dyn. Syst. B, 20 (2015), 3375.

[170]

N. Wiener, The theory of prediction,, in Modern Mathematics for the Engineer (ed. E. F. Beckenbach), (1956).

[171]

A. D. Wissner-Gross and C. E. Freer, Causal entropic forces,, Phys. Rev. Lett., 110 (2013). doi: 10.1103/PhysRevLett.110.168702.

[172]

M. Zanin, L. Zunino, O. A. Rosso and D. Papo, Permutation entropy and its main biomedical and econophysics applications: A review,, Entropy, 14 (2012), 1553. doi: 10.3390/e14081553.

[173]

D. Zhang, X. Jia, H. Ding, D. Ye and N. V. Thakor, Application of Tsallis entropy to EEG: Quantifying the presence of burst suppression after asphyxial cardiac arrest in rats,, IEEE Trans. Bio.-Med. Eng., 57 (2010), 867.

[1]

Yujun Zhu. Preimage entropy for random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2007, 18 (4) : 829-851. doi: 10.3934/dcds.2007.18.829

[2]

Jérôme Rousseau, Paulo Varandas, Yun Zhao. Entropy formulas for dynamical systems with mistakes. Discrete & Continuous Dynamical Systems - A, 2012, 32 (12) : 4391-4407. doi: 10.3934/dcds.2012.32.4391

[3]

Katayun Barmak, Eva Eggeling, Maria Emelianenko, Yekaterina Epshteyn, David Kinderlehrer, Richard Sharp, Shlomo Ta'asan. An entropy based theory of the grain boundary character distribution. Discrete & Continuous Dynamical Systems - A, 2011, 30 (2) : 427-454. doi: 10.3934/dcds.2011.30.427

[4]

John Kieffer and En-hui Yang. Ergodic behavior of graph entropy. Electronic Research Announcements, 1997, 3: 11-16.

[5]

Ryszard Rudnicki. An ergodic theory approach to chaos. Discrete & Continuous Dynamical Systems - A, 2015, 35 (2) : 757-770. doi: 10.3934/dcds.2015.35.757

[6]

Thierry de la Rue. An introduction to joinings in ergodic theory. Discrete & Continuous Dynamical Systems - A, 2006, 15 (1) : 121-142. doi: 10.3934/dcds.2006.15.121

[7]

Zhiming Li, Lin Shu. The metric entropy of random dynamical systems in a Hilbert space: Characterization of invariant measures satisfying Pesin's entropy formula. Discrete & Continuous Dynamical Systems - A, 2013, 33 (9) : 4123-4155. doi: 10.3934/dcds.2013.33.4123

[8]

João Ferreira Alves, Michal Málek. Zeta functions and topological entropy of periodic nonautonomous dynamical systems. Discrete & Continuous Dynamical Systems - A, 2013, 33 (2) : 465-482. doi: 10.3934/dcds.2013.33.465

[9]

Xiongping Dai, Yunping Jiang. Distance entropy of dynamical systems on noncompact-phase spaces. Discrete & Continuous Dynamical Systems - A, 2008, 20 (2) : 313-333. doi: 10.3934/dcds.2008.20.313

[10]

Wen-Guei Hu, Song-Sun Lin. On spatial entropy of multi-dimensional symbolic dynamical systems. Discrete & Continuous Dynamical Systems - A, 2016, 36 (7) : 3705-3717. doi: 10.3934/dcds.2016.36.3705

[11]

Vieri Benci, C. Bonanno, Stefano Galatolo, G. Menconi, M. Virgilio. Dynamical systems and computable information. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 935-960. doi: 10.3934/dcdsb.2004.4.935

[12]

Xiaomin Zhou. A formula of conditional entropy and some applications. Discrete & Continuous Dynamical Systems - A, 2016, 36 (7) : 4063-4075. doi: 10.3934/dcds.2016.36.4063

[13]

Xiongping Dai, Yu Huang, Mingqing Xiao. Realization of joint spectral radius via Ergodic theory. Electronic Research Announcements, 2011, 18: 22-30. doi: 10.3934/era.2011.18.22

[14]

Cristina Lizana, Vilton Pinheiro, Paulo Varandas. Contribution to the ergodic theory of robustly transitive maps. Discrete & Continuous Dynamical Systems - A, 2015, 35 (1) : 353-365. doi: 10.3934/dcds.2015.35.353

[15]

Enrico Valdinoci. Contemporary PDEs between theory and applications. Discrete & Continuous Dynamical Systems - A, 2015, 35 (12) : i-i. doi: 10.3934/dcds.2015.35.12i

[16]

Jonathan H. Tu, Clarence W. Rowley, Dirk M. Luchtenburg, Steven L. Brunton, J. Nathan Kutz. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 2014, 1 (2) : 391-421. doi: 10.3934/jcd.2014.1.391

[17]

J. G. Ollason, N. Ren. A general dynamical theory of foraging in animals. Discrete & Continuous Dynamical Systems - B, 2004, 4 (3) : 713-720. doi: 10.3934/dcdsb.2004.4.713

[18]

Krešimir Burazin, Marko Vrdoljak. Homogenisation theory for Friedrichs systems. Communications on Pure & Applied Analysis, 2014, 13 (3) : 1017-1044. doi: 10.3934/cpaa.2014.13.1017

[19]

Karsten Keller, Sergiy Maksymenko, Inga Stolz. Entropy determination based on the ordinal structure of a dynamical system. Discrete & Continuous Dynamical Systems - B, 2015, 20 (10) : 3507-3524. doi: 10.3934/dcdsb.2015.20.3507

[20]

Mariko Arisawa, Hitoshi Ishii. Some properties of ergodic attractors for controlled dynamical systems. Discrete & Continuous Dynamical Systems - A, 1998, 4 (1) : 43-54. doi: 10.3934/dcds.1998.4.43

2016 Impact Factor: 0.994

Metrics

  • PDF downloads (9)
  • HTML views (1)
  • Cited by (6)

[Back to Top]