February  2008, 2(1): 1-13. doi: 10.3934/amc.2008.2.1

Entropy estimators with almost sure convergence and an O(n-1) variance

1. 

Department of Physics and Computer Science, Wilfrid Laurier University, Waterloo, Ontario N2L3C5, Canada

2. 

Department of Computer Science, Yaroslavl State University, Yaroslavl, 150000, Russian Federation

Received  April 2007 Revised  October 2007 Published  January 2008

The problem of the estimation of the entropy rate of a stationary ergodic process $\mu$ is considered. A new nonparametric entropy rate estimator is constructed for a sample of n sequences $(X_1$(1)$,\ldots, (X_m$(1)$),\ldots, (X_1$(n) $,\ldots, (X_m$(n)$)$ independently generated by $\mu$. It is shown that, for $m=O(\log n)$, the estimator converges almost surely and its variance is upper-bounded by $O(n$−1$)$ for a large class of stationary ergodic processes with a finite state space. As the order $O(n$−1$)$ of the variance growth on $n$ is the same as that of the optimal Cramer-Rao lower bound, presented is the first near-optimal estimator in the sense of the variance convergence.
Citation: Alexei Kaltchenko, Nina Timofeeva. Entropy estimators with almost sure convergence and an O(n-1) variance. Advances in Mathematics of Communications, 2008, 2 (1) : 1-13. doi: 10.3934/amc.2008.2.1
[1]

Shinsuke Koyama, Lubomir Kostal. The effect of interspike interval statistics on the information gain under the rate coding hypothesis. Mathematical Biosciences & Engineering, 2014, 11 (1) : 63-80. doi: 10.3934/mbe.2014.11.63

[2]

John Kieffer and En-hui Yang. Ergodic behavior of graph entropy. Electronic Research Announcements, 1997, 3: 11-16.

[3]

Guy Cohen, Jean-Pierre Conze. The CLT for rotated ergodic sums and related processes. Discrete & Continuous Dynamical Systems - A, 2013, 33 (9) : 3981-4002. doi: 10.3934/dcds.2013.33.3981

[4]

Marcello Delitala, Tommaso Lorenzi. A mathematical model for value estimation with public information and herding. Kinetic & Related Models, 2014, 7 (1) : 29-44. doi: 10.3934/krm.2014.7.29

[5]

Wenjun Xia, Jinzhi Lei. Formulation of the protein synthesis rate with sequence information. Mathematical Biosciences & Engineering, 2018, 15 (2) : 507-522. doi: 10.3934/mbe.2018023

[6]

Jean-Yves Le Boudec. The stationary behaviour of fluid limits of reversible processes is concentrated on stationary points. Networks & Heterogeneous Media, 2013, 8 (2) : 529-540. doi: 10.3934/nhm.2013.8.529

[7]

M. A. Efendiev. On the compactness of the stable set for rate independent processes. Communications on Pure & Applied Analysis, 2003, 2 (4) : 495-509. doi: 10.3934/cpaa.2003.2.495

[8]

T. J. Sullivan, M. Koslowski, F. Theil, Michael Ortiz. Thermalization of rate-independent processes by entropic regularization. Discrete & Continuous Dynamical Systems - S, 2013, 6 (1) : 215-233. doi: 10.3934/dcdss.2013.6.215

[9]

Robert Azencott, Yutheeka Gadhyan. Accurate parameter estimation for coupled stochastic dynamics. Conference Publications, 2009, 2009 (Special) : 44-53. doi: 10.3934/proc.2009.2009.44

[10]

Jean René Chazottes, E. Ugalde. Entropy estimation and fluctuations of hitting and recurrence times for Gibbsian sources. Discrete & Continuous Dynamical Systems - B, 2005, 5 (3) : 565-586. doi: 10.3934/dcdsb.2005.5.565

[11]

José Gómez-Torrecillas, F. J. Lobillo, Gabriel Navarro. Information--bit error rate and false positives in an MDS code. Advances in Mathematics of Communications, 2015, 9 (2) : 149-168. doi: 10.3934/amc.2015.9.149

[12]

Jong-Shenq Guo, Bei Hu. Blowup rate estimates for the heat equation with a nonlinear gradient source term. Discrete & Continuous Dynamical Systems - A, 2008, 20 (4) : 927-937. doi: 10.3934/dcds.2008.20.927

[13]

Martin Heida, Alexander Mielke. Averaging of time-periodic dissipation potentials in rate-independent processes. Discrete & Continuous Dynamical Systems - S, 2017, 10 (6) : 1303-1327. doi: 10.3934/dcdss.2017070

[14]

Young-Sam Kwon. On the well-posedness of entropy solutions for conservation laws with source terms. Discrete & Continuous Dynamical Systems - A, 2009, 25 (3) : 933-949. doi: 10.3934/dcds.2009.25.933

[15]

Nikolai Dokuchaev. On strong causal binomial approximation for stochastic processes. Discrete & Continuous Dynamical Systems - B, 2014, 19 (6) : 1549-1562. doi: 10.3934/dcdsb.2014.19.1549

[16]

H.Thomas Banks, Shuhua Hu. Nonlinear stochastic Markov processes and modeling uncertainty in populations. Mathematical Biosciences & Engineering, 2012, 9 (1) : 1-25. doi: 10.3934/mbe.2012.9.1

[17]

Antoine Hochart. An accretive operator approach to ergodic zero-sum stochastic games. Journal of Dynamics & Games, 2019, 6 (1) : 27-51. doi: 10.3934/jdg.2019003

[18]

Fritz Colonius. Invariance entropy, quasi-stationary measures and control sets. Discrete & Continuous Dynamical Systems - A, 2018, 38 (4) : 2093-2123. doi: 10.3934/dcds.2018086

[19]

Yoora Kim, Gang Uk Hwang, Hea Sook Park. Feedback limited opportunistic scheduling and admission control for ergodic rate guarantees over Nakagami-$m$ fading channels. Journal of Industrial & Management Optimization, 2009, 5 (3) : 553-567. doi: 10.3934/jimo.2009.5.553

[20]

Jiongxuan Zheng, Joseph D. Skufca, Erik M. Bollt. Heart rate variability as determinism with jump stochastic parameters. Mathematical Biosciences & Engineering, 2013, 10 (4) : 1253-1264. doi: 10.3934/mbe.2013.10.1253

2018 Impact Factor: 0.879

Metrics

  • PDF downloads (5)
  • HTML views (0)
  • Cited by (6)

Other articles
by authors

[Back to Top]