Entropy is the only finitely observable invariant

Pages: 93 - 105,
Issue 1,
January
2007 doi:10.3934/jmd.2007.1.93

Donald Ornstein - Department of Mathematics, Stanford University, Stanford, CA 94305, United States (email)

Benjamin Weiss - Einstein Institute of Mathematics, Edmond J. Safra Campus, Givat Ram, The Hebrew University of Jerusalem, Jerusalem, 91904, Israel (email)

Abstract:
Our main purpose is to present a surprising new characterization of the
Shannon entropy of stationary ergodic processes. We will use two basic
concepts: *isomorphism of stationary processes* and a notion of
*finite observability*, and we will see how one is led, inevitably, to
Shannon's entropy. A function $J$ with values in some metric space, defined on
all finite-valued, stationary, ergodic processes is said to be
*finitely observable (FO)* if there is a sequence of functions
$S_{n}(x_{1},x_{2},...,x_{n})$ that for all processes $\mathcal{X}$ converges
to $J(\mathcal{X})$ for almost every realization $x_{1}^{\infty}$ of
$\mathcal{X}$. It is called an *invariant* if it returns the same value
for isomorphic processes. We show that any finitely observable invariant is
necessarily a continuous function of the entropy. Several extensions of this
result will also be given.

Keywords: entropy, finitely observable, isomorphism invariants, finitary isomorphism.

Mathematics Subject Classification: Primary 37A35, Secondary 60G10.

Received: May 2006;
Available Online: October 2006.