Entropy rate

The entropy rate of a stochastic process is, informally, the time density of the average information in a stochastic process. For stochastic processes with a countable index, the entropy rate $H (X)$ is the limit of the joint entropy of $n$ members of the process $X k$ divided by $n$ , as $n$ tends to infinity:

$H(X) = \lim_{n \to \infty} \frac{1}{n} H(X_1, X_2, \dots X_n)$