A theorem from information theory that is a simple consequence of the weak law of large
numbers. It states that if a set of values 
, 
, ..., 
is drawn independently from a random variable 
distributed according to 
, then the joint probability 
satisfies
 |
where 
is the entropy of the random variable 
.
