TOPICS
Search

Central Limit Theorem

Let X_1,X_2,...,X_N be a set of N independent random variates and each X_i have an arbitrary probability distribution P(x_1,...,x_N) with mean mu_i and a finite variance sigma_i^2. Then the normal form variate

X_(norm)=(sum_(i=1)^(N)x_i-sum_(i=1)^(N)mu_i)/(sqrt(sum_(i=1)^(N)sigma_i^2))
(1)

has a limiting cumulative distribution function which approaches a normal distribution.

Under additional conditions on the distribution of the addend, the probability density itself is also normal (Feller 1971) with mean mu=0 and variance sigma^2=1. If conversion to normal form is not performed, then the variate

X=1/Nsum_(i=1)^Nx_i
(2)

is normally distributed with mu_X=mu_x and sigma_X=sigma_x/sqrt(N).

Kallenberg (1997) gives a six-line proof of the central limit theorem. For an elementary, but slightly more cumbersome proof of the central limit theorem, consider the inverse Fourier transform of P_X(f).

F_f^(-1)[P_X(f)](x)=int_(-infty)^inftye^(2piifX)P(X)dX
(3)
=int_(-infty)^inftysum_(n=0)^(infty)((2piifX)^n)/(n!)P(X)dX
(4)
=sum_(n=0)^(infty)((2piif)^n)/(n!)int_(-infty)^inftyX^nP(X)dX
(5)
=sum_(n=0)^(infty)((2piif)^n)/(n!)<X^n>.
(6)

Now write

<X^n>=<N^(-n)(x_1+x_2+...+x_N)^n> =int_(-infty)^inftyN^(-n)(x_1+...+x_N)^nP(x_1)...P(x_N)dx_1...dx_N,
(7)

so we have

F_f^(-1)[P_X(f)](x)=sum_(n=0)^(infty)((2piif)^n)/(n!)<X^n>
(8)
=sum_(n=0)^(infty)((2piif)^n)/(n!)int_(-infty)^inftyN^(-n)(x_1+...+x_N)^n×P(x_1)...P(x_N)dx_1...dx_N
(9)
=int_(-infty)^inftysum_(n=0)^(infty)[(2piif(x_1+...+x_N))/N]^n1/(n!)P(x_1)...P(x_N)dx_1...dx_N
(10)
=int_(-infty)^inftye^(2piif(x_1+...+x_N)/N)P(x_1)...P(x_N)dx_1...dx_N
(11)
=[int_(-infty)^inftye^(2piifx_1/N)P(x_1)dx_1]×...×[int_(-infty)^inftye^(2piifx_N/N)P(x_N)dx_N]
(12)
=[int_(-infty)^inftye^(2piifx/N)P(x)dx]^N
(13)
={int_(-infty)^infty[1+((2piif)/N)x+1/2((2piif)/N)^2x^2+...]P(x)dx}^N
(14)
=[1+(2piif)/N<x>-((2pif)^2)/(2N^2)<x^2>+O(N^(-3))]^N
(15)
=exp{Nln[1+(2piif)/N<x>-((2pif)^2)/(2N^2)<x^2>+O(N^(-3))]}.
(16)

Now expand

ln(1+x)=x-1/2x^2+1/3x^3+...,
(17)

so

F_f^(-1)[P_X(f)](x) approx exp{N[(2piif)/N<x>-((2pif)^2)/(2N^2)<x^2>+1/2((2piif)^2)/(N^2)<x>^2+O(N^(-3))]}
(18)
=exp[2piif<x>-((2pif)^2(<x^2>-<x>^2))/(2N)+O(N^(-2))]
(19)
approx exp[2piifmu_x-((2pif)^2sigma_x^2)/(2N)],
(20)

since

mu_x=<x>
(21)
sigma_x^2=<x^2>-<x>^2.
(22)

Taking the Fourier transform,

P_X=int_(-infty)^inftye^(-2piifx)F^(-1)[P_X(f)]df
(23)
=int_(-infty)^inftye^(2piif(mu_x-x)-(2pif)^2sigma_x^2/2N)df.
(24)

This is of the form

int_(-infty)^inftye^(iaf-bf^2)df,
(25)

where a=2pi(mu_x-x) and b=(2pisigma_x)^2/2N. But this is a Fourier transform of a Gaussian function, so

int_(-infty)^inftye^(iaf-bf^2)df=e^(-a^2/4b)sqrt(pi/b)
(26)

(e.g., Abramowitz and Stegun 1972, p. 302, equation 7.4.6). Therefore,

P_X=sqrt(pi/(((2pisigma_x)^2)/(2N)))exp{(-[2pi(mu_x-x)]^2)/(4((2pisigma_x)^2)/(2N))}
(27)
=sqrt((2piN)/(4pi^2sigma_x^2))exp[-(4pi^2(mu_x-x)^22N)/(4·4pi^2sigma_x^2)]
(28)
=(sqrt(N))/(sigma_xsqrt(2pi))e^(-(mu_x-x)^2N/2sigma_x^2).
(29)

But sigma_X=sigma_x/sqrt(N) and mu_X=mu_x, so

P_X=1/(sigma_Xsqrt(2pi))e^(-(mu_X-x)^2/2sigma_X^2).
(30)

The "fuzzy" central limit theorem says that data which are influenced by many small and unrelated random effects are approximately normally distributed.

See also

Berry-Esséen Theorem, Fourier Transform--Gaussian, Lindeberg Condition, Lindeberg-Feller Central Limit Theorem, Lyapunov Condition Explore this topic in the MathWorld classroom

Explore with Wolfram|Alpha

References

Abramowitz, M. and Stegun, I. A. (Eds.). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover, 1972.Feller, W. "The Fundamental Limit Theorems in Probability." Bull. Amer. Math. Soc. 51, 800-832, 1945.Feller, W. An Introduction to Probability Theory and Its Applications, Vol. 1, 3rd ed. New York: Wiley, p. 229, 1968.Feller, W. An Introduction to Probability Theory and Its Applications, Vol. 2, 3rd ed. New York: Wiley, 1971.Kallenberg, O. Foundations of Modern Probability. New York: Springer-Verlag, 1997.Lindeberg, J. W. "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung." Math. Z. 15, 211-225, 1922.Spiegel, M. R. Theory and Problems of Probability and Statistics. New York: McGraw-Hill, pp. 112-113, 1992.Trotter, H. F. "An Elementary Proof of the Central Limit Theorem." Arch. Math. 10, 226-234, 1959.Zabell, S. L. "Alan Turing and the Central Limit Theorem." Amer. Math. Monthly 102, 483-494, 1995.

Referenced on Wolfram|Alpha

Central Limit Theorem

Cite this as:

Weisstein, Eric W. "Central Limit Theorem." From MathWorld--A Wolfram Web Resource. https://mathworld.wolfram.com/CentralLimitTheorem.html

Subject classifications