TOPICS
Search

Statistical Correlation


For two random variates X and Y, the correlation is defined bY

 cor(X,Y)=(cov(X,Y))/(sigma_Xsigma_Y),
(1)

where sigma_X denotes standard deviation and cov(X,Y) is the covariance of these two variables. For the general case of variables X_i and X_j, where i,j=1, 2, ..., n,

 cor(X_i,X_j)=(cov(X_i,X_j))/(sqrt(V_(ii)V_(jj))),
(2)

where V_(ii) are elements of the covariance matrix. In general, a correlation gives the strength of the relationship between variables. For i=j,

 cor(X_i,X_i)=(cov(X_i,X_i))/(sigma_i^2)=1.
(3)

The variance of any quantity is always nonnegative by definition, so

 var(X/(sigma_X)+Y/(sigma_Y))>=0.
(4)

From a property of variances, the sum can be expanded

 var(X/(sigma_X))+var(Y/(sigma_Y))+2cov(X/(sigma_X),Y/(sigma_Y))>=0
(5)
 1/(sigma_X^2)var(X)+1/(sigma_Y^2)var(Y)+2/(sigma_Xsigma_Y)cov(X,Y)>=0
(6)
 1+1+2/(sigma_Xsigma_Y)cov(X,Y)=2+2/(sigma_Xsigma_Y)cov(X,Y)>=0.
(7)

Therefore,

 cor(X,Y)=(cov(X,Y))/(sigma_Xsigma_Y)>=-1.
(8)

Similarly,

 var(X/(sigma_X)-Y/(sigma_Y))>=0
(9)
 var(X/(sigma_X))+var(-Y/(sigma_Y))+2cov(X/(sigma_X),-Y/(sigma_Y))>=0
(10)
 1/(sigma_X^2)var(X)+1/(sigma_Y^2)var(Y)-2/(sigma_Xsigma_Y)cov(X,Y)>=0
(11)
 1+1-2/(sigma_Xsigma_Y)cov(X,Y)=2-2/(sigma_Xsigma_Y)cov(X,Y)>=0.
(12)

Therefore,

 cor(X,Y)=(cov(X,Y))/(sigma_Xsigma_Y)<=1,
(13)

so -1<=cor(X,Y)<=1.

For a linear combination of two variables,

var(Y-bX)=var(Y)+var(-bX)+2cov(Y,-bX)
(14)
=var(Y)+b^2var(X)-2bcov(X,Y)
(15)
=sigma_Y^2+b^2sigma_X^2-2bcov(X,Y)
(16)
=sigma_Y^2+b^2sigma_X^2-2bsigma_Xsigma_Ycor(X,Y).
(17)

Examine the cases where cor(X,Y)=+/-1,

 cor(X,Y)=(cov(X,Y))/(sigma_Xsigma_Y)=+/-1
(18)
 var(Y-bX)=b^2sigma_X^2+sigma_Y^2∓2bsigma_Xsigma_Y=(bsigma_X∓sigma_Y)^2.
(19)

The variance will be zero if b=+/-sigma_Y/sigma_X, which requires that the argument of the variance is a constant. Therefore, y-bx=a, so y=a+bx. If cor(X,Y)=+/-1, y is either perfectly correlated (b>0) or perfectly anticorrelated (b<0) with x.


See also

Covariance, Covariance Matrix, Variance

Explore with Wolfram|Alpha

Cite this as:

Weisstein, Eric W. "Statistical Correlation." From MathWorld--A Wolfram Web Resource. https://mathworld.wolfram.com/StatisticalCorrelation.html

Subject classifications