TOPICS
Search

Least Squares Fitting--Perpendicular Offsets


LeastSquaresOffsets

In practice, the vertical offsets from a line (polynomial, surface, hyperplane, etc.) are almost always minimized instead of the perpendicular offsets. This provides a fitting function for the independent variable X that estimates y for a given x (most often what an experimenter wants), allows uncertainties of the data points along the x- and y-axes to be incorporated simply, and also provides a much simpler analytic form for the fitting parameters than would be obtained using a fit based on perpendicular offsets.

The residuals of the best-fit line for a set of n points using unsquared perpendicular distances d_i of points (x_i,y_i) are given by

 R__|_=sum_(i=1)^nd_i.
(1)

Since the perpendicular distance from a line y=a+bx to point i is given by

 d_i=(|y_i-(a+bx_i)|)/(sqrt(1+b^2)),
(2)

the function to be minimized is

 R__|_=sum_(i=1)^n(|y_i-(a+bx_i)|)/(sqrt(1+b^2)).
(3)

Unfortunately, because the absolute value function does not have continuous derivatives, minimizing R__|_ is not amenable to analytic solution. However, if the square of the perpendicular distances

 R__|_^2=sum_(i=1)^n([y_i-(a+bx_i)]^2)/(1+b^2)
(4)

is minimized instead, the problem can be solved in closed form. R__|_^2 is a minimum when

 (partialR__|_^2)/(partiala)=2/(1+b^2)sum_(i=1)^n[y_i-(a+bx_i)](-1)=0
(5)

and

 (partialR__|_^2)/(partialb)=2/(1+b^2)sum_(i=1)^n[y_i-(a+bx_i)](-x_i)+sum_(i=1)^n([y_i-(a+bx_i)]^2(-1)(2b))/((1+b^2)^2)=0.
(6)

The former gives

a=(sum_(i=1)^(n)y_i-bsum_(i=1)^(n)x_i)/n
(7)
=y^_-bx^_,
(8)

and the latter

 (1+b^2)sum_(i=1)^n[y_i-(a+bx_i)]x_i+bsum_(i=1)^n[y_i-(a+bx_i)]^2=0.
(9)

But

[y-(a+bx)]^2=y^2-2(a+bx)y+(a+bx)^2
(10)
=y^2-2ay-2bxy+a^2+2abx+b^2x^2,
(11)

so (10) becomes

(1+b^2)(sum_(i=1)^(n)x_iy_i-asum_(i=1)^(n)x_i-bsum_(i=1)^(n)x_i^2)+b(sum_(i=1)^(n)y_i^2-2asum_(i=1)^(n)y_i-2bsum_(i=1)^(n)x_iy_i+a^2sum_(i=1)^(n)1+2absum_(i=1)^(n)x_i+b^2sum_(i=1)^(n)x_i^2)=0
(12)
[(1+b^2)(-b)+b(b^2)]sum_(i=1)^(n)x_i^2+[(1+b^2)-2b^2]sum_(i=1)^(n)x_iy_i+bsum_(i=1)^(n)y_i^2+[-a(1+b^2)+2ab^2]sum_(i=1)^(n)x_i-2absum_(i=1)^(n)y_i+ba^2sum_(i=1)^(n)1=0
(13)
-bsum_(i=1)^(n)x_i^2+(1-b^2)sum_(i=1)^(n)x_iy_i+bsum_(i=1)^(n)y_i^2+a(b^2-1)sum_(i=1)^(n)x_i-2absum_(i=1)^(n)y_i+ba^2n=0.
(14)

Plugging (◇) into (14) then gives

 -bsum_(i=1)^nx_i^2+(1-b^2)sum_(i=1)^nx_iy_i+bsum_(i=1)^ny_i^2+1/n(b^2-1)(sum_(i=1)^ny_i-bsum_(i=1)^nx_i)sum_(i=1)^nx_i-2/n(sum_(i=1)^ny_i-bsum_(i=1)^nx_i)bsum_(i=1)^ny_i+b/n(sum_(i=1)^ny_i-bsum_(i=1)^nx_i)^2 
=0
(15)

After a fair bit of algebra, the result is

 b^2+(sum_(i=1)^(n)y_i^2-sum_(i=1)^(n)x_i^2+1/n[(sum_(i=1)^(n)x_i)^2-(sum_(i=1)^(n)y_i)^2])/(1/nsum_(i=1)^(n)x_isum_(i=1)^(n)y_i-sum_(i=1)^(n)x_iy_i)b-1=0.
(16)

So define

B=1/2([sum_(i=1)^ny_i^2-1/n(sum_(i=1)^ny_i)^2]-[sum_(i=1)^nx_i^2-1/n(sum_(i=1)^nx_i)^2])/(1/nsum_(i=1)^nx_isum_(i=1)^ny_i-sum_(i=1)^nx_iy_i)
(17)
=1/2((sum_(i=1)^ny_i^2-ny^_^2)-(sum_(i=1)^nx_i^2-nx^_^2))/(nx^_y^_-sum_(i=1)^nx_iy_i),
(18)

and the quadratic formula gives

 b=-B+/-sqrt(B^2+1),
(19)

with a found using (◇). Note the rather unwieldy form of the best-fit parameters in the formulation. In addition, minimizing R__|_^2 for a second- or higher-order polynomial leads to polynomial equations having higher order, so this formulation cannot be extended.


See also

Least Squares Fitting

Explore with Wolfram|Alpha

References

Sardelis, D. and Valahas, T. "Least Squares Fitting-Perpendicular Offsets." http://library.wolfram.com/infocenter/MathSource/5292/.

Cite this as:

Weisstein, Eric W. "Least Squares Fitting--Perpendicular Offsets." From MathWorld--A Wolfram Web Resource. https://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html

Subject classifications