Regressions without calculator

[jmsdg7]

im a senior in high school taking ap calculus, I've done regressions here and there with the calculator but i was wondering how do do it without the calculator, i obviously know how to get a linear equation out of 2 points, but how is it done with more points and higher degrees?

i understand that you need 2 points for a line, 3 for a quadradic, 4 for a cubic and 5 for a quartic but i was hoping someone could show me how it is done.

 

[Gunthi]

The simplest method i know is the Least Squares, there are many others.

I can show you the general form for any number of points and any type of equation, but it takes a little hard work and algebra.

In short therms, you have to find the parameters that minimize the difference $$\sum [f(x)-q(x)]^2$$, where $$q(x)$$ is the function you want to minimize, and $$f(x)$$ are the points you are given.

I'll try to post something more elaborate this week.

 

[EnumaElish]

Let the model be y = a + b x + u. Parameters of the model are a and b, u is the error term.

Variables y, x (and u) are each N-by-1 vectors.

Let X = [1 x] be the N-by-2 matrix. The first column of X is a vector of 1's. The second column of X is identical to vector x.

Let Z be the inverse of X'X. Z is a 2-by-2 matrix.

Then we can write [a b]' = ZX'y, which is 2-by-1. Parameter a is the first (top) element of ZX'y. Parameter b is the second element of ZX'y.

For higher order polynomials, substitute [x x^2 x^3 ...] for x, and [b1 b2 b3 ...] for b.

 

[statdad]

Of course, for simple regression, the matrix approach mentioned above is equivalent to the following equations:
The slope is given by

$$b = \frac{\sum{(x-\bar x)(y - \bar y)}}{\sum (x-\bar x)^2}$$

and the y-intercept is given by

$$a = \bar y - b \bar x$$