Proof of least Squares estimators

  • Thread starter julion
  • Start date
  • #1
1
0

Main Question or Discussion Point

Hey guys, long time lurker, first time poster!
Just having some trouble with something..Im probably just looking at it the wrong way, but I was wondering if anyone could help me with this..

Im trying to prove that by choosing b0 and b1 to minimize
http://img24.imageshack.us/img24/7/partas.jpg [Broken]
you obtain the least squares estimators, namely:
http://img15.imageshack.us/img15/3641/partbx.jpg [Broken]

also just wondering how you can prove that OLS minimizes the sum of squares function.
I know it has something to do with second derivatives, but im a bit stuck.
Thanks!
 
Last edited by a moderator:

Answers and Replies

  • #2
It's a standard maximization problem. Set up the sum of squared errors (SSE), differentiate with respect to beta, set to zero, solve for beta. For a maximum, verify that the second derivative at the beta value you found in the first step is negative.
 
  • #3
2
0
Hey guys, long time lurker, first time poster!
Just having some trouble with something..Im probably just looking at it the wrong way, but I was wondering if anyone could help me with this..

Im trying to prove that by choosing b0 and b1 to minimize
http://img24.imageshack.us/img24/7/partas.jpg [Broken]
you obtain the least squares estimators, namely:
http://img15.imageshack.us/img15/3641/partbx.jpg [Broken]

also just wondering how you can prove that OLS minimizes the sum of squares function.
I know it has something to do with second derivatives, but im a bit stuck.
Thanks!
could you expand how to do that with a little bit more help please?
 
Last edited by a moderator:
  • #4
statdad
Homework Helper
1,495
35
Treat

[tex]
S(b_0, b_1) = \sum_{i=1}^n \left(y_i - (b_0 + b_1 x_i)\right)^2
[/tex]

as a function of [tex] b_0 [/tex] and [tex] b_1 [/tex], and solve this system of equations - the solutions will give the formulas for the estimates of slope and intercept.

[tex]
\begin{align*}
\frac{\partial S}{\partial b_0} & = 0\\
\frac{\partial S}{\partial b_1} & = 0
\end{align*}
[/tex]
 
  • #5
2
0
thanks :)
 

Related Threads for: Proof of least Squares estimators

Replies
12
Views
9K
Replies
4
Views
966
  • Last Post
Replies
1
Views
769
  • Last Post
Replies
11
Views
4K
  • Last Post
Replies
0
Views
1K
  • Last Post
Replies
1
Views
5K
Replies
2
Views
578
  • Last Post
Replies
7
Views
2K
Top