# Least squares estimation

Tags:
1. Oct 30, 2015

### bonfire09

1. The problem statement, all variables and given/known data
Suppose that object 1 weighs $\theta_1$ and object two weights $\theta_2$. Then each object is weighed once and then together getting three observations $y_1,y_2,y_3$. The scale gives unbiased weights with normally distributed error (constant variance) Find the least square estimates for $\theta_1$ and $\theta_2$.
2. Relevant equations
The ls estimates for theta 1 and theta 2 is $\hat{\theta}=(X^TX)^{-1}X^TY$

3. The attempt at a solution
I wrote the full mode as such
$y_1=\theta_1+\epsilon_1$
$y_2=\theta_2+\epsilon_2$
$y_3=\theta_1+\theta_2+\epsilon_3$
Then it follows that in matrix form we get $\begin{bmatrix} y_{1} \\ y_{2} \\ y_{3} \end{bmatrix} = \begin{bmatrix} \ \theta_1 \\ \theta_2 \\ \theta_1+\theta_2 \end{bmatrix}+\begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \epsilon_3 \end{bmatrix}$ $=\begin{bmatrix} y_{1} \\ y_{2} \\ y_{3} \end{bmatrix} = \begin{bmatrix} 1 &0 \\ 0 & 1 \\1 & 1 \end{bmatrix} \begin{bmatrix} \theta_1 \\ \theta_2 \end{bmatrix}+\begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \epsilon_3 \end{bmatrix}$.

Then from here I got as my final answer
$\hat{\theta}=(X^TX)^{-1}X^TY=\begin{bmatrix} \hat{\theta_1} \\ \hat{\theta_2} \end{bmatrix} =\begin{bmatrix} y_1-y_2 \\ \frac{-1}{2}y_1+y_2+\frac{1}{2}y_3 \end{bmatrix}$ The thing the I am not sure about is if I wrote my full model correctly. I was thinking that the observations are just the true weight of the object with some random error. But I am not sure.

Last edited: Oct 30, 2015
2. Oct 30, 2015

### Ray Vickson

According to your model you are dealing with a very smart scale. It says to itself "I remember the errors $\epsilon_1$ and $\epsilon_2$ that I made when the guy weighed objects 1 and 2 separately. I see now that he is weighing the two objects together, so I had better add up the two errors I made before". As I said, a very smart scale indeed.

Last edited: Nov 1, 2015
3. Oct 30, 2015

### Staff: Mentor

Right, but the random error applies to the measurements, not to the weights.

4. Oct 30, 2015

### bonfire09

Oops now I fixed that. I don't know why I placed that in the first place. So I changed $\epsilon_1+\epsilon_2$ to $\epsilon_3$. I think it looks correct now.

Last edited: Oct 30, 2015
5. Oct 30, 2015

### Ray Vickson

Yes, I would say so.