# Least squares problem: am I solving it correctly?

Tags:
1. Jan 11, 2016

### Granger

1. The problem statement, all variables and given/known data
In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system:
x + y + z = 1
x + z = 0
y = 0

2. The attempt at a solution
So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1) row 2 (1,0,1) and row 3 (0,1,0); x being the column (x_1,x_2,x_3) and b being the column (1,0,0).

So I did it and I reached to the solution (x_1, \frac {1}{3}, \frac {1}{3} + x_1)

And I expanded this solution in two vectors (0, \frac {1}{3}, \frac {1}{3}) and (1,0,1).

So these are the least square solutions and the one with shorter length is the first one.

My doubt is if I'm doing this correctly or if I made any mistake because I used an online calculator that only give one least square solution. Can someone help me to verify my attempt? Thanks!

2. Jan 11, 2016

### Ray Vickson

Your LS solution is OK, but you have not found the one with the shortest length. You need to figure out what value of x_1 leads to the smallest value of $L$, where $L$ is the "length" of the solution, with $L^2 = x^2 + y^2 + z^2$. Note that minimizing $L$ will be equivalent to the simpler problem of minimizing $L^2$, and that just amounts to finding the smallest value of a quadratic function of x_1.

BTW: the reason for the discrepancy is that some LS programs give just one of the LS solutions, even when there are many to choose from. That is because they apply an algorithm in which certain tests are performed and certain steps taken in accord with the tests, but if there are a choice of many possible steps, they just take one of them (chosen according to some well-defined rule, usually).

3. Jan 11, 2016

### Granger

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!

4. Jan 11, 2016

### SteamKing

Staff Emeritus
When writing Latex in posts either enclose the expressions in double # for in-line latex or double \$ for regular multi-line Latex. Also, use the Preview button located at the Lower right hand corner to check how the formatting will appear before posting.

Example: $(x_1, \frac{1}{3}, \frac{1}{3} + x_1)$

5. Jan 11, 2016

### Ray Vickson

I don't think your final solution is correct.

Last edited: Jan 12, 2016
6. Jan 12, 2016

### Granger

Then how do you think I should do it? I'm not understanding...

7. Jan 12, 2016

### Ray Vickson

What did you do? Show your work.

8. Jan 12, 2016

### Granger

I already explained in my first post !

9. Jan 12, 2016

### Ray Vickson

No, you didn't. You explained how you got $(x,y,z) = (x_1, \frac{1}{3}, \frac{1}{3} + x_1)$. But, how do you go from that to $(-\frac{8}{3}, \frac{1}{3}, -\frac{7}{3})\,$? You did not show that. Besides, that answer is incorrect.

10. Jan 12, 2016

### Granger

So what I did was:

I took the square of the length:

$$(x_1)^2 + \frac {2}{9} + \frac {2}{3} x_1$$

And then I calculated the 1st derivative of this expression and zero gave me value for x_1: -8/3

I made a substitution of this value in my original expression and I got my solution

Last edited: Jan 12, 2016
11. Jan 12, 2016

### Ray Vickson

Well, the square of the length is
$$x^2 + y^2 + z^2 =x_1^2 + (1/3)^2 + (x_1 + 1/3)^2$$.
That is a bit different from what you wrote.

12. Jan 12, 2016

### Granger

Right, thanks! I corrected that and now I have as a solution $$(\frac{-1}{6}, \frac{1}{3}, \frac{1}{6})$$ is this correct now?

13. Jan 12, 2016

Yes.