Least squares problem: am I solving it correctly?

[Granger]

Homework Statement

In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system:
x + y + z = 1
x + z = 0
y = 0

2. The attempt at a solution
So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1) row 2 (1,0,1) and row 3 (0,1,0); x being the column (x_1,x_2,x_3) and b being the column (1,0,0).

So I did it and I reached to the solution (x_1, \frac {1}{3}, \frac {1}{3} + x_1)

And I expanded this solution in two vectors (0, \frac {1}{3}, \frac {1}{3}) and (1,0,1).

So these are the least square solutions and the one with shorter length is the first one.

My doubt is if I'm doing this correctly or if I made any mistake because I used an online calculator that only give one least square solution. Can someone help me to verify my attempt? Thanks!

 

[Ray Vickson]

Homework Statement

In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system:
x + y + z = 1
x + z = 0
y = 0

2. The attempt at a solution
So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1) row 2 (1,0,1) and row 3 (0,1,0); x being the column (x_1,x_2,x_3) and b being the column (1,0,0).

So I did it and I reached to the solution (x_1, \frac {1}{3}, \frac {1}{3} + x_1)

And I expanded this solution in two vectors (0, \frac {1}{3}, \frac {1}{3}) and (1,0,1).

So these are the least square solutions and the one with shorter length is the first one.

My doubt is if I'm doing this correctly or if I made any mistake because I used an online calculator that only give one least square solution. Can someone help me to verify my attempt? Thanks!

Your LS solution is OK, but you have not found the one with the shortest length. You need to figure out what value of x_1 leads to the smallest value of $L$, where $L$ is the "length" of the solution, with $L^2 = x^2 + y^2 + z^2$. Note that minimizing $L$ will be equivalent to the simpler problem of minimizing $L^2$, and that just amounts to finding the smallest value of a quadratic function of x_1.

BTW: the reason for the discrepancy is that some LS programs give just one of the LS solutions, even when there are many to choose from. That is because they apply an algorithm in which certain tests are performed and certain steps taken in accord with the tests, but if there are a choice of many possible steps, they just take one of them (chosen according to some well-defined rule, usually).

 

[Granger]

Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!

 

[SteamKing]

Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!

When writing Latex in posts either enclose the expressions in double # for in-line latex or double $ for regular multi-line Latex. Also, use the Preview button located at the Lower right hand corner to check how the formatting will appear before posting.

Example: $(x_1, \frac{1}{3}, \frac{1}{3} + x_1)$

 

[Ray Vickson]

Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

*********************
I don't understand your question, but it does not sound right to me. Basically all your LS solutions lie on a line (t, 1/3, t + 1/3) in (x,y,z)-space [because as you vary t those points trace out a line], and you want to find the point on the line that comes closest to the origin (0,0,0). You will not go through the origin, so the distance will never equal 0, but some points on the line are a lot farther away from the origin than others. For some reason, the person setting the question wants the closest point.

*************************

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!

I don't think your final solution is correct.

 

[Granger]

I don't think your final solution is correct.

Then how do you think I should do it? I'm not understanding...

 

[Ray Vickson]

Then how do you think I should do it? I'm not understanding...

What did you do? Show your work.

 

[Granger]

I already explained in my first post !

 

[Ray Vickson]

I already explained in my first post !

No, you didn't. You explained how you got $(x,y,z) = (x_1, \frac{1}{3}, \frac{1}{3} + x_1)$. But, how do you go from that to $(-\frac{8}{3}, \frac{1}{3}, -\frac{7}{3})\,$? You did not show that. Besides, that answer is incorrect.

 

[Granger]

So what I did was:

I took the square of the length:

$$ (x_1)^2 + \frac {2}{9} + \frac {2}{3} x_1 $$

And then I calculated the 1st derivative of this expression and zero gave me value for x_1: -8/3

I made a substitution of this value in my original expression and I got my solution

 

[Ray Vickson]

So what I did was:

I took the square of the length:

$$ (x_1)^2 + \frac {2}{9} + \frac {2}{3} x_1 $$

And then I calculated the 1st derivative of this expression and zero gave me value for x_1: -8/3

I made a substitution of this value in my original expression and I got my solution

Well, the square of the length is
x^2 + y^2 + z^2 =x_1^2 + (1/3)^2 + (x_1 + 1/3)^2.
That is a bit different from what you wrote.

 

[Granger]

Right, thanks! I corrected that and now I have as a solution $$ (\frac{-1}{6}, \frac{1}{3}, \frac{1}{6}) $$ is this correct now?

 

[Ray Vickson]

Right, thanks! I corrected that and now I have as a solution $$ (\frac{-1}{6}, \frac{1}{3}, \frac{1}{6}) $$ is this correct now?

Yes.