• Support PF! Buy your school textbooks, materials and every day products Here!

Least squares problem: am I solving it correctly?

158
7
1. Homework Statement
In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system:
x + y + z = 1
x + z = 0
y = 0

2. The attempt at a solution
So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1) row 2 (1,0,1) and row 3 (0,1,0); x being the column (x_1,x_2,x_3) and b being the column (1,0,0).

So I did it and I reached to the solution (x_1, \frac {1}{3}, \frac {1}{3} + x_1)

And I expanded this solution in two vectors (0, \frac {1}{3}, \frac {1}{3}) and (1,0,1).

So these are the least square solutions and the one with shorter length is the first one.

My doubt is if I'm doing this correctly or if I made any mistake because I used an online calculator that only give one least square solution. Can someone help me to verify my attempt? Thanks!
 

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,720
1. Homework Statement
In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system:
x + y + z = 1
x + z = 0
y = 0

2. The attempt at a solution
So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1) row 2 (1,0,1) and row 3 (0,1,0); x being the column (x_1,x_2,x_3) and b being the column (1,0,0).

So I did it and I reached to the solution (x_1, \frac {1}{3}, \frac {1}{3} + x_1)

And I expanded this solution in two vectors (0, \frac {1}{3}, \frac {1}{3}) and (1,0,1).

So these are the least square solutions and the one with shorter length is the first one.

My doubt is if I'm doing this correctly or if I made any mistake because I used an online calculator that only give one least square solution. Can someone help me to verify my attempt? Thanks!
Your LS solution is OK, but you have not found the one with the shortest length. You need to figure out what value of x_1 leads to the smallest value of ##L##, where ##L## is the "length" of the solution, with ##L^2 = x^2 + y^2 + z^2##. Note that minimizing ##L## will be equivalent to the simpler problem of minimizing ##L^2##, and that just amounts to finding the smallest value of a quadratic function of x_1.

BTW: the reason for the discrepancy is that some LS programs give just one of the LS solutions, even when there are many to choose from. That is because they apply an algorithm in which certain tests are performed and certain steps taken in accord with the tests, but if there are a choice of many possible steps, they just take one of them (chosen according to some well-defined rule, usually).
 
158
7
Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!
 

SteamKing

Staff Emeritus
Science Advisor
Homework Helper
12,794
1,665
Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!
When writing Latex in posts either enclose the expressions in double # for in-line latex or double $ for regular multi-line Latex. Also, use the Preview button located at the Lower right hand corner to check how the formatting will appear before posting.

Example: ##(x_1, \frac{1}{3}, \frac{1}{3} + x_1) ##
 

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,720
Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

*********************
I don't understand your question, but it does not sound right to me. Basically all your LS solutions lie on a line (t, 1/3, t + 1/3) in (x,y,z)-space [because as you vary t those points trace out a line], and you want to find the point on the line that comes closest to the origin (0,0,0). You will not go through the origin, so the distance will never equal 0, but some points on the line are a lot farther away from the origin than others. For some reason, the person setting the question wants the closest point.

*************************


Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!
I don't think your final solution is correct.
 
Last edited:

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,720
158
7
I already explained in my first post !
 

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,720
I already explained in my first post !
No, you didn't. You explained how you got ##(x,y,z) = (x_1, \frac{1}{3}, \frac{1}{3} + x_1)##. But, how do you go from that to ##(-\frac{8}{3}, \frac{1}{3}, -\frac{7}{3})\,##? You did not show that. Besides, that answer is incorrect.
 
158
7
So what I did was:

I took the square of the length:

$$ (x_1)^2 + \frac {2}{9} + \frac {2}{3} x_1 $$

And then I calculated the 1st derivative of this expression and zero gave me value for x_1: -8/3

I made a substitution of this value in my original expression and I got my solution
 
Last edited:

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,720
So what I did was:

I took the square of the length:

$$ (x_1)^2 + \frac {2}{9} + \frac {2}{3} x_1 $$

And then I calculated the 1st derivative of this expression and zero gave me value for x_1: -8/3

I made a substitution of this value in my original expression and I got my solution
Well, the square of the length is
[tex] x^2 + y^2 + z^2 =x_1^2 + (1/3)^2 + (x_1 + 1/3)^2 [/tex].
That is a bit different from what you wrote.
 
158
7
Right, thanks! I corrected that and now I have as a solution $$ (\frac{-1}{6}, \frac{1}{3}, \frac{1}{6}) $$ is this correct now?
 

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,720
Right, thanks! I corrected that and now I have as a solution $$ (\frac{-1}{6}, \frac{1}{3}, \frac{1}{6}) $$ is this correct now?
Yes.
 

Want to reply to this thread?

"Least squares problem: am I solving it correctly?" You must log in or register to reply here.

Related Threads for: Least squares problem: am I solving it correctly?

  • Last Post
Replies
7
Views
2K
Replies
12
Views
3K
  • Last Post
Replies
1
Views
927
  • Last Post
Replies
2
Views
1K
Replies
0
Views
2K
Replies
14
Views
802
  • Last Post
Replies
7
Views
583

Hot Threads

Top