Least squares problem: am I solving it correctly?

In summary, the LS solution for this system is (x_1, \frac{1}{3}, \frac{1}{3} + x_1), which is closest to the origin.
  • #1
Granger
168
7

Homework Statement


In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system:
x + y + z = 1
x + z = 0
y = 0

2. The attempt at a solution
So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1) row 2 (1,0,1) and row 3 (0,1,0); x being the column (x_1,x_2,x_3) and b being the column (1,0,0).

So I did it and I reached to the solution (x_1, \frac {1}{3}, \frac {1}{3} + x_1)

And I expanded this solution in two vectors (0, \frac {1}{3}, \frac {1}{3}) and (1,0,1).

So these are the least square solutions and the one with shorter length is the first one.

My doubt is if I'm doing this correctly or if I made any mistake because I used an online calculator that only give one least square solution. Can someone help me to verify my attempt? Thanks!
 
Physics news on Phys.org
  • #2
Granger said:

Homework Statement


In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system:
x + y + z = 1
x + z = 0
y = 0

2. The attempt at a solution
So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1) row 2 (1,0,1) and row 3 (0,1,0); x being the column (x_1,x_2,x_3) and b being the column (1,0,0).

So I did it and I reached to the solution (x_1, \frac {1}{3}, \frac {1}{3} + x_1)

And I expanded this solution in two vectors (0, \frac {1}{3}, \frac {1}{3}) and (1,0,1).

So these are the least square solutions and the one with shorter length is the first one.

My doubt is if I'm doing this correctly or if I made any mistake because I used an online calculator that only give one least square solution. Can someone help me to verify my attempt? Thanks!

Your LS solution is OK, but you have not found the one with the shortest length. You need to figure out what value of x_1 leads to the smallest value of ##L##, where ##L## is the "length" of the solution, with ##L^2 = x^2 + y^2 + z^2##. Note that minimizing ##L## will be equivalent to the simpler problem of minimizing ##L^2##, and that just amounts to finding the smallest value of a quadratic function of x_1.

BTW: the reason for the discrepancy is that some LS programs give just one of the LS solutions, even when there are many to choose from. That is because they apply an algorithm in which certain tests are performed and certain steps taken in accord with the tests, but if there are a choice of many possible steps, they just take one of them (chosen according to some well-defined rule, usually).
 
  • #3
Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!
 
  • #4
Granger said:
Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!
When writing Latex in posts either enclose the expressions in double # for in-line latex or double $ for regular multi-line Latex. Also, use the Preview button located at the Lower right hand corner to check how the formatting will appear before posting.

Example: ##(x_1, \frac{1}{3}, \frac{1}{3} + x_1) ##
 
  • Like
Likes Granger
  • #5
Granger said:
Thanks for the reply!

So in that case I need to minimize the square of the length of (x_1, \frac{1}{3}, \frac{1}{3} + x_1) right? Because if I minimize the square length of (x_1, 0, x_1) I reach to a zero solution, right?

*********************
I don't understand your question, but it does not sound right to me. Basically all your LS solutions lie on a line (t, 1/3, t + 1/3) in (x,y,z)-space [because as you vary t those points trace out a line], and you want to find the point on the line that comes closest to the origin (0,0,0). You will not go through the origin, so the distance will never equal 0, but some points on the line are a lot farther away from the origin than others. For some reason, the person setting the question wants the closest point.

*************************


Than I can write that the family of the least squares solution is (x_1, \frac{1}{3}, \frac{1}{3} + x_1). And the minimum solution is given by (\frac{-8}{3}, \frac{1}{3}, \frac{-7}{3})

Thanks for the help!

I don't think your final solution is correct.
 
Last edited:
  • #6
Ray Vickson said:
I don't think your final solution is correct.

Then how do you think I should do it? I'm not understanding...
 
  • #7
Granger said:
Then how do you think I should do it? I'm not understanding...

What did you do? Show your work.
 
  • #8
I already explained in my first post !
 
  • #9
Granger said:
I already explained in my first post !

No, you didn't. You explained how you got ##(x,y,z) = (x_1, \frac{1}{3}, \frac{1}{3} + x_1)##. But, how do you go from that to ##(-\frac{8}{3}, \frac{1}{3}, -\frac{7}{3})\,##? You did not show that. Besides, that answer is incorrect.
 
  • #10
So what I did was:

I took the square of the length:

$$ (x_1)^2 + \frac {2}{9} + \frac {2}{3} x_1 $$

And then I calculated the 1st derivative of this expression and zero gave me value for x_1: -8/3

I made a substitution of this value in my original expression and I got my solution
 
Last edited:
  • #11
Granger said:
So what I did was:

I took the square of the length:

$$ (x_1)^2 + \frac {2}{9} + \frac {2}{3} x_1 $$

And then I calculated the 1st derivative of this expression and zero gave me value for x_1: -8/3

I made a substitution of this value in my original expression and I got my solution

Well, the square of the length is
[tex] x^2 + y^2 + z^2 =x_1^2 + (1/3)^2 + (x_1 + 1/3)^2 [/tex].
That is a bit different from what you wrote.
 
  • #12
Right, thanks! I corrected that and now I have as a solution $$ (\frac{-1}{6}, \frac{1}{3}, \frac{1}{6}) $$ is this correct now?
 
  • #13
Granger said:
Right, thanks! I corrected that and now I have as a solution $$ (\frac{-1}{6}, \frac{1}{3}, \frac{1}{6}) $$ is this correct now?
Yes.
 

1. What is the least squares problem?

The least squares problem is a mathematical optimization technique used to find the best fit line or curve for a set of data points. It involves minimizing the sum of the squared differences between the actual data points and the predicted values from the model.

2. How do I know if I am solving the least squares problem correctly?

You can check if you are solving the least squares problem correctly by verifying that the predicted values from your model have the smallest possible sum of squared differences compared to the actual data points. Additionally, you can use statistical tests, such as the coefficient of determination (R2), to assess the accuracy of your model.

3. What are the assumptions for a correct least squares solution?

The assumptions for a correct least squares solution include linearity (the relationship between variables can be described by a straight line), independence of errors (the errors should not be correlated with each other), and homoscedasticity (the variance of the errors should be constant across all values of the independent variable).

4. Can I use the least squares method for any type of data?

The least squares method can be used for both continuous and categorical data, however, there are certain assumptions that need to be met for the results to be valid. For example, for categorical data, the dependent variable should be binary (e.g. yes or no) and the independent variable should be continuous.

5. Are there any limitations to the least squares method?

One limitation of the least squares method is that it assumes a linear relationship between the variables, which may not always be the case in real-world data. Additionally, it is sensitive to outliers and may not perform well if the data is heavily skewed or has a high degree of multicollinearity.

Similar threads

  • Precalculus Mathematics Homework Help
Replies
32
Views
723
  • Precalculus Mathematics Homework Help
Replies
21
Views
1K
  • Precalculus Mathematics Homework Help
Replies
1
Views
456
  • Precalculus Mathematics Homework Help
2
Replies
57
Views
3K
  • Precalculus Mathematics Homework Help
Replies
2
Views
464
  • Precalculus Mathematics Homework Help
Replies
5
Views
394
  • Precalculus Mathematics Homework Help
Replies
21
Views
705
  • Precalculus Mathematics Homework Help
Replies
1
Views
467
  • Precalculus Mathematics Homework Help
Replies
6
Views
1K
  • Precalculus Mathematics Homework Help
Replies
5
Views
761
Back
Top