Linear Algebra problem (Least Squares?)

Click For Summary
SUMMARY

The discussion centers on solving a linear algebra problem involving the least squares method to minimize the distance between two points R = (x,x,x) and S = (y,3y,-1) in 3-dimensional space. The user attempts to apply the equation ATAx = ATb but encounters issues resulting in a zero matrix, indicating a misunderstanding of the application. The correct approach involves minimizing the expression ||R - S||² using gradient descent, specifically setting the gradient ∇||R - S||² = 0 to find optimal values for x and y.

PREREQUISITES
  • Understanding of linear algebra concepts, particularly least squares optimization.
  • Familiarity with matrix operations, including matrix inversion and multiplication.
  • Knowledge of gradient descent and its application in optimization problems.
  • Experience with partial derivatives and their role in finding minima of functions.
NEXT STEPS
  • Study the derivation and application of the least squares method in linear algebra.
  • Learn about gradient descent techniques for optimizing multivariable functions.
  • Explore matrix calculus, focusing on the differentiation of matrix expressions.
  • Practice solving optimization problems using numerical methods in software like MATLAB or Python.
USEFUL FOR

Students and professionals in mathematics, engineering, and data science who are dealing with optimization problems in linear algebra, particularly those focused on least squares methods.

Pratha
Messages
2
Reaction score
0
Linear Algebra problem (Least Squares? - Distance between lines)

Homework Statement



We have two points R = (x,x,x) and S = (y,3y,-1). All we know is that they are on lines somewhere in 3-space and that they don't cross. Need to find an x and y that minimize || R - S ||2

Homework Equations



ATAx = ATb

The Attempt at a Solution



I tried using the equation above, i.e. inverting (ATA) and multiplying both sides with that, but the resulting matrix that I got was a 2x1 matrix of zeros. This is definitely not the right answer. I also tried using (C+D(t)-b)2... for each coord and doing a partial derivative for C and D, but I ended up getting the same equation for both derivatives, which I am sure is not right.

I am very confused and not sure where to go from here.
 
Last edited:
Physics news on Phys.org
Why don't you minimize it in the usual way, \nabla ||R-S||^2 = 0 ?
 
clamtrox said:
Why don't you minimize it in the usual way, \nabla ||R-S||^2 = 0 ?

That is what I tried. At least that's what I think I tried. That was where the (C+D(t)-b)2... etc, was about in my previous post. (C+Dx - y)2 + (C+Dx - 3y)2 + (C+Dx + 1)2.

But, since the t (x) values are all x's, they cancel with the two's after I do the partial derivative w/respect to D, and both derivatives end up the same. Is there something I'm missing, or am I doing something wrong?
 

Similar threads

  • · Replies 25 ·
Replies
25
Views
4K
  • · Replies 10 ·
Replies
10
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 22 ·
Replies
22
Views
2K
Replies
8
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
Replies
1
Views
2K