Minimization solution of three equations in two variables

Click For Summary
SUMMARY

The discussion focuses on minimizing the solution of three equations in two variables using optimization techniques. The participant proposes using the "distance from a point to a line" formula to calculate distances from an arbitrary point (X0, Y0) within the triangle formed by the equations. By squaring and summing these distances, they aim to minimize the resulting function D(X0, Y0). The conversation also touches on the least squares solution, specifically referencing the formula X = (A^TA)^{-1}A^TY, which is confirmed to be a straightforward method for achieving this optimization.

PREREQUISITES
  • Understanding of optimization techniques
  • Familiarity with regression analysis
  • Knowledge of linear algebra, specifically matrix operations
  • Proficiency in the "distance from a point to a line" formula
NEXT STEPS
  • Research the application of least squares regression in multiple dimensions
  • Explore optimization algorithms such as gradient descent
  • Learn about the geometric interpretation of linear regression
  • Investigate advanced topics in multivariable calculus related to optimization
USEFUL FOR

Mathematicians, data scientists, and engineers interested in optimization techniques and regression analysis for solving systems of equations.

barryj
Messages
856
Reaction score
51
Homework Statement
given these three equations: I know I have more equations than variables. However, isn't there a way to find the closest solution, some sort of regression solution?
2x - y = -3
-2x - y = -4
-2.1x - y = 4
Relevant Equations
2x - y = -3
-2x - y = -4
-2.1x - y = 4
I do not know the solution.
 
Physics news on Phys.org
I am thinking about how regression is performed. Let's assume I plot the three equations and they form a triange where they intersect. I can use the "distance from a point to a line" formula to get the distance from an arbitrary point , (X0,Y0) within the triangle to each of the lines. I think i could then square and add the distances and try to minimize the resulting function D(X0,Y0) . It seems that this could be extended to find the best fit of multiple lines or even planes. I guess this is the topic of optimization. I do not know if this is a good way or not.
 
barryj said:
isn't there a way to find the closest solution, some sort of regression solution?
Given ##AX=Y##, ##A^TAX=A^TY##.
IF ##A^TA## is nonsingular you have ##X=(A^TA)^{-1}A^TY##.
This can be shown to be the least sum squares solution.
 
Amazing! Hard to imagine it is this simple
Thanks.
 

Similar threads

  • · Replies 5 ·
Replies
5
Views
2K
Replies
7
Views
2K
  • · Replies 9 ·
Replies
9
Views
2K
  • · Replies 27 ·
Replies
27
Views
6K
  • · Replies 5 ·
Replies
5
Views
2K
Replies
3
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 19 ·
Replies
19
Views
2K