Undergrad Linear least-squares method and row multiplication of matrix

Click For Summary
In an overdetermined system represented by Ax = b, the least-squares method provides the best approximate solution, denoted as "c." When a specific row of the system is multiplied by a constant k, both the corresponding row in matrix A and vector b must be adjusted, resulting in a new system Bx = d. This transformation alters the weighting of the equations, leading to a different least-squares solution than c. Unlike consistent systems, where all equations can be satisfied simultaneously, the modified equation's increased weight affects the overall solution. Thus, the least-squares method's objective of minimizing deviations results in different outcomes when rows are scaled.
Mesud1
Messages
2
Reaction score
0
Suppose that I have an overdetermined equation system in matrix form:

Ax = b

Where x and b are column vectors, and A has the same number of rows as b, and x has less rows than both.

The least-squares method could be used here to obtain the best possible approximative solution. Let's call this solution "c".

Now, suppose I multiply some row of the equation system with a constant k. Let's say this row is the second row. In that case, I must multiply the 2nd row of A with k, as well as the 2nd row of b. This yields a new equation system, let's write it as:

Bx = d

If I use the method of least squares on the second system, I get a new solution that is different from c. Why is the solution different? Since I performed an elementary row operation on the first system to obtain the second system, shouldn't the two systems be equivalent, and therefore have the same least-squares solution?

When I did the same thing with a consistent system, I got the same solution for both systems.
 
Mathematics news on Phys.org
The least squares method minimizes the sum of the deviations of the left hand from the right hand side. If you multiply one equation by c, this equation gets more weight in the sum and the optimal solution will be different. This doesn't happen if all equations can be fulfilled identically.
 
  • Like
Likes Mesud1
DrDu said:
The least squares method minimizes the sum of the deviations of the left hand from the right hand side. If you multiply one equation by c, this equation gets more weight in the sum and the optimal solution will be different. This doesn't happen if all equations can be fulfilled identically.

Makes perfect sense, thank you.
 

Thread 'Area of Overlapping Squares'

Here is a little puzzle from the book 100 Geometric Games by Pierre Berloquin. The side of a small square is one meter long and the side of a larger square one and a half meters long. One vertex of the large square is at the center of the small square. The side of the large square cuts two sides of the small square into one- third parts and two-thirds parts. What is the area where the squares overlap?
View full post »

Similar threads

High School Can elementary matrix operations change the solutions to a system of equations?
  • · Replies 10 ·
Replies
10
Views
2K
Python Partial pivoting in getting the reduced row echelon form of a matrix
  • · Replies 2 ·
Replies
2
Views
2K
Do Row Operations Maintain Equivalence in Matrix Subtraction?
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad Row space, Column space, Null space & Left null space
  • · Replies 8 ·
Replies
8
Views
3K
Undergrad How do Tensors "work" in relation to linear algebraic objects?
  • · Replies 7 ·
Replies
7
Views
738
MHB System of equations - Relative error
  • · Replies 16 ·
Replies
16
Views
4K
Transformations to both sides of a matrix equation
  • · Replies 25 ·
Replies
25
Views
2K
Undergrad Vector subspace and basis vectors in the context of data science
  • · Replies 6 ·
Replies
6
Views
3K
Undergrad Dimension and solution to matrix-vector product
  • · Replies 4 ·
Replies
4
Views
3K
High School Attempt to solve this system of three linear equations
  • · Replies 10 ·
Replies
10
Views
2K