Finding Least Square Solution for a System of Linear Equations

Click For Summary
SUMMARY

The discussion focuses on finding the least squares solution for the linear equation system represented by the matrix \(\begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix} x = \begin{pmatrix} 2 \\ 2 \end{pmatrix}\). The user explores using orthogonal projections and the Gram-Schmidt process to convert the basis vectors into orthogonal vectors for accurate projections. They express the solution as \(\mathbf{z} = \mathbf{b} - \mathbf{w}\), emphasizing the need for orthogonality in the basis vectors to achieve a valid least squares solution. The discussion reveals the complexities involved in ensuring that the vector \(\mathbf{b}\) lies within the range of matrix \(K\).

PREREQUISITES
  • Understanding of linear algebra concepts, particularly least squares solutions
  • Familiarity with orthogonal projections and their mathematical formulation
  • Knowledge of the Gram-Schmidt process for orthogonalization
  • Proficiency in vector operations, including dot products and norms
NEXT STEPS
  • Study the Gram-Schmidt process in detail to understand orthogonalization of vectors
  • Learn about orthogonal projections and their applications in least squares problems
  • Explore the properties of positive definite matrices and their relevance in solving linear systems
  • Investigate the implications of the rank of a matrix on the existence of least squares solutions
USEFUL FOR

Students and professionals in mathematics, engineering, and data science who are working with linear equations and optimization techniques, particularly those interested in least squares methods and vector projections.

Mindscrape
Messages
1,854
Reaction score
1
This is a pretty basic question, but I just want to make sure. The question is to find the least square solution for
\newcommand{\colv}[2] {\left(\begin{array}{c} #1 \\ #2 \end{array}\right)}<br /> \left(<br /> \begin{array}{cc}<br /> 1 &amp; 2\\<br /> 2 &amp; 4<br /> \end{array}<br /> \right) x = \colv{2}{2}

I can just find the orthogonal projection of the two vectors, right? In other words, use the find w = a_1\mathbf{v_1} + a_2\mathbf{v_2}
\newcommand{\colv}[2] {\left(\begin{array}{c} #1 \\ #2 \end{array}\right)}<br /> a_1 {\colv{1}{2}} + a_2 {\colv{2}{4}} = {\colv{w_1}{w_2}}
where
a_1 = \frac{&lt;b,v_1&gt;}{||v_1||^2}
and
a_2 = \frac{&lt;b,v_2&gt;}{||v_2||^2}

I don't really want to use the method with positive definite because the numbers turn out sticky.

Then solution can be expressed as \mathbf{z} = \mathbf{b} - \mathbf{w}, such that z is orthogonal to the range of K?
 
Last edited:
Physics news on Phys.org
Two problems I just thought of though. If I want to do an orthogonal projection the basis vectors must be orthogonal. I suppose I could convert to an orthogonal basis.

Furthermore, K^T K = Kx will still not give an answer such that b is in the range of K. No wonder this problem was assigned, it's not as basic as I thought. :(

So if I convert the second vector to an orthogonal vector with Gram-Schmitt then I can use the orthogonal projection as the least squares answer?
 

Similar threads

  • · Replies 3 ·
Replies
3
Views
2K
Replies
6
Views
2K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
9
Views
2K
Replies
0
Views
1K
  • · Replies 2 ·
Replies
2
Views
2K
Replies
6
Views
1K