Finding Least Square Solution for a System of Linear Equations

In summary, the question is about finding the least square solution for a given matrix and vector. The solution involves finding the orthogonal projection of the two vectors and expressing it in terms of the given basis vectors. However, there are some complications such as the basis vectors needing to be orthogonal and the use of Gram-Schmidt conversion in order to apply the orthogonal projection method.
  • #1
Mindscrape
1,861
1
This is a pretty basic question, but I just want to make sure. The question is to find the least square solution for
[tex] \newcommand{\colv}[2] {\left(\begin{array}{c} #1 \\ #2 \end{array}\right)}
\left(
\begin{array}{cc}
1 & 2\\
2 & 4
\end{array}
\right) x = \colv{2}{2}[/tex]

I can just find the orthogonal projection of the two vectors, right? In other words, use the find [tex]w = a_1\mathbf{v_1} + a_2\mathbf{v_2}[/tex]
[tex]\newcommand{\colv}[2] {\left(\begin{array}{c} #1 \\ #2 \end{array}\right)}
a_1 {\colv{1}{2}} + a_2 {\colv{2}{4}} = {\colv{w_1}{w_2}}[/tex]
where
[tex] a_1 = \frac{<b,v_1>}{||v_1||^2}[/tex]
and
[tex] a_2 = \frac{<b,v_2>}{||v_2||^2}[/tex]

I don't really want to use the method with positive definite because the numbers turn out sticky.

Then solution can be expressed as [tex]\mathbf{z} = \mathbf{b} - \mathbf{w}[/tex], such that z is orthogonal to the range of K?
 
Last edited:
Physics news on Phys.org
  • #2
Two problems I just thought of though. If I want to do an orthogonal projection the basis vectors must be orthogonal. I suppose I could convert to an orthogonal basis.

Furthermore, K^T K = Kx will still not give an answer such that b is in the range of K. No wonder this problem was assigned, it's not as basic as I thought. :(

So if I convert the second vector to an orthogonal vector with Gram-Schmitt then I can use the orthogonal projection as the least squares answer?
 

What is the purpose of finding the least square solution for a system of linear equations?

The purpose of finding the least square solution for a system of linear equations is to find the best fit line or curve that minimizes the sum of the squared errors between the actual data points and the predicted values. This method is particularly useful when the system of equations is inconsistent or when there are more equations than unknowns.

How is the least square solution calculated?

The least square solution is calculated by minimizing the sum of the squared errors using a mathematical formula known as the Normal Equations. This involves taking the partial derivatives of the sum of squared errors with respect to each unknown variable, setting them equal to zero, and solving the resulting system of equations.

What are the advantages of using the least square solution method?

The least square solution method is advantageous because it allows for a unique solution to be found for a system of linear equations, even when the system is inconsistent. It also provides a best fit line or curve that can be used for predictive modeling and forecasting. Additionally, it is a widely used and well-studied method that is easy to implement.

What are the limitations of the least square solution method?

The least square solution method may not always provide the best fit for a given data set, as it only considers the sum of the squared errors and does not take into account the distribution of the errors. It also assumes that the errors are normally distributed and independent, which may not always be the case. Additionally, the method can be computationally intensive for large systems of equations.

Are there any real-world applications of the least square solution method?

Yes, the least square solution method has numerous real-world applications, including linear regression analysis in statistics, curve fitting in engineering and physics, and image and signal processing. It is also commonly used in finance for time series analysis and forecasting. Additionally, many optimization problems can be solved using the least square solution method.

Similar threads

  • Calculus and Beyond Homework Help
Replies
2
Views
367
  • Calculus and Beyond Homework Help
Replies
5
Views
291
  • Calculus and Beyond Homework Help
Replies
9
Views
1K
  • Calculus and Beyond Homework Help
Replies
0
Views
156
  • Calculus and Beyond Homework Help
Replies
6
Views
536
  • Calculus and Beyond Homework Help
Replies
2
Views
526
  • Calculus and Beyond Homework Help
Replies
6
Views
1K
  • Calculus and Beyond Homework Help
Replies
2
Views
196
  • Calculus and Beyond Homework Help
Replies
7
Views
825
  • Calculus and Beyond Homework Help
Replies
7
Views
708
Back
Top