# Finding orthonormal set using Gram-Schmidt and least squares

by tatianaiistb
Tags: gram-schmidt, least squares, subspaces
 P: 47 1. The problem statement, all variables and given/known data A) Find an orthonormal set q1, q2, q3 for which q1,q2 span the column space A [1 1; 2, -1; -2,4] (this is a 3x2 matrix). B) Which fundamental subspace contains q3? C) What is the least-squares solution of Ax=b if b=[1 2 7]T? 2. Relevant equations Gram-Schmidt 3. The attempt at a solution A) So I figured out q1, q2, and q3 using Gram-Schmidt process. These are q1 = [1/3 2/3 -2/3]T, q2 = [2/3 1/3 2/3]T, and q3 = +-[-2/3 2/3 1/3]T B) Don't know how to determine this C) For the least square, since they are orthonormal sets, I used the formula x= [q1Tb; q2Tb] However, I'm not getting the right answer. In addition, I tried doing it the long way using the formula x=(ATA)-1ATb, I'm getting even different numbers... Please help! 1. The problem statement, all variables and given/known data 2. Relevant equations 3. The attempt at a solution
 P: 182 You found the orthonormal set (Q), but that is not equal to A. What is A equal to?
 P: 47 [1 1 2 -1 = A -2 4] I'm now only having trouble with part C. Thanks!
 P: 182 Finding orthonormal set using Gram-Schmidt and least squares Yes, I understand. But there is another way to represent $A$ using the orthogonal basis that you constructed and another factor of $A$. This factor, call it $R$, is an upper right triangular matrix. $R$ is constructed such that $QR = A$. We note $Q$ is orthogonal so $Q^{T} = Q$. Thus, using this factorization for $A$ in the normal equations yields a linear triangualr system for solving the Least Squares problem.
 P: 182 I should add that, as part of the construction of Q, you will have already computed the elements of R.
 P: 47 Thanks Theo! I do understand about A=QR and the construction of it. But I don't understand how that can help me finding x(hat), which is the least-squares solution. What I was thinking is that x(hat)=Q^T*b because Q has already orthonormal vectors. I'm getting that x(hat) = (-3 6 3)^T, but it says the solution is supposed to be x(hat) = [1 2]^T which they supposedly got from q1^Tb and q2^Tb. However, if I multiply the very same q's I got (which I was confirmed are correct) by the given b, I'm not getting the given answer. So I'm going in circles... Any help?
P: 182
Firstly, let me make a correction to one of my previous posts (post #4).

 We note $Q$ is orthogonal so $Q^{T}=Q$.

 We note $Q$ is orthogonal so $Q^{T}=Q^{-1}$.
Secondly, return to the Normal equations for the Least Squares solution:

$$A^{T}A \, x = A^{T} \, b$$
Now substitute the factor expresion for $A$, ie, $A=QR$
$$(QR)^{T}(QR)\,x = (QR)^{T}b$$
$$R^{T}(Q^{T}Q)\,R\,x = R^{T}Q^{T}b$$
$$R^{T}R\,x = R^{T}Q^{T}b$$
since $Q^{T}Q=I$. Now pre-multiply both sides by $R^{-T}$ to yield
$$R\,x = Q^{T}b$$
It is this system that you need to solve for $x$ to yield the Least Squares solution. Remember, just because you found the orthogonal basis for $A$ doesn't mean its equal to $A$ . There is a certain amount of translation and rotation of that basis to yield that $A$ (hence the $R$ factor).

Remember, $R$ is triangular, so a simple backsolve operation will yield the solution (no need to compute $R^{-1}$).

 Related Discussions Calculus & Beyond Homework 5 Calculus & Beyond Homework 6 Calculus & Beyond Homework 6 Linear & Abstract Algebra 9 Linear & Abstract Algebra 2