Finding orthonormal set using GramSchmidt and least squaresby tatianaiistb Tags: gramschmidt, least squares, subspaces 

#1
Oct2611, 09:20 PM

P: 47

1. The problem statement, all variables and given/known data
A) Find an orthonormal set q1, q2, q3 for which q1,q2 span the column space A [1 1; 2, 1; 2,4] (this is a 3x2 matrix). B) Which fundamental subspace contains q3? C) What is the leastsquares solution of Ax=b if b=[1 2 7]^{T}? 2. Relevant equations GramSchmidt 3. The attempt at a solution A) So I figured out q1, q2, and q3 using GramSchmidt process. These are q1 = [1/3 2/3 2/3]^{T}, q2 = [2/3 1/3 2/3]^{T}, and q3 = +[2/3 2/3 1/3]^{T} B) Don't know how to determine this C) For the least square, since they are orthonormal sets, I used the formula x= [q1^{T}b; q2^{T}b] However, I'm not getting the right answer. In addition, I tried doing it the long way using the formula x=(A^{T}A)^{1}A^{T}b, I'm getting even different numbers... Please help! 1. The problem statement, all variables and given/known data 2. Relevant equations 3. The attempt at a solution 



#2
Oct2711, 12:06 PM

P: 181

You found the orthonormal set (Q), but that is not equal to A. What is A equal to?




#3
Oct2711, 01:46 PM

P: 47

[1 1
2 1 = A 2 4] I'm now only having trouble with part C. Thanks! 



#4
Oct2711, 02:00 PM

P: 181

Finding orthonormal set using GramSchmidt and least squares
Yes, I understand. But there is another way to represent [itex]A[/itex] using the orthogonal basis that you constructed and another factor of [itex]A[/itex]. This factor, call it [itex]R[/itex], is an upper right triangular matrix. [itex]R[/itex] is constructed such that [itex]QR = A[/itex]. We note [itex]Q[/itex] is orthogonal so [itex]Q^{T} = Q[/itex]. Thus, using this factorization for [itex]A[/itex] in the normal equations yields a linear triangualr system for solving the Least Squares problem.




#5
Oct2711, 02:02 PM

P: 181

I should add that, as part of the construction of Q, you will have already computed the elements of R.




#6
Oct2711, 06:50 PM

P: 47

Thanks Theo! I do understand about A=QR and the construction of it. But I don't understand how that can help me finding x(hat), which is the leastsquares solution.
What I was thinking is that x(hat)=Q^T*b because Q has already orthonormal vectors. I'm getting that x(hat) = (3 6 3)^T, but it says the solution is supposed to be x(hat) = [1 2]^T which they supposedly got from q1^Tb and q2^Tb. However, if I multiply the very same q's I got (which I was confirmed are correct) by the given b, I'm not getting the given answer. So I'm going in circles... Any help? 



#7
Oct2811, 06:54 AM

P: 181

Firstly, let me make a correction to one of my previous posts (post #4).
[tex] A^{T}A \, x = A^{T} \, b [/tex] Now substitute the factor expresion for [itex]A[/itex], ie, [itex]A=QR[/itex] [tex](QR)^{T}(QR)\,x = (QR)^{T}b[/tex] [tex]R^{T}(Q^{T}Q)\,R\,x = R^{T}Q^{T}b[/tex] [tex]R^{T}R\,x = R^{T}Q^{T}b[/tex] since [itex]Q^{T}Q=I[/itex]. Now premultiply both sides by [itex]R^{T}[/itex] to yield [tex]R\,x = Q^{T}b[/tex] It is this system that you need to solve for [itex]x[/itex] to yield the Least Squares solution. Remember, just because you found the orthogonal basis for [itex]A[/itex] doesn't mean its equal to [itex]A[/itex] . There is a certain amount of translation and rotation of that basis to yield that [itex]A[/itex] (hence the [itex]R[/itex] factor). Remember, [itex]R[/itex] is triangular, so a simple backsolve operation will yield the solution (no need to compute [itex]R^{1}[/itex]). 


Register to reply 
Related Discussions  
GramSchmidt procedure to find orthonormal basis  Calculus & Beyond Homework  5  
Orthonormal basis/GramSchmidt [Easy?]  Calculus & Beyond Homework  6  
GramSchmidt orthonormal basis  Calculus & Beyond Homework  6  
GramSchmidt problem  I'm stuck  Linear & Abstract Algebra  9  
Gram Schmidt  Linear & Abstract Algebra  2 