1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Finding orthonormal set using Gram-Schmidt and least squares

  1. Oct 26, 2011 #1
    1. The problem statement, all variables and given/known data

    A) Find an orthonormal set q1, q2, q3 for which q1,q2 span the column space A [1 1; 2, -1; -2,4] (this is a 3x2 matrix).

    B) Which fundamental subspace contains q3?

    C) What is the least-squares solution of Ax=b if b=[1 2 7]T?

    2. Relevant equations


    3. The attempt at a solution

    A) So I figured out q1, q2, and q3 using Gram-Schmidt process. These are q1 = [1/3 2/3 -2/3]T, q2 = [2/3 1/3 2/3]T, and q3 = +-[-2/3 2/3 1/3]T

    B) Don't know how to determine this

    C) For the least square, since they are orthonormal sets, I used the formula x= [q1Tb; q2Tb] However, I'm not getting the right answer. In addition, I tried doing it the long way using the formula x=(ATA)-1ATb, I'm getting even different numbers... Please help!
    1. The problem statement, all variables and given/known data

    2. Relevant equations

    3. The attempt at a solution
  2. jcsd
  3. Oct 27, 2011 #2
    You found the orthonormal set (Q), but that is not equal to A. What is A equal to?
  4. Oct 27, 2011 #3
    [1 1
    2 -1 = A
    -2 4]

    I'm now only having trouble with part C. Thanks!
  5. Oct 27, 2011 #4
    Yes, I understand. But there is another way to represent [itex]A[/itex] using the orthogonal basis that you constructed and another factor of [itex]A[/itex]. This factor, call it [itex]R[/itex], is an upper right triangular matrix. [itex]R[/itex] is constructed such that [itex]QR = A[/itex]. We note [itex]Q[/itex] is orthogonal so [itex]Q^{T} = Q[/itex]. Thus, using this factorization for [itex]A[/itex] in the normal equations yields a linear triangualr system for solving the Least Squares problem.
  6. Oct 27, 2011 #5
    I should add that, as part of the construction of Q, you will have already computed the elements of R.
  7. Oct 27, 2011 #6
    Thanks Theo! I do understand about A=QR and the construction of it. But I don't understand how that can help me finding x(hat), which is the least-squares solution.

    What I was thinking is that x(hat)=Q^T*b because Q has already orthonormal vectors.

    I'm getting that x(hat) = (-3 6 3)^T, but it says the solution is supposed to be x(hat) = [1 2]^T which they supposedly got from q1^Tb and q2^Tb. However, if I multiply the very same q's I got (which I was confirmed are correct) by the given b, I'm not getting the given answer. So I'm going in circles... Any help?
  8. Oct 28, 2011 #7
    Firstly, let me make a correction to one of my previous posts (post #4).

    should read

    Secondly, return to the Normal equations for the Least Squares solution:

    A^{T}A \, x = A^{T} \, b
    Now substitute the factor expresion for [itex]A[/itex], ie, [itex]A=QR[/itex]
    [tex](QR)^{T}(QR)\,x = (QR)^{T}b[/tex]
    [tex]R^{T}(Q^{T}Q)\,R\,x = R^{T}Q^{T}b[/tex]
    [tex]R^{T}R\,x = R^{T}Q^{T}b[/tex]
    since [itex]Q^{T}Q=I[/itex]. Now pre-multiply both sides by [itex]R^{-T}[/itex] to yield
    [tex]R\,x = Q^{T}b[/tex]
    It is this system that you need to solve for [itex]x[/itex] to yield the Least Squares solution. Remember, just because you found the orthogonal basis for [itex]A[/itex] doesn't mean its equal to [itex]A[/itex] . There is a certain amount of translation and rotation of that basis to yield that [itex]A[/itex] (hence the [itex]R[/itex] factor).

    Remember, [itex]R[/itex] is triangular, so a simple backsolve operation will yield the solution (no need to compute [itex]R^{-1}[/itex]).
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook