Finding orthonormal set using Gram-Schmidt and least squares

Homework Statement

A) Find an orthonormal set q1, q2, q3 for which q1,q2 span the column space A [1 1; 2, -1; -2,4] (this is a 3x2 matrix).

B) Which fundamental subspace contains q3?

C) What is the least-squares solution of Ax=b if b=[1 2 7]T?

Gram-Schmidt

The Attempt at a Solution

A) So I figured out q1, q2, and q3 using Gram-Schmidt process. These are q1 = [1/3 2/3 -2/3]T, q2 = [2/3 1/3 2/3]T, and q3 = +-[-2/3 2/3 1/3]T

B) Don't know how to determine this

C) For the least square, since they are orthonormal sets, I used the formula x= [q1Tb; q2Tb] However, I'm not getting the right answer. In addition, I tried doing it the long way using the formula x=(ATA)-1ATb, I'm getting even different numbers... Please help!

Answers and Replies

Related Calculus and Beyond Homework Help News on Phys.org
You found the orthonormal set (Q), but that is not equal to A. What is A equal to?

[1 1
2 -1 = A
-2 4]

I'm now only having trouble with part C. Thanks!

Yes, I understand. But there is another way to represent $A$ using the orthogonal basis that you constructed and another factor of $A$. This factor, call it $R$, is an upper right triangular matrix. $R$ is constructed such that $QR = A$. We note $Q$ is orthogonal so $Q^{T} = Q$. Thus, using this factorization for $A$ in the normal equations yields a linear triangualr system for solving the Least Squares problem.

I should add that, as part of the construction of Q, you will have already computed the elements of R.

Thanks Theo! I do understand about A=QR and the construction of it. But I don't understand how that can help me finding x(hat), which is the least-squares solution.

What I was thinking is that x(hat)=Q^T*b because Q has already orthonormal vectors.

I'm getting that x(hat) = (-3 6 3)^T, but it says the solution is supposed to be x(hat) = [1 2]^T which they supposedly got from q1^Tb and q2^Tb. However, if I multiply the very same q's I got (which I was confirmed are correct) by the given b, I'm not getting the given answer. So I'm going in circles... Any help?

Firstly, let me make a correction to one of my previous posts (post #4).

We note $Q$ is orthogonal so $Q^{T}=Q$.
should read

We note $Q$ is orthogonal so $Q^{T}=Q^{-1}$.
Secondly, return to the Normal equations for the Least Squares solution:

$$A^{T}A \, x = A^{T} \, b$$
Now substitute the factor expresion for $A$, ie, $A=QR$
$$(QR)^{T}(QR)\,x = (QR)^{T}b$$
$$R^{T}(Q^{T}Q)\,R\,x = R^{T}Q^{T}b$$
$$R^{T}R\,x = R^{T}Q^{T}b$$
since $Q^{T}Q=I$. Now pre-multiply both sides by $R^{-T}$ to yield
$$R\,x = Q^{T}b$$
It is this system that you need to solve for $x$ to yield the Least Squares solution. Remember, just because you found the orthogonal basis for $A$ doesn't mean its equal to $A$ . There is a certain amount of translation and rotation of that basis to yield that $A$ (hence the $R$ factor).

Remember, $R$ is triangular, so a simple backsolve operation will yield the solution (no need to compute $R^{-1}$).