Orthogonal basis to find projection onto a subspace

In summary: If we use ##\mathbb P_n##, the space of real polynomials of desgee ##\le n##, use the standard basis in this space.
  • #1
Mr Davis 97
1,462
44
I know that to find the projection of an element in R^n on a subspace W, we need to have an orthogonal basis in W, and then applying the formula formula for projections.
However, I don;t understand why we must have an orthogonal basis in W in order to calculate the projection of another vector onto W. Why can't we use just some normal basis in W. Is it to make the computation easier, or is there a deeper reason why we can't find projections without orthogonal bases?
 
Physics news on Phys.org
  • #2
It's not that we can't calculate the projection without an orthogonal basis for W, it's that the usual and simplest method of calculating that projection has to use an orthogonal basis. That method is as follows. Let ##B=\{\vec v_1,...,\vec v_m\}## be an orthogonal basis for subspace ##W## of vector space ##V## and let ##u\in V##.

Then the projection ##\pi_W(\vec u)## of ##\vec u## onto ##W## is:
$$\sum_{k=1}^m \vec v_k\frac{\vec u\cdot \vec v_k}{\|\vec v_k\|^2}$$

That formula doesn't work if ##B## is not orthogonal because it will double count some components of the projection vector, because the terms of that sum may not be mutually orthogonal. To convince yourself of that, try working out the projection of ##(1, 1, 1)## onto the vector space spanned by orthogonal basis ##(0,1,0),\ (0, 0,1)## and compare that with what you get when you use the above formula with non-orthogonal basis ##(0,1,0),\ (0, 1,1)##.

In most cases (possibly always?) it is possible to derive an orthogonal basis from a non-orthogonal one, so that the above method can be used.
 
  • Like
Likes Mr Davis 97
  • #3
andrewkirk said:
It's not that we can't calculate the projection without an orthogonal basis for W, it's that the usual and simplest method of calculating that projection has to use an orthogonal basis. That method is as follows. Let ##B=\{\vec v_1,...,\vec v_m\}## be an orthogonal basis for subspace ##W## of vector space ##V## and let ##u\in V##.

Then the projection ##\pi_W(\vec u)## of ##\vec u## onto ##W## is:
$$\sum_{k=1}^m \vec v_k\frac{\vec u\cdot \vec v_k}{\|\vec v_k\|^2}$$

That formula doesn't work if ##B## is not orthogonal because it will double count some components of the projection vector, because the terms of that sum may not be mutually orthogonal. To convince yourself of that, try working out the projection of ##(1, 1, 1)## onto the vector space spanned by orthogonal basis ##(0,1,0),\ (0, 0,1)## and compare that with what you get when you use the above formula with non-orthogonal basis ##(0,1,0),\ (0, 1,1)##.

In most cases (possibly always?) it is possible to derive an orthogonal basis from a non-orthogonal one, so that the above method can be used.

So how would you calculate the projection onto a subspace with a non-orthogonal basis? Is it usually the case that it is easier to find an orthogonal basis from a non-orthogonal basis and then use the formula you gave than it is to try to calculate the projection just from the original non-orthogonal basis?
 
  • #4
Mr Davis 97 said:
Is it usually the case that it is easier to find an orthogonal basis from a non-orthogonal basis and then use the formula you gave than it is to try to calculate the projection just from the original non-orthogonal basis?
Yes, I'd just do that. I started thinking about a method in which I used the formula with the non-orthog basis and then proceeded to deduct the overlapping components. But then I realized that what I was doing was basically just the same as what one does when one converts a non-orthogonal basis to an orthogonal one.

To convert non-orthog basis ##\vec v_1,...,\vec v_m## to an orthogonal basis ##\vec u_1,...,\vec u_m##, just set
$$\vec u_1=\vec v_1$$
$$\forall k\in\{2,...,m\}:\ \vec u_k=\vec v_k-\sum_{j=1}^{k-1} \vec u_j\frac{\vec v_k\cdot \vec u_j}{\|\vec u_j\|^2}$$
 
  • Like
Likes Mr Davis 97
  • #5
If we use the standard inner product in ##\mathbb R^n##, for which the standard basis is orthonormal, we can use the least square method to find the orthogonal projection onto a subspace of ##\mathbb R^n##: Form the matrix ##A## whose column vectors are the given, possibly non-orthonormal, basis of the subspace (it does not even need to be a basis, the vectors just need to span the subspace). Let ##\mathbf b## be the vector you want to project onto the subspace.
Then, find the least square solution(s) ##\mathbf x## of the system ##A\mathbf x=\mathbf b##, i.e. the solution(s) of the system ##A^TA\mathbf x=A^T\mathbf b##. Then, ##A\mathbf x## is the orthogonal projection of ##b## onto the subspace generated by the column vectors of ##A##, and ##\mathbf x## is the coordinate vector of the projection w.r.t. the given basis of the subspace. (If the columns of ##A## are not linearly independent, then ##x## is not unique.)

All this works only if we have an orthonormal basis to begin with, and express the basis vectors of the subspace w.r.t this basis (in our case, the standard basis). If we use ##\mathbb P_n##, the space of real polynomials of desgee ##\le n##, use the standard basis in this space, and define an inner product as an integral, then the standard basis is not orthonormal w.r.t. this inner product, and the above metod fails in this case.
 
  • Like
Likes andrewkirk
  • #6
The method Erland described give raise to the following simple formula for the orthogonal projection: put vectors that form a basis in ##W## as columns in a matrix, call it ##A##. Then the matrix of the orthogonal projection ##P_W## is given by $$P_W= A(A^TA)^{-1}A^T. $$ To find the projection of a vector ##\mathbb x## onto ##W## you just multiply it by this matrix.
 

1. What is an orthogonal basis?

An orthogonal basis is a set of vectors that are mutually perpendicular (orthogonal) and have a unit length. This means that the dot product of any two vectors in the set is 0, and each vector has a magnitude of 1.

2. How is an orthogonal basis used to find the projection onto a subspace?

An orthogonal basis can be used to find the projection onto a subspace by representing the subspace as a linear combination of the basis vectors. The projection of a vector onto a subspace is then the sum of the orthogonal projections of that vector onto each basis vector.

3. What is the significance of using an orthogonal basis for finding projections onto a subspace?

Using an orthogonal basis has the advantage of simplifying the calculation of projections onto a subspace. Since the basis vectors are orthogonal, the projection of a vector onto each basis vector can be calculated separately and then summed together.

4. How is an orthogonal basis determined for a given subspace?

An orthogonal basis for a subspace can be determined using the Gram-Schmidt process. This involves starting with a set of linearly independent vectors that span the subspace and then using a series of orthogonalization and normalization steps to create an orthogonal basis.

5. Can an orthogonal basis be used to find projections onto any subspace?

Yes, an orthogonal basis can be used to find projections onto any subspace. However, the basis vectors must be chosen carefully to ensure that they are orthogonal and that they span the subspace.

Similar threads

Replies
2
Views
937
Replies
2
Views
740
  • Linear and Abstract Algebra
Replies
9
Views
380
  • Linear and Abstract Algebra
Replies
6
Views
758
  • Linear and Abstract Algebra
Replies
14
Views
1K
Replies
3
Views
2K
  • Linear and Abstract Algebra
Replies
2
Views
1K
  • Linear and Abstract Algebra
Replies
8
Views
2K
  • Linear and Abstract Algebra
2
Replies
52
Views
2K
  • Linear and Abstract Algebra
Replies
9
Views
1K
Back
Top