Linear Algebra orthogonal basis and orthogonal projection

[infinitylord]

I was placed into honors calculus III for school. I was happy about this and I consider myself to be a pretty quick learner in math. However, my teacher is using many notations and terms that I am completely unfamiliar with. Mostly, I believe, because I've never taken linear algebra. I am familiar with the idea of vectors and many parts of linear algebra from my physics courses, but I'm still not at the level I think I need. I was assigned this as a "challenge problem."

Let b=(3,1,1) and P be the plane through the origin given by x+y+2z=0.

(a) Find an orthogonal basis for P. That is, find two nonzero orthogonal vectors V$_{1}$, V$_{2}$ $\in$ P.

(b)find the orthogonal projection of b onto P. That is, find Proj$_{V_{1}}$b+Proj$_{V_{2}}$b.

Homework Equations

$\vec{P}$=($\vec{v}$$\bullet$$\vec{a}$/||$\vec{a}||^{2}$)$\vec{a}$

The Attempt at a Solution

I have no idea where to even begin. I tried looking up what an orthogonal basis is, but everywhere I looked they used topological symbols I wasn't the slightest bit familiar with. It's also not in my book. I know how to solve for an orthognonal projection for the most part by using the equation above, but I really have no idea what an orthogonal prjection actually is or what its significance is. I really don't want anyone to just give me the answer. I really need to understand this all. Thank you!

 

[valerioperi]

Are you familiar with the concept of basis? Then an orthogonal basis is just a basis of vectors which are mutual perpendicular. That means in your two dimensional space that v1• v2 = 0, where the • rappresent, in your case, the usual scalar product between vectors in R^n.
Once you have figured out a basis you can solve the projection problem.

 

[infinitylord]

actually, I am not familiar with the concept of a basis or a projection. But I do understand the idea of orthogonality and the fact that the dot product of two orthogonal vectors will be equal to zero.

 

[Fredrik]

I think you need to take a little break from the problem and study the concepts "inner product", "norm", "orthogonal", "linearly independent", "basis", "subspace" and "span" (as in "the subspace spanned by the set S"). Ask a question if you get stuck on something.

Are you somewhat familiar with sets, so that you understand notation like this? $x\in A\subseteq B$. This would make it easier to explain some things.

 

[HallsofIvy]

"Let P be the plane through the origin given by x+y+2z=0[/quote]
The first thing to do is to recognize that any point in P can be written as (x, y, z) with x+ y+ 2z= 0, That equation is the same as x= -y- 2z so that any point in P can be written as (-y- 2z, y, z)= y(-1, 1, 0)+ z(-2, 0, 1). That tells you that {(-1, 1, 0), (-2, 0, 1)} is a basis for P.

Now, how is the "dot product" defined in R³?

 

[infinitylord]

Thank you for your help.
-Inner product I am very familiar with.
-Norm is just the length or magnitude of a vector.
-Orthogonal is the same thing as perpendicular. And the inner product of two orthogonal vectors will be equal to zero.
I just looked up the rest and I believe I understand it for the most part. (tell me if I've got it)
-span is basically all of the possible linear combinations of a set of vectors.
-a subspace is where any vectors that are contained within the subspace can be added together or multiplied by some scalar and still remain in the subspace. It also must contain the zero vector.
-I know that a set of vectors has to be either linearly dependent or independent. And it's where zero is the only solution to all of the vectors? So does that mean that all of the vectors in the set are in different spans?
-And a basis is basically when a set of vectors span the vector space (which is every possible point on the graph?) and are also linearly independent.
Really my only confusion still is with the idea of linear independence.
And I understand the ∈ symbol means that x is an element of A, but I'm not sure what the other one means.

 

[Fredrik]

I just looked up the rest and I believe I understand it for the most part. (tell me if I've got it)
-span is basically all of the possible linear combinations of a set of vectors.

It's also the smallest subspace that has that set of vectors as a subset.

-a subspace is where any vectors that are contained within the subspace can be added together or multiplied by some scalar and still remain in the subspace. It also must contain the zero vector.

I like to define it as a subset that's also a vector space. A subset is a vector space if and only if the conditions you mentioned are satisfied.

-I know that a set of vectors has to be either linearly dependent or independent. And it's where zero is the only solution to all of the vectors? So does that mean that all of the vectors in the set are in different spans?

One way of describing a linearly independent set S is to say that no element of S is a linear combination of the other elements of S.

-And a basis is basically when a set of vectors span the vector space (which is every possible point on the graph?) and are also linearly independent.

Yes, this is a good way to define it. There are many equivalent ways. Two examples: 1. It's a maximal linearly independent set (i.e. a linearly independent set that isn't a proper subset of any linearly independent set). 2. It's a minimal spanning set for the vector space (i.e. a set that spans the vector space, and doesn't have any proper subsets that span the vector space).

Really my only confusion still is with the idea of linear independence.

That's the most difficult of these concepts. I like this statement of the definition: A finite set $\{x_1,\dots,x_n\}$ is said to be linearly independent if the following implication holds for all $a_1,\dots,a_n\in\mathbb R$.

If $\sum_{i=1}^n a_i x_i =0$ then $a_i=0$ for all $i\in\{1,\dots,n\}$.​

An infinite set is said to be linearly independent if all of its finite subsets are.

And I understand the ∈ symbol means that x is an element of A, but I'm not sure what the other one means.

$A\subseteq B$ means "A is a subset of B", i.e. that every element of A is an element of B. Many books use the symbol $\subset$ instead of $\subseteq$.

Well done. It looks like you understand these things pretty well. I will try to answer your original questions in the next post.

 

[Fredrik]

Let V be a vector space. A set $S\subseteq V$ is said to be orthonormal if

a) For all $x,y\in S$, x is orthogonal to y.
b) For all $x\in S$, $\|x\|=1$.

An orthonormal basis is a basis that's also an orthonormal set.

For example, the set $\{(1,0),(1,1)\}$ is a basis for $\mathbb R^2$, but not an orthonormal basis. $\{(1,0),(0,1)\}$ is an orthonormal basis.

Now, regarding orthogonal projections...

Let V be a finite-dimensional inner product space. Let U be a subspace of V. There's a theorem that says that for each $x\in V$, there's a unique pair $(y,z)$ such that $y\in U$, $z$ is orthogonal to every element of U, and $x=y+z$. The vector y is called the orthogonal projection of x onto U.

I recently explained orthogonal projections onto a 1-dimensional subspace in another thread. This explanation may be useful to you too, even though you're interested in a 2-dimensional subspace. So I'll quote myself.

You need to know how to find the projection of an arbitrary vector $\vec x$ onto a 1-dimensional subspace U. Let $\vec y$ and $\vec z$ be the unique vectors such that $\vec y$ is in U, $\vec z$ is orthogonal (perpendicular) to every vector in U, and $\vec x=\vec y+\vec z$.

We're looking for a formula for $\vec y$. If $\vec u$ is a unit vector in U, there's a unique real number r such that $\vec y=r\vec u$. So we can write $\vec x=r\vec u+\vec z$. Now what do you get if you use this formula to compute $\vec u\cdot\vec x$? The result is simply r. (You should verify that). So we have $\vec y=r\vec u=(\vec u\cdot\vec x)\vec u$.

The vector $\vec y$ is by definition the orthogonal projection of $\vec x$ onto U.

Right, two vectors are perpendicular if and only if their dot product is zero. So we get
$$\vec u\cdot\vec x=\vec u\cdot (r\vec u+\vec z)=r(\vec u\cdot\vec u)+\vec u\cdot\vec z=r|\vec u|^2+0=r.$$ And since $\vec y=r\vec u$, this means that $\vec y=(\vec u\cdot\vec x)\vec u$.

 

[infinitylord]

Okay, the mathematical representation you put of linear independence makes sense to me. And orthonormal is basically orthogonal unit vectors then? Since they must be perpendicular and have a norm of 1.
The question is beginning to make more sense now, although admittedly I am still not completely sure where to begin. I basically have to find two vectors (besides the zero vector) that are contained within the subspace P in R^2 and that are perpendicular to each other. Does this men there is more than one correct answer? And while researching the equation of a plane, I found that I need to find the normal vector. Which would be <1,1,2> in this case, correct?

 

[Fredrik]

Okay, the mathematical representation you put of linear independence makes sense to me. And orthonormal is basically orthogonal unit vectors then? Since they must be perpendicular and have a norm of 1.
The question is beginning to make more sense now, although admittedly I am still not completely sure where to begin. I basically have to find two vectors (besides the zero vector) that are contained within the subspace P in R^2 and that are perpendicular to each other. Does this men there is more than one correct answer? And while researching the equation of a plane, I found that I need to find the normal vector. Which would be <1,1,2> in this case, correct?

Right, there are infinitely many orthonormal bases, and yes, that is a normal vector of that plane.

As for part b), recall that the decomposition x=y+z of a vector into a sum of one vector that's in the subspace, and one that's orthogonal to it, is unique. So if you can think of any way to write b as a sum of an element of P and one that's orthogonal to P, you will have found your orthogonal projection.

 

[infinitylord]

Thank you very much! I believe I understand it all now.
a) For $\vec{V_{1}}$, $\vec{V_{2}}\in$P. $\vec{V_{1}}$ can be arbitrary so long as it satisfies the condition a+b+2c=0. So I chose $\vec{V_{1}}$=(-1,1,0)
$\vec{V_{2}}$ must be orthogonal to $\vec{V_{1}}$, so I chose it to be (1,1,0). And that is the answer I stuck with.
b) I used the formula I wrote in the original question. Once for $\vec{V_{1}}$ and once for $\vec{V_{2}}$. I got Proj$_{\vec{V_{1}}}$b=(1,-1,0) and Proj$_{\vec{V_{2}}}$b=(2,2,0).

 

[valerioperi]

Almost there :thumbs: Unfortunately V2 doesn't belong to your subspace but you are definitely on the right way. Once you'll find the projection of b on v1 and v2, then the projection on the plane is the sum of the two, am I right @Fredrik?

 

[infinitylord]

Oh! you're right, I hadn't even realized it didn't belong to the subspace. I just re-did it and got $\vec{V_{2}}$=(1,1,-1).
it satisfies a+b+2c=0, and <$\vec{V_{1}}$,$\vec{V_{2}}$>=-1+1+0=0. Therefore, they are orthogonal. This would then make the projection for b onto $\vec{V_{2}}$=(1,1,-1).
And the sum of the projections i.e. the projection of b onto P should then be (2,0,-1). Right?

 

[Fredrik]

Almost there :thumbs: Unfortunately V2 doesn't belong to your subspace but you are definitely on the right way. Once you'll find the projection of b on v1 and v2, then the projection on the plane is the sum of the two, am I right @Fredrik?

Yes, that's right.

infinitylord, can you find an orthonormal basis for $\mathbb R^3$, with two of the vectors in P and the third orthogonal to P, and then write b as a linear combination of these three basis vectors?