How Do Projections and the Cauchy-Schwarz Inequality Connect in Linear Algebra?

[braindead101]

(a)Let u be a nonzero vector in R$$^{n}$$. For all v$$\epsilon$$R$$^{n}$$, show that proj$$_{u}$$(proj$$_{u}$$(v)) = proj$$_{u}$$(v) and proj$$_{u}$$(v - proj$$_{u}$$(v)) = $$\vec{0}$$

(b) An alternate proof of the Cauchy-Schwarz inequality. For v,w $$\epsilon$$R$$^{n}$$, consider the function q: R -> R defined by q(t) = ($$\vec{v}+t\vec{w}) \bullet (\vec{v}+t\vec{w}).$$ Explain why q(t) >= 0 for all t
$$\epsilon$$R. By interpreting q(t) as a quadratic polynomial in t, show that |$$\vec{v} \bullet \vec{w} <= ||\vec{v}|| ||\vec{w}||.$$
HINT: For a,b,c $$\epsilon$$R, we have at^2 +bt + c >= 0 for all t $$\epsilon$$R if and only if a > 0 and b^2 - 4ac <= 0.


um i don't know how to fix the formula but i think you understand what I mean.


I really need a starting point, i have no idea how to do it, especially part (b), for part a i have tried proving lhs = rhs by starting with lhs, but nothing is cancelling. I just used the projection formula and projected v onto u and then nothing cancels so i project the (projected v onto u) piece onto the u again, and i am having a hard time simplifying, I'm not sure if this is even the right way to solve it.

any information would be great

 

[Mindscrape]

Wrong forum, but that's okay. First off, think about what part a is saying. If you project a vector onto a plane, what will happen to it if you project it onto the plane again? Nothing, it simply projects to its own projection. There are a lot of ways to do projections, one of which shows the answer pretty quick, which one are you using?

Part b is mostly just going through the algebraic steps. Really just start doing the inner product, and you should start to see what it is talking about... (Hint: use the fact that <v,v> = ||v||^2.)

 

[braindead101]

i have to prove (a) algebraically, i can prove it with diagram and words, but not algebraically, i am having troubel doing that

 

[braindead101]

okay so for (b) i have expanded and collected the terms.

q(t) = v$$\bullet$$v + v$$\bullet$$tw + tw$$\bullet$$v + tw$$\bullet$$tw
q(t) = |v|^2 + 2(v$$\bullet$$tw) + |tw|^2

so do i use the hint and say that in order for q(t) >= 0 it must satisfy a>0 and b^2-4ac <=0 which it does.

but what about to show that |v$$\bullet$$w| <= ||v|| ||w||, how do i go about doing that?

 

[Hurkyl]

(for (a)) Well, can you at least show us how far you've gotten? That would make it easier to point out what you're missing (or have done wrong).

 

[braindead101]

and i wasn't taught a specific way to do projections, just the formula itself.

 

[Mindscrape]

Sorry, I suck at matrix LaTeX, so I'm not going to use it, hope everything is still legible. So the projection matrix that I like to use, and think makes sense is

P = aa^T/(a^T a) = aa^T/||a||^2

What is PP = P^2? By definition of projections it should it P.

 

[braindead101]

I'm going to use . as dot product

To Prove: proj u (proj u (v)) = proj u (v)

proj u ((u.v / u.u) u)
= proj u ((u.v / |u|^2) u)
= [u . (u.v/ |u|^2)u ]u / u.u
= [u . (u.v/ |u|^2)u ]u / |u|^2

and I'm stuck here

 

[braindead101]

P = aa^T/(a^T a) = aa^T/||a||^2

i don't understand this at all, can you please explain this

 

[Mindscrape]

Alright, it is really the same thing that you have. Let's call your formula x*.

x* = (a^T b/a^T a)a_vec = a(a.b/a.a) = (a_vec a^T/a^Ta)b

The part in the parentheses P = (aa^T/a^Ta) is the projection matrix for projecting b to a. The projection vector would be
p = P b_vec = (aa^T/a^Ta) b

It all comes from orthogonality. If you want to prove it, or show it, for yourself then draw a vector b on paper, and then another vector a. The projection of be to a will be a portion of a p = x*a, and the orthogonal leg will be (b - x*a). You know the vectors b and a, so solve for x*.

Make sense? Also, look at the projection matrix to make sure the "dimensions" make sense. aa^T is a matrix, a^Ta is a scalar, and b is a vector.

 

[braindead101]

okay so i understand the fact that we have the same thing, but how do i proceed from there?
drawing it on paper, i know that it is 0, but i cannot prove this algebraically still, i am still stuck in the same place

 

[Mindscrape]

So, were you able to prove that P^2 = P?

Also, for part b you are right so far, and it can be summed up as
0<=||v+tw||^2=<v+tw,v+tw>=||v||^2 + 2t<v,w> + t^2||w||^2

Now
||v||^2 + 2t<v,w> + t^2||w||^2

looks an awful lot like a parabola as a function of t.

what if you say
polynomial(t) = p(t) = ||v||^2 + 2t<v,w> + t^2||w||^2?

Can you figure out t, you were given a pretty good hint?

 

[braindead101]

i'm not sure how to go from P to PP, how do i explain what I'm doing
i really don't understand the projection matrix thing
for (a) , i don't know how to solve it still, i just know that I am using the proj formula, and doing it twice, but why is that the same as proj^2 like you said, i don't really understand that.

 

[braindead101]

where did the c come from, and why did you get rid of the t^2 in front of the w but not in front of the v,w

 

[Mindscrape]

Whoops, sorry, the c should be a t, or the t should be a c. I'm used to constants as c and made a Freudian slip or something.

So there are two parts to a. I will show you the first part for free. Do you want projection u to v, or v to u, sorry I don't remember how the notation goes? I'm doing u to v, so you can switch if you need. The result is the same.

Proj_u(Proj_u v) = ...

Let's do the part in the parentheses first

$$\vec{p} = (Proj_u v) = \frac{vv^T}{v^T v} u$$

This part is just some vector p_vec, and we want to make a second projection onto that vector, so it is really:

$$Proj_u( \vec{p}) = \frac{vv^T}{v^T v} \vec{p}$$

Put the two together

$$Proj_u(Proj_u v) = \frac{vv^T}{v^T v} \frac{vv^T}{v^T v} u$$

Note that v^T v is a scalar, and can be moved or canceled how you want.

$$Proj_u(Proj_u v) = \frac{vv^Tvv^T}{v^T v v^T v} u = \frac{vv^T}{v^T v} u = (Proj_u v)$$