Understanding the Geometric Interpretation of the Gram-Schmidt Process

[EngWiPy]

Hi,

What is the geometric interpretation of the Gram-Schmidt orthonormalization process? I mean, you will find everywhere that the procedure is as follows:

1. set u1=a1 => e1=u1/||u1||
2. u2=a2-<a2,e1>e1 => e2=u2/||u2||
.
.
.

As a comment on the second line, you will read that this is done to ensure that u2 is orthogonal on e1. But how is that? What is the geometric interpretation of this?

Thanks in advance

 

[yifli]

Hi,

What is the geometric interpretation of the Gram-Schmidt orthonormalization process? I mean, you will find everywhere that the procedure is as follows:

1. set u1=a1 => e1=u1/||u1||
2. u2=a2-<a2,e1>e1 => e2=u2/||u2||
.
.
.

As a comment on the second line, you will read that this is done to ensure that u2 is orthogonal on e1. But how is that? What is the geometric interpretation of this?

Thanks in advance

<a2,e1>e1 is the projection of a2 onto e1, then a2-<a2,e1>e1 is the vector orthogonal to e1

 

[EngWiPy]

<a2,e1>e1 is the projection of a2 onto e1, then a2-<a2,e1>e1 is the vector orthogonal to e1

How is that? Can you elaborate more? Obviously I have problems in the basics.!

 

[klackity]

Here's the simplest way to see it:

1. We know that two vectors a and b are orthogonal if and only if <a,b> = 0.

2. We can see that <e1, a2 - <a2,e1>e1> = <e1,a2> - <e1,<a2,e1>e1> = <e1,a2> - <a2,e1><e1,e1> = <e1,a2> - <a2,e1> = <e1,a2> - <e1,a2> = 0

3. Therefore, e1 and a2 - <a2,e1>e1 are orthogonal.

 

[chogg]

If you're looking for a geometric interpretation of Gram-Schmidt, the finest I've seen comes originally from Hestenes and Sobczyk, "Clifford Algebra to Geometric Calculus". It involves the wedge product. Here's a quick exposition of the main ideas: both in algebraic form, and in plain English.

I'll call the original (non-orthogonal) frame $a_i$, and the orthogonal frame $c_i$. Assume there are $n$ vectors.

First, we need a way to represent "the subspace spanned by the first $k$ vectors". This is given by
$$A_k = a_1 \wedge a_2 \wedge \ldots \wedge a_k$$
If you don't know what the wedge product means, simply think of $A_k$ as "the subspace spanned by the first $k$ vectors.

Now, the subspace spanned by all the vectors is of course $A_n$. We will write this down, then multiply by unity in a clever way.
$$\begin{align}<br /> a_1 \wedge a_2 \wedge \ldots \wedge a_n &= A_n \\<br /> &= (A_1A_1^{-1})(A_2A_2^{-1}) \ldots (A_{n-1}A_{n-1}^{-1})A_n \\<br /> &= A_1(A_1^{-1}A_2)(A_2^{-1}\ldots)\ldots(\ldots A_{n-1})(A_{n-1}^{-1}A_n) \\<br /> &= c_1(c_2)(c_3)\ldots(c_{n-1})(c_n) \\<br /> \end{align}$$
Here, the expression for the $k$th vector is
$$c_k = A_{k-1}^{-1}A_k$$
What we have done is to write a pure $n$-vector (i.e. $A_n$) as the geometric product of $n$ different vectors (i.e. the $c_k$). These vectors must therefore all be mutually orthogonal.

What did we do, in plain English? Well, to find the k'th Gram-Schmidt vector,
1) Take the subspace spanned by the first $k$ vectors
2) Remove the subspace spanned by the first $(k-1)$ vectors
In other words: we keep only what the $k$th vector "adds", only what it gets us that we couldn't get before.

Note that the usual Gram-Schmidt is an iterative procedure: you actually need to calculate $c_1, c_2, \ldots, c_{k-1}$ before you can get $c_k$. Not so with this exposition: we can directly write down an expression for any $c_k$ involving only the original $a_i$. Moreover, the expression is a simple one with clear geometric meaning. A very elegant take on this well-known algorithm -- I was delighted when I first read it.

 

[zcd]

You're basically shaving off from a vector until it's orthogonal to previous vectors. Orthogonal vectors have an inner product of 0, so that if your inner product is not 0 it has some projection on original vectors. If you subtract off that part so that the inner product is indeed 0, then you'll leave a vector that's orthogonal.

Think of having two planks of wood that are obliquely aligned. If you shine a light from above you'll see a shadow, so you shave down the wood by the amount of the shadow casted on it. This leaves no shadow meaning the two planks are normal.