Understanding the Geometric Interpretation of the Gram-Schmidt Process

  • Thread starter EngWiPy
  • Start date
  • Tags
    Process
In summary, The Gram-Schmidt Process is a mathematical procedure used to orthonormalize a set of vectors in a vector space. It is important because it allows for the creation of an orthonormal basis from a set of linearly independent vectors. The process works by taking a set of vectors and systematically transforming them into a new set of orthonormal vectors. However, it can be computationally expensive and may not accurately represent the original vector space if the original set of vectors is not linearly independent. Real-world applications of the Gram-Schmidt Process include computer graphics, signal processing, and quantum mechanics.
  • #1
EngWiPy
1,368
61
Hi,

What is the geometric interpretation of the Gram-Schmidt orthonormalization process? I mean, you will find everywhere that the procedure is as follows:

1. set u1=a1 => e1=u1/||u1||
2. u2=a2-<a2,e1>e1 => e2=u2/||u2||
.
.
.

As a comment on the second line, you will read that this is done to ensure that u2 is orthogonal on e1. But how is that? What is the geometric interpretation of this?

Thanks in advance
 
Physics news on Phys.org
  • #2
S_David said:
Hi,

What is the geometric interpretation of the Gram-Schmidt orthonormalization process? I mean, you will find everywhere that the procedure is as follows:

1. set u1=a1 => e1=u1/||u1||
2. u2=a2-<a2,e1>e1 => e2=u2/||u2||
.
.
.

As a comment on the second line, you will read that this is done to ensure that u2 is orthogonal on e1. But how is that? What is the geometric interpretation of this?

Thanks in advance

<a2,e1>e1 is the projection of a2 onto e1, then a2-<a2,e1>e1 is the vector orthogonal to e1
 
  • #3
yifli said:
<a2,e1>e1 is the projection of a2 onto e1, then a2-<a2,e1>e1 is the vector orthogonal to e1

How is that? Can you elaborate more? Obviously I have problems in the basics.!
 
  • #4
Here's the simplest way to see it:

1. We know that two vectors a and b are orthogonal if and only if <a,b> = 0.

2. We can see that <e1, a2 - <a2,e1>e1> = <e1,a2> - <e1,<a2,e1>e1> = <e1,a2> - <a2,e1><e1,e1> = <e1,a2> - <a2,e1> = <e1,a2> - <e1,a2> = 0

3. Therefore, e1 and a2 - <a2,e1>e1 are orthogonal.
 
  • #5
If you're looking for a geometric interpretation of Gram-Schmidt, the finest I've seen comes originally from Hestenes and Sobczyk, "Clifford Algebra to Geometric Calculus". It involves the wedge product. Here's a quick exposition of the main ideas: both in algebraic form, and in plain English.

I'll call the original (non-orthogonal) frame [itex]a_i[/itex], and the orthogonal frame [itex]c_i[/itex]. Assume there are [itex]n[/itex] vectors.

First, we need a way to represent "the subspace spanned by the first [itex]k[/itex] vectors". This is given by
[tex]A_k = a_1 \wedge a_2 \wedge \ldots \wedge a_k[/tex]
If you don't know what the wedge product means, simply think of [itex]A_k[/itex] as "the subspace spanned by the first [itex]k[/itex] vectors.

Now, the subspace spanned by all the vectors is of course [itex]A_n[/itex]. We will write this down, then multiply by unity in a clever way.
[tex]
\begin{align}
a_1 \wedge a_2 \wedge \ldots \wedge a_n &= A_n \\
&= (A_1A_1^{-1})(A_2A_2^{-1}) \ldots (A_{n-1}A_{n-1}^{-1})A_n \\
&= A_1(A_1^{-1}A_2)(A_2^{-1}\ldots)\ldots(\ldots A_{n-1})(A_{n-1}^{-1}A_n) \\
&= c_1(c_2)(c_3)\ldots(c_{n-1})(c_n) \\
\end{align}
[/tex]
Here, the expression for the [itex]k[/itex]th vector is
[tex]
c_k = A_{k-1}^{-1}A_k
[/tex]
What we have done is to write a pure [itex]n[/itex]-vector (i.e. [itex]A_n[/itex]) as the geometric product of [itex]n[/itex] different vectors (i.e. the [itex]c_k[/itex]). These vectors must therefore all be mutually orthogonal.

What did we do, in plain English? Well, to find the k'th Gram-Schmidt vector,
1) Take the subspace spanned by the first [itex]k[/itex] vectors
2) Remove the subspace spanned by the first [itex](k-1)[/itex] vectors
In other words: we keep only what the [itex]k[/itex]th vector "adds", only what it gets us that we couldn't get before.

Note that the usual Gram-Schmidt is an iterative procedure: you actually need to calculate [itex]c_1, c_2, \ldots, c_{k-1}[/itex] before you can get [itex]c_k[/itex]. Not so with this exposition: we can directly write down an expression for any [itex]c_k[/itex] involving only the original [itex]a_i[/itex]. Moreover, the expression is a simple one with clear geometric meaning. A very elegant take on this well-known algorithm -- I was delighted when I first read it.
 
  • #6
You're basically shaving off from a vector until it's orthogonal to previous vectors. Orthogonal vectors have an inner product of 0, so that if your inner product is not 0 it has some projection on original vectors. If you subtract off that part so that the inner product is indeed 0, then you'll leave a vector that's orthogonal.

Think of having two planks of wood that are obliquely aligned. If you shine a light from above you'll see a shadow, so you shave down the wood by the amount of the shadow casted on it. This leaves no shadow meaning the two planks are normal.
 

What is the Gram-Schmidt Process?

The Gram-Schmidt Process is a mathematical procedure used to orthonormalize a set of vectors in a vector space. It is commonly used in linear algebra and has applications in fields such as physics and engineering.

Why is the Gram-Schmidt Process important?

The Gram-Schmidt Process is important because it allows for the creation of an orthonormal basis from a set of linearly independent vectors. This is useful in many mathematical and scientific applications, such as solving systems of linear equations and performing projections in vector spaces.

How does the Gram-Schmidt Process work?

The Gram-Schmidt Process works by taking a set of vectors and systematically transforming them into a new set of orthonormal vectors. This is achieved through a series of orthogonal projections and normalization steps.

What are the limitations of the Gram-Schmidt Process?

One limitation of the Gram-Schmidt Process is that it can be computationally expensive for large sets of vectors. Additionally, if the original set of vectors is not linearly independent, the resulting orthonormal basis may not accurately represent the original vector space.

What are some real-world applications of the Gram-Schmidt Process?

The Gram-Schmidt Process has many real-world applications, including computer graphics and image processing, signal processing, and quantum mechanics. It is also used in data compression and in solving optimization problems in fields such as finance and economics.

Similar threads

  • Linear and Abstract Algebra
Replies
14
Views
1K
  • Linear and Abstract Algebra
Replies
4
Views
1K
Replies
2
Views
1K
Replies
3
Views
2K
Replies
2
Views
2K
  • Calculus and Beyond Homework Help
Replies
16
Views
1K
  • Calculus and Beyond Homework Help
Replies
7
Views
12K
  • Calculus and Beyond Homework Help
Replies
10
Views
3K
  • Advanced Physics Homework Help
Replies
3
Views
2K
  • Calculus and Beyond Homework Help
Replies
6
Views
2K
Back
Top