Confused about the concepts of dual spaces, dual bases, reflexivity and annihilators

Philmac

My background in linear algebra is pretty basic: high school math and a first year course about matrix math. Now I'm reading a book about finite-dimensional vector spaces and there are a few concepts that are just absolutely bewildering to me: dual spaces, dual bases, reflexivity and annihilators. The book I'm reading explains everything in extremely general terms and doesn't provide any numerical examples, so I can't wrap my head around any of this. I'd really appreciate it if my loose understanding of these concepts could be critiqued/corrected and, if possible, some simple numerical examples could be provided. I really can't make heads or tails of some of this.

Note: I've never seen this bracket notation before, so I'll briefly introduce it in case it isn't something standard:
[x,y][itex]\equiv[/itex]y(x)

First, dual spaces. My understanding of a linear functional is that it's a black box where vectors go in and scalars come out (e.g. dot product). The dual space V' of a vector space V, is the set of all linear functionals that can be applied to that vector space. So, why is this called a "space"? How can things like integration and dot products (i.e. operations) form a space? The author also refers to the elements of V' as "vectors" -- how can an operation be a vector? My understanding of a vector is that it is a value with both magnitude and direction. Obviously, operations produce values, but V' is the set of operations, not values.

Second, dual bases. I just don't understand this at all, so I'll just provide the definition in this book:
I think ∂_ij is the Kronecker delta, but I'm not 100% sure.
So, what I think this means is that there is one operation in V' for each V, for which y_j(x_i)=1 for j=i and 0 for all j≠i. But does V' necessarily have dimension n?

Third, reflexivity. I just don't understand this at all. Here's the definition in the book:

Fourth, annihilators. I think I understand this concept somewhat, but the proofs presented don't make sense to me. My understanding of an annihilator is that it is any subset of V' which evaluates to 0 for all x in V. The thing I'm confused about is the annihilator of an annihilator.
Now, I'm willing to accept this proposition, but the proof is relatively short yet I cannot make sense of it. The proof is:
The problem I have here is "By definition, M⁰⁰ is the set of all vectors x such that [x,y]=0 for all y in M⁰". Shouldn't it be the set of all vectors z (or whatever letter you like) in V'?

micromass

Let's begin by clearing this up. Once you get this, we'll get to the next thing:

You're having a major, major misconception. You think of vector as some kind of "arrow" with both a magnitude and direction. While this is certainly true in basic math, this is absolutely false when you get to more advanced spaces.

In fact, an [itex]\mathbb{R}[/itex]-vector space is any set equipped with an addition and a scalar multiplication (which satisfy some elementary axioms). All a vector is, is an element of a vector space.

The easiest example of a vector space is of course [itex]\mathbb{R}^n[/itex], whose elements can indeed be seen as "arrows" with a magnitude and a direction.

However, this is far from the only example of a vector space. For example:

[tex]\{f:[0,1]\rightarrow \mathbb{R}~\vert~\text{f continuous}\}[/tex]

is also a vector space! All this means is that the sum and scalar multiplication of continuous functions gives us a continuous function. And it would be very awkward to see this set as a collection "arrows". The vectors of these set are now continuous functions!

Other vector spaces are the polynomials, the differentiable functions, etc... The thing I want you to realize is that a vector space is a very broad concept. There's a lot that can be a vector space, not just "arrows with maginute and direction".

When given a vector space V (which can be anything), we can form the dual space

[tex]V^\prime=\{f:V\rightarrow \mathbb{R}~\vert~f~\text{linear}\}[/tex]

This is a vector space. All this means is that the sum and scalar product of linear functions is linear. There's nothing more to it. The vectors here are simply linear functions!!

So remember: a vector space can be anything!!!! Once you understand this, we can move on to your next questions. But I feel that you must grasp this first.

Philmac

Thank you very much! There is a list of axioms defining both fields and vector spaces at the beginning of the book I'm reading. I do agree that a dual space satisfies these axioms. However, I errantly believed that "vectors" had to have some geometric interpretation. I thought that perhaps vectors were more abstract than I had previously believed, and you've confirmed that for me. So a vector (space) is simply anything that satisfies the axioms, nothing more than that. I believe I'm ready to hear explanations for the rest of these concepts :)

micromass

OK, dual bases then/

The definition is maybe a bit more abstract then was possible. But that's not necessary a bad thing.

Let's first look at the vector space [itex]V=\mathbb{R}^n[/itex], which is the nice vector space of arrows. Elements in the dual space are now just linear functions [itex]T:\mathbb{R}^n\rightarrow \mathbb{R}[/itex].

The first important observation is that T is completely determined on how it acts on a basis. That is, if you would know that

[tex]T(1,0,...,0)=y_1,T(0,1,0,...,0)=y_2,..., T(0,0,0,...,1)=y_n[/tex]

then you know completely what x does on every element:

[tex]T(a_1,...,a_n)=a_1y_1+...+a_ny_n[/tex]

So it just suffices to say what T does on a basis.

Now, if V is an arbitrary vector space, then the same holds: we can define a linear function by saying what it does on the basis. Now, let's define such a function. Take a basis [itex]\{e_1,...,e_n\}[/itex] and define

[tex]T_1(e_1)=1, T_k(e_k)=0~\text{for k>1}[/tex]

In general, we have

[tex]T_i(e_i)=1,~T_k(e_i)=0~\text{if}~k\neq i[/tex]

Even more abstractly put: [itex]T_i(e_k)=\delta_{ik}[/itex], where we indeed have the Kronecker delta.

What is our T in our nice space [itex]\mathbb{R}^n[/itex]? Well, you can easily see that

[tex]T_i(a_1,...,a_n)=a_i[/tex]

so T_i is simply the i'th projection!!

Now, when V is finite-dimensional, I claim it is the case that T_i is a basis for V' (the dual space). This means nothing more then:

• The T_i are linearly independent:
  [tex]\lambda_1 T_1+...+\lambda_n T_n=0~\Rightarrow~\lambda_1=...=\lambda_n=0[/tex]
  Indeed, if [itex]\lambda_1 T_1+...+\lambda_n T_n=0[/itex], then
  
  [tex]\lambda_1 T_1(x)+...+\lambda_n T_n(x)=0[/tex]
  
  for every vector x. So try the vectors e_i as our x's.
• The T_i span the space.
  This means only that every functional T can be written as
  
  [tex]\lambda_1 T_1+...+\lambda_n T_n=T[/tex]
  
  So, we must find [itex]\lambda_i[/itex] such that the above is true. But, it suffices to take [itex]\lambda_i=T(e_i)[/itex] here. Then you can easily check that for any x, it holds that
  
  [tex]\lambda_1T_1(x)+...+\lambda_nT_n(x)=T(x)[/tex]

I hope that clarifies this.

By the way, which book are you reading?

Philmac

Thanks again! The book I'm reading is "Finite-Dimensional Vector Spaces" by Paul R. Halmos.

I've read your post many times but I'm having a great deal of trouble understanding it. Math notation has always been extremely confusing for me (I really need concrete examples with numbers), so I apologize if my questions are extremely basic. Here's what I'm having trouble with:

What exactly is T? I understand that it is a map from Rⁿ to R¹, but are you saying that each element of V'=T_n or are you saying that V'=T? What are the arguments of T? x? The elements y₁...y_n are operations, so how can anything equate to them? Perhaps this goes back to my original problem with the notion of a dual space.


I sort of understand this... For example, in R³ a basis is (1,0,0),(0,1,0),(0,0,1), so I understand the Kronecker delta here, but a dual base is made up of operations -- what guarantees that each operation in V' will match up with the right (or any) element of the base of V such that it evaluates to 1 (and 0 for all the other elements of the base)?

If I were in the familiar territory of R² or R³ I would understand this completely, but what does it mean when you project an operation? Say, onto the "integration axis". What would this mean? Say T_k=integration, and a_i=3. Would this mean that the operation being performed consists (in part) of integrating the argument and multiplying the result by 3?

I understand your discussion of linear independence, but how does this work with a dual space? How is it even possible for the elements of a dual space to be linearly dependent? How does one combine, say, multiplication and the dot product to produce integration?

micromass

Hmmm, maybe not the best book for beginners...

Indeed, T is nothing more or nothing less then a linear map [itex]T:R^n\rightarrow R[/itex]. That's all it is.

No, T_n and T are certainly elements of V'. Why? Because T is a linear map from V to R, and V' is simply the set of such a linear maps.
I'm not claiming that V'=T or something. This would make little sense, since V' is a vector space and T is an operator.

The arguments of T are elements of Rⁿ. So, we can do T(2,3,2) if n=3, for example. In general notation, I write (x₁,...,x_n) for an n-tuple in Rⁿ.

Sorry about this. I didn't mean y₁,..., y_n as operations here. When I wrote them, I just meant them to be real numbers!!

For example, we can have T(1,0,...,0)=2, and then y₁=2. I do not mean y_i to be an operator here.

Could you explain this more? What do you mean with "mathcing up with the right element such that it evaluates to 1"??

I only meant this explanation to be in R² or R³. There are notions of "projections" in arbitrary vector spaces, but I don't think that now is a good time to discuss this.



Elements of a dual space V' are linear independent by definition if for all x in V it holds that

[tex]a_1T_1(x)+...+a_nT_n(x)=0~\Rightarrow~a_1=...=a_n=0[/tex]

For example, let me look at the dual space of [itex]\mathbb{R}^2[/itex]. Take two operators:

[tex]T:\mathbb{R}^2\rightarrow \mathbb{R}:(x,y)\rightarrow 2x+y[/tex]

and

[tex]S:\mathbb{R}^2\rightarrow \mathbb{R}:(x,y)\rightarrow x+2y[/tex]

these are elements of (R²)' because the maps are linear. I claim that they are linearly independent. This means that

If for all (x,y) it holds that aT(x,y)+bS(x,y)=0, then a=b=0.
So, let's assume that aT(x,y)+bS(x,y)=0 for all x and y. Translating this gives us

[tex]0=aT(x,y)+bS(x,y)=a(2x+y)+b(x+2y)=(2a+b)x+(a+2b)y[/tex]

This must hold for all x and y. So in particular for x=1 and y=0. So, if we fill that in, we get

[tex]2a+b=0[/tex]

But it must also hold for x=0 and y=1. So, filling that in, we get

[tex]a+2b=0[/tex]

So, if aT(x,y)+bS(x,y)=0 for all x and y, then certainly it must hold true that

[tex]2a+b=0~\text{and}~a+2b=0[/tex]

but this only holds for a=0 and b=0. So S and T are independent.

Philmac

My understanding of the Kronecker delta ∂_ij is that it evaluates to 1 when i=j and to 0 when i≠j. So, doesn't this mean that y_j(x_i) must evaluate to 1 for all i=j and 0 for all i≠j for the Kronecker delta to make sense here? What guarantee is there that this will happen? In R^≤3 this makes sense because any basis can be reduced to something like (1,0,0), etc. but with operations there is no reducing, there is simply the operation (e.g. integration). I think I'm just not understanding this part at all, now that I think about it some more.

Oh, I see. That makes perfect sense.

micromass

What guarantee?? You define things that way. You define y_i such that

[tex]y_i(e_i)=1~\text{and}~y_i(e_j)=0~\text{if}~i\neq j[/tex]

This always happens by definition!

Philmac

Is it always possible to do this?

micromass

Yes, given a vector space and given a basis [itex]\{e_1,...,e_n\}[/itex], I can define a linear function just by specifying what it will do on the basis.

For example, the following function is linear:

[itex]T(\lambda_1 e_1+...+\lambda_n e_n)=\lambda_i[/itex]

and it will be the function such that [itex]T(e_j)=\delta_{ij}[/tex].

Can I find a linear function that will send every [itex]T(e_i)[/itex] to 2? Yes!

[itex]T(\lambda_1e_1+...+\lambda_n e_n)=2(\lambda_1+...+\lambda_n)[/itex]

will be such a function. I can let the basis go to anything!!!! That's exactly what bases are good for.

Philmac

But how do you know that the appropriate operations will be available? And how do you know that V' is always n-dimensional? Take R¹ for example, I can think of at least two linear operations: multiplication and integration. So wouldn't V' have a dimension of at least 2 (even though n=1)?

micromass

Multiplication and integration are not linear operations on R. All the linear functionals on R have the form

[tex]f:R\rightarrow R:x\rightarrow \lambda x[/tex]

for a certain [itex]\lambda[/itex].

Philmac

Hm, alright. Thank you. I think I'm ready for reflexivity.

Fredrik

You're looking for an isomorphism from V into V''. This is very easy, because it turns out that the first function we can think of from V into V'' (except for constant functions of course) is an isomorphism. We want to define a function f:V→V'', so we must specify a member of V'' for each x. This member of V'' will of course be denoted by f(x). A member of V'' is defined by specifying what we get when it acts on an arbitrary member of V'. So we must specify f(x)(ω) for each ω in V'. f(x)(ω) is supposed to be a real number, and ω(x) is a real number. So...

For each x in V, we define f(x) in V'' by f(x)(ω)=ω(x) for all ω in V'. This defines a function f:V→V''.

Now you just need to verify that this function is linear and bijective onto V''.

By the way, I think V* is a more common notation than V'.

Philmac

Thank you! I still don't quite understand yet, but I think I'm a bit closer. If I'm not mistaken, it seems that we define the elements of V'' such that regardless of the value of y (or omega) in V', there is a bijective map (an isomorphism, in this context, I believe) between V and V'', in other words, each value of V'' corresponds to one, and only one, value in x. And since y (or omega) doesn't matter, there is only one value of [x₀, y] which corresponds to any element z₀ of V''. I hope that made sense

Also, what you said seems very similar to something in my textbook. I couldn't make sense of it, but after reading what you said I think it makes slightly more sense. Perhaps you could clear some things up about these concepts. Here it is:

I understand the part about [x, y₀], but I really don't follow the reasoning about [x₀, y]. [x, y] is a scalar, so how does keeping the value of x constant suddenly make (what is seemingly) the exact same thing an element of V''?



Also, backing up a bit to dual bases, I'd just like to verify that I understand. I'd appreciate it if someone could let me know if this example makes sense.

V=R³
X={(1,0,0),(0,1,0),(0,0,1)} is a basis in V
V'={ax₁+bx₂+cx₃|a,b,c[itex]\in[/itex]R}
X'={x₁,x₂,x₃}

Fredrik

Not entirely. The first thing that sounds weird to me is "regardless of the value of y (or omega)". The claim that V is isomorphic to V'' has nothing to do with any specific member of V,V' or V''. I guess that could be your point, but to say it this way is like saying that regardless of the value of q, we have 1+1=2. It's true, but it's weird to mention q when we could have just said that 1+1=2.

If you would like to use the [,] notation, we could say that for each x in V, there's exactly one z in V'' such that [y,z]=[x,y] for all y in V'. This z is denoted by f(x), and this defines the function f.

(If I understand the [,] notation correctly, [y,z]=[x,y] means z(y)=y(x)).

It will make more sense after you have verified that the f I defined is an isomorphism (i.e. that it's linear and bijective).

I don't like the [,] notation, and I'm not crazy about this author's way of explaining it either. You already understand that for each x in V and each y in V', [x,y]=y(x) is a real number. The author is saying that for each y in V', the function that takes x to [x,y] is a function from V into ℝ that we already had a notation for (this function is denoted by y). Then he's saying that for each x in V, the function that takes y to [x,y] is a function from V' to ℝ. Let's denote this function by g. We have [tex]g(ay+bz) = [x,ay+bz] = (ay+bz)(x) = (ay)(x)+(bz)(x) = a(y(x))+b(z(x)) = a[x,y]+b[x,z]=ag(y)+bg(z),[/tex] so g is linear. That means that it's a member of V''.

You're right that if X is a basis for V=ℝ³ and X' is its dual basis, then the members of V' can be uniquely expressed as linear combinations of members of X'. However, if X' is any basis for V', then the members of V' can still be uniquely expressed as linear combinations of members of X'.

What you mentioned is a property of any basis, not just the dual basis.

Philmac

I like your q, 1+1=2 analogy. That, in combination with what you said below, triggered a bit of an epiphany. V' is the set of functionals that take x as their argument and V'' is the set of functionals that take y as their argument. I find this notation to be slightly misleading (I'm probably just misunderstanding something) -- is there a V'''?

Exactly.

This makes much more sense now, thank you. However, now that I think about it again, how can there only be one z_i in V'' for each x_i in V? Changing the value of y_j will change the value of [x_i,y_j], so shouldn't V'' have dimension dimVxdimV'?

Great, I guess that means I have at least some grasp on the idea :)