What topics from linear algebra do I need to study tensors?

[AdrianZ]

Hi
What topics in linear algebra do I need to know to start learning tensors?
I know the following topics from linear algebra: 1-equation systems 2-vector spaces(linear independence, span, basis, important subspaces of a vector space) 3-linear transformations(kernel, Image, Isomorphic vector spaces) 4-Eigenvalues and Eigenvectors 5- Polynomials 6-determinants.
What other tools from linear algebra do I need to know to understand tensors? Where should I begin from to understand tensors little by little? and my final goal is understand the applications of tensors in Physics and then in pure mathematics.

 

[lavinia]

Hi
What topics in linear algebra do I need to know to start learning tensors?
I know the following topics from linear algebra: 1-equation systems 2-vector spaces(linear independence, span, basis, important subspaces of a vector space) 3-linear transformations(kernel, Image, Isomorphic vector spaces) 4-Eigenvalues and Eigenvectors 5- Polynomials 6-determinants.
What other tools from linear algebra do I need to know to understand tensors? Where should I begin from to understand tensors little by little? and my final goal is understand the applications of tensors in Physics and then in pure mathematics.

you know enough. feel free to ask questions

 

[chiro]

Hi
What topics in linear algebra do I need to know to start learning tensors?
I know the following topics from linear algebra: 1-equation systems 2-vector spaces(linear independence, span, basis, important subspaces of a vector space) 3-linear transformations(kernel, Image, Isomorphic vector spaces) 4-Eigenvalues and Eigenvectors 5- Polynomials 6-determinants.
What other tools from linear algebra do I need to know to understand tensors? Where should I begin from to understand tensors little by little? and my final goal is understand the applications of tensors in Physics and then in pure mathematics.

What kind of objects are you talking about? Are they standard finite-rank vector spaces, or something like a Hilbert space?

 

[Fredrik]

You really just need to understand linear functions, matrices, linear independence and bases. Make sure that you understand the relationship between linear functions and matrices described e.g. in post #3 in this thread.

 

[mathwonk]

the basic concept in linear algebra is linear functions. tensors are one step up, and concern multilinear functions. i.e. tensors are a topic in multilinear algebra.

a function is multilinear, if it is linear in each variable separately, so indeed linearity is the only prerequisite.the basic example of a multilinear function is multiplication. so tensors are a generalization of multiplication, except the multiplication may not be commutative.

 

[AdrianZ]

you know enough. feel free to ask questions

well, here are the questions that I'm struggling with for now: What is a dual space? (And why we need it?) What is a co-variant/contra-variant component of a tensor?

This is how I understand tensors: A tensor is a mapping from the Cartesian product of m vector spaces and n dual vector spaces to a field like real numbers which is linear in each of its components. such a tensor would be of rank m+n and would have m contravariant/covariant components and n covariant/contravariant components. This is one of the many definitions of tensors that I've seen.

What kind of objects are you talking about? Are they standard finite-rank vector spaces, or something like a Hilbert space?

I meant finite vector spaces over an arbitrary field.

You really just need to understand linear functions, matrices, linear independence and bases. Make sure that you understand the relationship between linear functions and matrices described e.g. in post #3 in this thread.

I fully understand linear independence, spans and bases. I'm also confident doing calculations with matrices.

the basic concept in linear algebra is linear functions. tensors are one step up, and concern multilinear functions. i.e. tensors are a topic in multilinear algebra.

a function is multilinear, if it is linear in each variable separately, so indeed linearity is the only prerequisite.


the basic example of a multilinear function is multiplication. so tensors are a generalization of multiplication, except the multiplication may not be commutative.

well, I guess a linear function is just another term for a linear transformation from one vector space to another. Is that right?

for now my real confusion arises from the meaning of the dual of a vector space. then I need to understand what co/contra variant vectors are and why we're interested in them.

 

[Fredrik]

I fully understand linear independence, spans and bases. I'm also confident doing calculations with matrices.

Excellent. You should make sure that you also understand the relationship between linear operators and matrices. The fastest way to do that if you don't know it already is to read the post I linked to in my previous post. You need this to understand how the concept of "components of a tensor" generalizes the concept of "components of a linear operator".

well, I guess a linear function is just another term for a linear transformation from one vector space to another. Is that right?

Yes, the terms linear function, linear map, linear operator and linear transformation, all mean the same thing in most books. Some authors only use the term operator when the domain and codomain are the same vector space. A linear functional is a linear map from a vector space over a field F into F.

What is a dual space? (And why we need it?) What is a co-variant/contra-variant component of a tensor?
...
for now my real confusion arises from the meaning of the dual of a vector space. then I need to understand what co/contra variant vectors are and why we're interested in them.

This post should be useful. You should also check out the three posts I link to at the end of it, and also this one.

 

[Fredrik]

Instead of posting that last link, I should have just quoted the relevant section.

Let V be a finite-dimensional vector space, and V* its dual space. A tensor of type (n,m) is a multilinear map $$T:\underbrace{V^*\times\cdots\times V^*}_{\text{n factors}}\times\underbrace{V\times\cdots\times V}_{\text{m factors}}\rightarrow\mathbb R.$$ The components of the tensor in a basis $\{e_i\}$ for V are the numbers $$T^{i_1\dots i_n}{}_{j_1\dots j_m}=T(e^{i_1},\dots,e^{i_n},e_{j_1},\dots,e_{j_m}),$$ where the $e^i$ are members of the dual basis of $\{e_i\}$. The multilinearity of T ensures that the components will change in a certain way when you change the basis. The rule that describes that change is called "the tensor transformation law".

I just wanted to include a definition of the components of a tensor.

 

[AdrianZ]

Excellent. You should make sure that you also understand the relationship between linear operators and matrices. The fastest way to do that if you don't know it already is to read the post I linked to in my previous post. You need this to understand how the concept of "components of a tensor" generalizes the concept of "components of a linear operator".

This is how I understand the relationship between linear transformations and matrices:
Suppose We have two vector spaces V and W and there is a mapping between them T: V → W that keeps linearity intact which means that for all v$\in$V, w$\in$W and the scalars α & β we have: T(αv+βw) = αT(v) + βT(w).
If we apply this linear transformation to a vector in V in a coordinate system equipped with the bases {e_i} we get:
T(α₁e₁+α₂e₂+...+α_ne_n) = α₁T(e₁) + α₂T(e₂) +...+ α_nT(e_n)
Now we can associate a matrix to this linear Transformation in the following way:
Let's show a vector v in V by an n*1 matrix (assuming it has n dimensions) and a vector w in W by an m*1 matrix(assuming it has m dimensions). then we can define the matrix A such that w=Av. the matrix A will be of the rank m*n. the columns of A will be the T(e_i)'s (the {e_i}'s are a basis for V, and the corresponding T(e_i)'s will form a basis for W). Hence we can write a vector w in W as a linear combination of the components of a vector v in V using the matrix multiplication Av. conversely, If you give me an equation like w=Av I can write it in the form T(α₁e₁+α₂e₂+...+α_ne_n) = α₁T(e₁) + α₂T(e₂) +...+ α_nT(e_n).

That's how I understand the relationship between linear transformations and matrices. correct me if I'm wrong.

Yes, the terms linear function, linear map, linear operator and linear transformation, all mean the same thing in most books. Some authors only use the term operator when the domain and codomain are the same vector space. A linear functional is a linear map from a vector space over a field F into F.

Thanks.

This post should be useful. You should also check out the three posts I link to at the end of it, and also this one.

I checked the thread, but the OP's question is not so understandable for me.

Instead of posting that last link, I should have just quoted the relevant section.
I just wanted to include a definition of the components of a tensor.

Doesn't that confirm the way I defined a tensor?
I'm still struggling with the concept of a dual space and dual bases. I don't understand what they are and why they are defined. (in fact I've seen very different definitions of a dual space which makes it only more confusing for me)

 

[Deveno]

the dual space V* for a vector space V sounds imposing, but it's really a pretty simple idea.

suppose we use the standard basis:

{e₁,e₂,...,e_n} for V, where

e_j = (0,0,0,...,1,...,0,0) <---1 in the j-th place.

the dual basis is the basis:

{φ¹,φ²,...,φⁿ} where

φⁱ(e_j) = δ_ij, the kroenecker delta.

you already know this basis, the dual basis element:

φⁱ(x₁,x₂,...,x_n) = x_i

is the just the i-th projection function.

(note: in tensor notation the coordinate indices of the vector x are usually written "up",

x = Ʃ x^je_j, so I'm breaking with tradition just for this post).

now, how do we normally calcuate a linear functional f: V→F?

we use linearity: f(v) = f(a₁e₁+a₂e₂+...+a_ne_n) = a₁f(e₁)+a₂f(e₂) +...a_nf(e_n).

now the f(e_j) are just field elements, which we can regard as the "coordinates" of f in V*, and each a_j can be regarding as φ^j(v).

so f(v) = f(e₁)φ¹(v)+f(e₂)φ²(v)+...f(e_n)φⁿ(v)

if we abbreviate f(e_i) by fⁱ, using the linearity of the projection functions we get:

f(v) = (f¹φ¹+f²φ²+...+fⁿφⁿ)(v)

(again, i note that it is traditional to write the "coordinates" of a linear functional "down", i am just writing it this way for clarity's sake in this one post)

in other words, we can write f = f¹φ¹+f²φ²+...+fⁿφⁿ, a linear functional in V* is just a linear combination of the projection functionals dual to our original basis for V.

that is, given a basis (a special subset of V, sort of the "instant coffee" description of V...just "add coordinates"), the projection functions that pick out the j-th coordinate in that basis, are the corresponding "special subset" of V* (the dual basis).

so, in the real plane, for example, we can describe any linear function from RxR→R, as a linear combination of "projection onto the x-axis" and "projection onto the y-axis".

you have seen these dual vectors before in another setting, although you may not have realized it. if we regard "x" as meaning the FUNCTION:

f(x,y) = x, then f' is the linear function with matrix [1 0], that is:

f'(a,b) = 1(a) + 0(b) = a

so df(a,b) (the differential of f at (a,b)) is just φ¹(a,b), dual to the tangent vector (1,0)_(a,b) (the unit vector in the x-direction with its start point at (a,b)). this is the actual meaning of the symbol "dx" in such expressions as:

$$\int P dy + Q dx$$

(hopefully this explains why "dx" isn't a number...it's a functional, you need to "hit it with a vector" to get a number out of it)

*********

tensors can be understood in terms of the tensor product. a good introduction to that is here:

http://www.dpmms.cam.ac.uk/~wtg10/tensors3.html

 

[AdrianZ]

the dual space V* for a vector space V sounds imposing, but it's really a pretty simple idea.

suppose we use the standard basis:

{e₁,e₂,...,e_n} for V, where

e_j = (0,0,0,...,1,...,0,0) <---1 in the j-th place.

the dual basis is the basis:

{φ¹,φ²,...,φⁿ} where

φⁱ(e_j) = δ_ij, the kroenecker delta.

well, I don't know what that notation means.φⁱ(e_j) = δ_ij. Does it mean that the inner product of φⁱ and e_i should be equal to kronecker delta or it means that if φ acts on e_i it'll produce a scalar? I guess the later is true. if that's true, then can we define the dual space of a vector space V as the space spanned by the basis
{φ¹,φ²,...,φⁿ} such that if any of them acts on a basis vector from V it produces a scalar number? (either 0 or 1).

now, how do we normally calcuate a linear functional f: V→F?

we use linearity: f(v) = f(a₁e₁+a₂e₂+...+a_ne_n) = a₁f(e₁)+a₂f(e₂) +...a_nf(e_n).

now the f(e_j) are just field elements, which we can regard as the "coordinates" of f in V*, and each a_j can be regarding as φ^j(v).

the f(e_j)'s are just field elements? I thought they were vectors? I start getting confused from this point.

 

[Fredrik]

This is how I understand the relationship between linear transformations and matrices:
Suppose We have two vector spaces V and W and there is a mapping between them T: V → W that keeps linearity intact which means that for all v$\in$V, w$\in$W and the scalars α & β we have: T(αv+βw) = αT(v) + βT(w).
If we apply this linear transformation to a vector in V in a coordinate system equipped with the bases {e_i} we get:
T(α₁e₁+α₂e₂+...+α_ne_n) = α₁T(e₁) + α₂T(e₂) +...+ α_nT(e_n)
Now we can associate a matrix to this linear Transformation in the following way:
Let's show a vector v in V by an n*1 matrix (assuming it has n dimensions) and a vector w in W by an m*1 matrix(assuming it has m dimensions). then we can define the matrix A such that w=Av. the matrix A will be of the rank m*n. the columns of A will be the T(e_i)'s (the {e_i}'s are a basis for V, and the corresponding T(e_i)'s will form a basis for W). Hence we can write a vector w in W as a linear combination of the components of a vector v in V using the matrix multiplication Av. conversely, If you give me an equation like w=Av I can write it in the form T(α₁e₁+α₂e₂+...+α_ne_n) = α₁T(e₁) + α₂T(e₂) +...+ α_nT(e_n).

That's how I understand the relationship between linear transformations and matrices. correct me if I'm wrong.

This is about half the story, but it also contains a mistake. If dim W>dim V, {T(e_i)} doesn't have enough members to be a basis for W. Compare it to what I said:

Suppose that $A:U\rightarrow V$ is linear, that $\{u_j\}$ is a basis for U, and $\{v_i\}$ is a basis for V. Consider the equation y=Ax, and expand in basis vectors. $$y=y_i v_i$$$$Ax=A(x_j u_j)=x_j Au_j= x_j (Au_j)_i v_i$$ I'm using the Einstein summation convention: Since we're always supposed to do a sum over the indices that appear exactly twice, we can remember that without writing any summation sigmas (and since the operator is linear, it wouldn't matter if we put the summation sigma to the left or right of the operator). Now define $A_{ij}=(Au_j)_i$. The above implies that $$y_i=x_j(Au_j)_i=A_{ij}x_j$$ Note that this can be interpreted as a matrix equation in component form. $y_i$ is the ith component of y in the basis $\{v_i\}$. $x_j$ is the jth component of x in the basis $\{u_j\}$. $A_{ij}$ is row i, column j, of the matrix of A in the pair of bases $\{u_j\}$, $\{v_i\}$.

This is the whole story. Note that I'm not just saying that there's a matrix equation corresponding to y=Ax. I'm also saying exactly what the components are, given a basis for U and a basis for V. In particular, this means that the components of a linear functional T in a basis {e_i} are just T(e_i). Compare that to the definition of the components of a tensor, and you will see that it's the same idea applied to multilinear functionals.

I checked the thread, but the OP's question is not so understandable for me.

If you read the the first paragraph of my post, you will see that I didn't understand it either, and wrote a reply that ignores everything he said. You can skip the first two paragraphs of my reply and start reading at "A manifold is a set with some other stuff defined on it, the most important being coordinate systems". The post gives you an overview of the basics of differential geometry, and the posts it links to provide some of the details.

Doesn't that confirm the way I defined a tensor?

Yes. I didn't include that to show you the definition of a tensor, but to show you the definition of the components of a tensor. (You said that tensors have components, but you didn't say what they are). I wanted to include such a definition because I quickly skimmed through the old posts that I suggested that you read, and didn't see one in there.

I'm still struggling with the concept of a dual space and dual bases. I don't understand what they are and why they are defined. (in fact I've seen very different definitions of a dual space which makes it only more confusing for me)

What other definitions have you seen? I don't think I've seen another one. The word "continuous" or "bounded" is sometimes thrown in there, as in "V* is the set of all bounded linear functions from V into ℝ". This addition to the definition is only relevant when V is infinite-dimensional. A linear function is continuous if and only if it's bounded, and as long as we're dealing with finite-dimensional spaces, all linear functions are continuous (if we use the isomorphism between V and ℝⁿ, and the standard norm on ℝ, to define a norm on V).

 

[Fredrik]

well, I don't know what that notation means.φⁱ(e_j) = δ_ij. Does it mean that the inner product of φⁱ and e_i should be equal to kronecker delta or it means that if φ acts on e_i it'll produce a scalar? I guess the later is true. if that's true, then can we define the dual space of a vector space V as the space spanned by the basis
{φ¹,φ²,...,φⁿ} such that if any of them acts on a basis vector from V it produces a scalar number? (either 0 or 1).

Yes (to everything except the first guess about inner products).

the f(e_j)'s are just field elements? I thought they were vectors? I start getting confused from this point.

He used the word "functional" to indicate that the codomain of f is ℝ (the field that was used in the definition of the vector space). ℝ is a 1-dimensional vector space, so you could call its members "vectors" if you want to, but as you know, that would be unconventional.

 

[Deveno]

well, I don't know what that notation means.φⁱ(e_j) = δ_ij. Does it mean that the inner product of φⁱ and e_i should be equal to kronecker delta or it means that if φ acts on e_i it'll produce a scalar? I guess the later is true. if that's true, then can we define the dual space of a vector space V as the space spanned by the basis
{φ¹,φ²,...,φⁿ} such that if any of them acts on a basis vector from V it produces a scalar number? (either 0 or 1).


the f(e_j)'s are just field elements? I thought they were vectors? I start getting confused from this point.

φⁱ(e_j) = δ_ij means that:

φⁱ(e_j) = 0 if i ≠ j,

φⁱ(e_i) = 1.

this is pretty obvious, the i-th coordinate of e_j is 0, unless i = j (that's how we define the standard basis).

it's not an "inner product", φⁱ is a function that when given a vector, returns its i-th coordinate (which is a field element). so, in R³ for example:

φ¹(1,2,4) = 1
φ²(1,2,4) = 2
φ³(1,2,4) = 4

so, yeah, φⁱ(e_j) is a scalar, and φⁱ is defined so that it kills every basis vector except e_i, and for e_i, it gives the i-th coordinate, which is...1 (it's the only non-zero coordinate e_i has).

in R³, the dual basis to {(1,0,0),(0,1,0),(0,0,1)} is the 3 linear functionals:

φ¹(x,y,z) = x
φ²(x,y,z) = y
φ³(x,y,z) = z.

any linear functional on R³ can be written as a linear combination of these 3.

for example, suppose f(x,y,z) = 3x + 2(y-z). that's a perfectly good linear functional.

we can write f = 3φ¹ + 2φ² - 2φ³, so in the dual space, with this particular dual basis, we have f = (3,2,-2) (just remember, this isn't really a "vector" but a way to specify WHICH linear functional we mean: take 3 times the first coordinate of a vector, then add 2 times the 2nd coordinate, then subtract 2 times the third coordinate, which is a perfectly good way to get a number from a vector).

and of course the f(e_j) are scalars! what does f do? it spits out a number, when you input a vector. well, a basis vector is still a vector, so when we apply a linear functional to a vector, we get a scalar.

 

[Fredrik]

well, I don't know what that notation means.φⁱ(e_j) = δ_ij. Does it mean that the inner product of φⁱ and e_i should be equal to kronecker delta or it means that if φ acts on e_i it'll produce a scalar? I guess the later is true. if that's true, then can we define the dual space of a vector space V as the space spanned by the basis
{φ¹,φ²,...,φⁿ} such that if any of them acts on a basis vector from V it produces a scalar number? (either 0 or 1).

I just realized that what you said suggests that you may have misunderstood the term "scalar". It doesn't mean "either 0 or 1". Also, regarding your suggested definition of the dual space, I don't understand the "such that" part of it. If that's supposed to be a definition of the φⁱ, it's incomplete.

 

[AdrianZ]

so, if I've understood it correctly, the dual space is nothing but a concept that arises from the linear 'functionals' on a vector space? (I didn't know what a functional meant before this thread). Can I say that any vector space has a corresponding dual space consisting of all functionals on V?so, the key to understanding dual spaces is to understand what linear functionals are. right?

I just realized that what you said suggests that you may have misunderstood the term "scalar". It doesn't mean "either 0 or 1". Also, regarding your suggested definition of the dual space, I don't understand the "such that" part of it. If that's supposed to be a definition of the φⁱ, it's incomplete.

I meant any scalars, not necessarily 0 and 1. my confusion arises from this part:

φi(ej) = δij means that:

φi(ej) = 0 if i ≠ j,

φi(ei) = 1.

this is pretty obvious, the i-th coordinate of ej is 0, unless i = j (that's how we define the standard basis).

it's not an "inner product", φi is a function that when given a vector, returns its i-th coordinate (which is a field element). so, in R3 for example:

φ1(1,2,4) = 1
φ2(1,2,4) = 2
φ3(1,2,4) = 4

We say that φⁱ(e_j)=δij. from that I understand that the only possible values for φ are 0 and 1. so, when I see the example, I see something totally different. It says φ3(1,2,4) = 4. the idea is pretty obvious, but I don't understand how φⁱ(e_j)=δij explains this.
so, if φⁱ(e_j)=δij, then δij is either 0 or 1. How can φ3(1,2,4) be equal to 4?

 

[Deveno]

the standard basis vectors e_j are special, they only have 0 or 1 coordinates.

so the dual basis to them, acting on them, will only return a 0 or 1 value. this doesn't mean that {0,1} is the range of the dual basis, in general a vector will have any possible field element as coordinate values.

we just define the dual basis to the standard basis by:

1) defining the values on a basis set
2) extending by linearity.

for example, earlier i wrote φ²(1,2,4) = 2. let's expand this a bit:

φ²(1,2,4) = φ²((1,0,0) + (0,2,0) + (0,0,4))

= φ²(1(1,0,0) + 2(0,1,0) + 4(0,0,1))

= φ²(1(1,0,0)) + φ²(2(0,1,0)) + φ²(4(0,0,1)) (by linearity!)

= 1(φ²(1,0,0)) + 2(φ²(0,1,0)) + 4(φ²(0,0,1)) (by linearity again!)

= (1)(0) + 2(1) + 4(0) (using the dual basis kroenecker delta, it's only 1 when the indices match)

= 0 + 2 + 0 = 2 (using...um, addition of real numbers).

 

[AdrianZ]

Yaayy! I guess I've understood what a dual basis is. now a dual space is the space spanned by that basis? If yes then I've understood what a dual space is.

now, what are co-variant and contra-variant vectors? How do they arise in math/physics?

 

[Fredrik]

In the simplest non-trivial case $V=\mathbb R^2$, $\phi^i(e_j)=\delta^i_j$ are the four equalities $$\begin{align} \phi^1(e_1) &=1\\ \phi^1(e_2) &=0\\ \phi^2(e_1) &=0\\ \phi^2(e_2) &=1 \end{align}$$ Since the $\phi^i$ are assumed to be linear, we have $\phi^i(v)=\phi^i(v^je_j)=v^j\phi^i(e_j)=v^j\delta^i_j=v^i$, and in particular $\phi^1(ae_1+be_2)=a\phi^1(e_1)+b\phi^1(e_2)=a$.

now, what are co-variant and contra-variant vectors? How do they arise in math/physics?

I explained that in the post I linked to earlier. The one that made you say that you didn't understand the OP's question.

now a dual space is the space spanned by that basis?

Yes, but that would be a weird way to define it. V* is the set of linear functionals from V into ℝ. Addition and scalar multiplication are defined in the obvious ways:

(f+g)(v)=f(v)+g(v)
(af)(v)=a(f(v))

These definitions give V* the structure of a vector space. Let f in V* and v in V be arbitrary. We have $f(v)=f(v^i e_i)=v^if(e_i)=\phi^i(v)f(e_i)$. Since v was arbitrary, this means that $f=f(e_i)\phi^i$. Since f was arbitrary, this means that $\{\phi^i\}$ spans V*.

 

[AdrianZ]

great. now I have almost understood the idea behind defining dual spaces and also the idea behind covariant and contravariant vectors. your post about co/contra-variant vectors was a great one.

now the only thing remained unclear for me is that I want to see some mathematical/physical examples of tensors. and I want to understand how we can interpret different physical situations with the new language I've learned. I have almost 0 level of knowledge about differential geometry, so don't go into very advanced topics from physics please (like GR).

Is there any other thing that I need to know about tensors?