# The tensor product of tensors confusion

• I
Summary:
Why is the tensor product of two tensors again a tensor?
> **Exercise.** Let T1and T2be tensors of type (r1 s1)and (r2 s2) respectively on a vector space V. Show that T1⊗
T2can be viewed as an (r1+r2 s1+s2)tensor, so that the
> tensor product of two tensors is again a tensor, justifying the
> nomenclature...
What I’m reading：《An introduction to tensors and group theory for physicists》Authors: Jeevanjee, Nadir. According to it，
a tensor is a multilinear function that eats r vectors as well as s dual vectors and produces a number...And "Give two finite-dimensional vector spaces V and W,we define their tensor product V⊗W to be the set of all C-valued bilinear functions on V*×W *..."
How do I define the tensor product of two tensors by these definition?And What the target object this tensor product acts on should look like？

fresh_42
Mentor
Summary:: Why is the tensor product of two tensors again a tensor?

> **Exercise.** Let T1and T2be tensors of type (r1 s1)and (r2 s2) respectively on a vector space V. Show that T1⊗
T2can be viewed as an (r1+r2 s1+s2)tensor, so that the
> tensor product of two tensors is again a tensor, justifying the
> nomenclature...
What I’m reading：《An introduction to tensors and group theory for physicists》Authors: Jeevanjee, Nadir. According to it，
a tensor is a multilinear function that eats r vectors as well as s dual vectors and produces a number...And "Give two finite-dimensional vector spaces V and W,we define their tensor product V⊗W to be the set of all C-valued bilinear functions on V*×W *..."
How do I define the tensor product of two tensors by these definition?And What the target object this tensor product acts on should look like？
A tensor is a multilinear function, but it doesn't produce a number. Only the dual vectors if fed by vectors turn into scalars, the part with the vectors remains.

We have
$$T_i=\underbrace{v_1\otimes v_2\otimes \ldots\otimes v_{r_i}}_{r_i\text{ many vectors}}\otimes \underbrace{v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_i}}_{s_i\text{ many covectors}}\quad (i=1,2)$$
so
$$T_1\otimes T_2=v_1\otimes v_2\otimes \ldots\otimes v_{r_1}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_1}\otimes v_1\otimes v_2\otimes \ldots\otimes v_{r_2}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_2}$$
which can be sorted to an equivalent tensor
$$T_1\otimes T_2=\underbrace{v_1\otimes v_2\otimes \ldots\otimes v_{r_1}\otimes v_1\otimes v_2\otimes \ldots\otimes v_{r_2}}_{r_1+r_2 \text{ many vectors}} \otimes \underbrace{v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_1}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_2}}_{s_1+s_2 \text{ many covectors}}$$

Here is an example of a ##(1,2)## tensor, which is a bilinear function:
https://www.physicsforums.com/insights/what-is-a-tensor/

andrewkirk
Homework Helper
Gold Member
A tensor is a multilinear function, but it doesn't produce a number. Only the dual vectors if fed by vectors turn into scalars, the part with the vectors remains.
I think we can say it produces a number. If we take the usual definition of a (m n) tensor over vector space V on field F as a multi-linear function whose domain is the cartesian product of a collection of m copies of V and n copies of the dual space V*, and whose range is F, then we can say the function 'produces a number', meaning just that it maps ordered tuples of m vectors and n covectors to numbers (scalars).

How do I define the tensor product of two tensors by these definition?And What the target object this tensor product acts on should look like？
If for ##i\in\{1,2\}##, tensor ##T_i## over vector space V and field F is ##(m_i\ \ n_i)##, it maps ##m_i## vectors in V (##v_{i,1}, v_{i,2}, ..., v_{i,m_i} ## ) and ##n_i## covectors in V* ( ## p_{i,1}, p_{i,2}, ..., p_{i,n_i}## ) to a number in F, denoted by :
$$T_i(v_{i,1}, v_{i,2}, ..., v_{i,m_i}, p_{i,1}, p_{i,2}, ..., p_{i,n_i})$$

then the vector ##T_1\otimes T_2## is ##(m_1+m_2\ \ n_1+n_2)## and maps ##m_1+m_2## vectors in V (##u_1,u_2, .., u_{m_1+m_2}## ) and ##n_1+n_2## covectors in V* ( ## q_1, q_2,..., q_{n_1+n_2}## ) to a number in F, denoted by
$$(T_1\otimes T_2) (u_1,u_2, .., u_{m_1+m_2}, q_1, q_2,..., q_{n_1+n_2})$$
which is equal to
$$T_1(u_1, u_2, ..., u_{m_1}, p_1, p_2, ..., p_{n_1})\times T_1(u_{m_1+1}, u_{m_1+2}, ..., u_{m_1+m_2}, p_{n_1+1}, p_{n_1+2}, ..., p_{n_1+n_2})$$
In other words, to get the result from applying the product tensor, you just multiply the results of applying the two mutiplicand tensors to their respective arguments taken from the appropriate places in the input tuple to the product.

fresh_42
Mentor
I think we can say it produces a number. If we take the usual definition of a (m n) tensor over vector space V on field F as a multi-linear function whose domain is the cartesian product of a collection of m copies of V and n copies of the dual space V*, and whose range is F, then we can say the function 'produces a number', meaning just that it maps ordered tuples of m vectors and n covectors to numbers (scalars).
No, we cannot. You can feed the covectors, but the vectors remain vectors. ##\sum u_i\otimes v_i## is a matrix and not a number.

weirdoguy
andrewkirk
Homework Helper
Gold Member
I think you are misreading the OP. 'it produces a number' doesn't say the tensor product is a number. It says that it is a function that delivers a result that is a number - ie the function's range is a space of numbers (typically ##\mathbb R## or ##\mathbb C##),

A tensor is a multilinear function, but it doesn't produce a number. Only the dual vectors if fed by vectors turn into scalars, the part with the vectors remains.

We have
$$T_i=\underbrace{v_1\otimes v_2\otimes \ldots\otimes v_{r_i}}_{r_i\text{ many vectors}}\otimes \underbrace{v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_i}}_{s_i\text{ many covectors}}\quad (i=1,2)$$
so
$$T_1\otimes T_2=v_1\otimes v_2\otimes \ldots\otimes v_{r_1}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_1}\otimes v_1\otimes v_2\otimes \ldots\otimes v_{r_2}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_2}$$
which can be sorted to an equivalent tensor
$$T_1\otimes T_2=\underbrace{v_1\otimes v_2\otimes \ldots\otimes v_{r_1}\otimes v_1\otimes v_2\otimes \ldots\otimes v_{r_2}}_{r_1+r_2 \text{ many vectors}} \otimes \underbrace{v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_1}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_2}}_{s_1+s_2 \text{ many covectors}}$$

Here is an example of a ##(1,2)## tensor, which is a bilinear function:
https://www.physicsforums.com/insights/what-is-a-tensor/

A tensor is a multilinear function, but it doesn't produce a number. Only the dual vectors if fed by vectors turn into scalars, the part with the vectors remains.

We have
$$T_i=\underbrace{v_1\otimes v_2\otimes \ldots\otimes v_{r_i}}_{r_i\text{ many vectors}}\otimes \underbrace{v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_i}}_{s_i\text{ many covectors}}\quad (i=1,2)$$
so
$$T_1\otimes T_2=v_1\otimes v_2\otimes \ldots\otimes v_{r_1}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_1}\otimes v_1\otimes v_2\otimes \ldots\otimes v_{r_2}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_2}$$
which can be sorted to an equivalent tensor
$$T_1\otimes T_2=\underbrace{v_1\otimes v_2\otimes \ldots\otimes v_{r_1}\otimes v_1\otimes v_2\otimes \ldots\otimes v_{r_2}}_{r_1+r_2 \text{ many vectors}} \otimes \underbrace{v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_1}\otimes v^*_1\otimes v^*_2\otimes \ldots\otimes v^*_{s_2}}_{s_1+s_2 \text{ many covectors}}$$

Here is an example of a ##(1,2)## tensor, which is a bilinear function:
https://www.physicsforums.com/insights/what-is-a-tensor/
Why can s1 covectors and r2 vectors commute with each other at the last step？

fresh_42
Mentor
I think you are misreading the OP. 'it produces a number' doesn't say the tensor product is a number. It says that it is a function that delivers a result that is a number - ie the function's range is a space of numbers (typically ##\mathbb R## or ##\mathbb C##),
Sure, but this is definitely wrong! Except we have tensors of type ##(0,m)##.

If we have ##(n,m)## tensors with ##n>0## then there is no way to make ##n## go away. E.g. a ##(1,2)## tensor is - considered as a function - an algebra multiplication. Since when produces an algebra multiplication a number?

fresh_42
Mentor
Why can s1 covectors and r2 vectors commute with each other at the last step？
It is not that you commute the factors. It is strictly speaking an isomorphism of vector spaces, or tensor spaces if you like. An isomorphism that isn't written down as it doesn't really carry information. It's simply convenient to sort them into the ##(n,m)## scheme. And if you consider a tensor space ##T(V)## over a vector space ##V## as an algebra with the multiplication ##(T_1,T_2)\longmapsto T_1\otimes T_2## then you have to apply this isomorphism. Loosely speaking: it doesn't matter if we are talking about length times height or height times length, although there would be a formal difference.

martinbn
Sure, but this is definitely wrong! Except we have tensors of type ##(0,m)##.

If we have ##(n,m)## tensors with ##n>0## then there is no way to make ##n## go away. E.g. a ##(1,2)## tensor is - considered as a function - an algebra multiplication. Since when produces an algebra multiplication a number?
I don't see what the problem is! The input is vectors and one forms, the output is a number.

fresh_42
Mentor
I don't see what the problem is! The input is vectors and one forms, the output is a number.
What number? Repetition does not make it right. Let ##V## be a vector space, and ##v\in V##. Then ##v## is obviously a vector and not a number. However, ##v## is a tensor.

If ##v\in V^*##, then ##v^*(w)## is a function that produces a number, but not in case ##v\in V##.

martinbn
What number? Repetition does not make it right. Let ##V## be a vector space, and ##v\in V##. Then ##v## is obviously a vector and not a number. However, ##v## is a tensor.

If ##v\in V^*##, then ##v^*(w)## is a function that produces a number, but not in case ##v\in V##.
In this case ##v## can be viewed as an element of ##V^{**}##, so for each ##\omega\in V^*## you get the number ##v(\omega)##.

fresh_42
Mentor
In this case ##v## can be viewed as an element of ##V^{**}##, so for each ##\omega\in V^*## you get the number ##v(\omega)##.
Yes, and the projection on the first coordinate is a number, too. The question was "what is a tensor" and not "what can I do to make it a number". To call a tensor a multilinear function into the scalar field is wrong. Your poor construction as an excuse is additional framework. It is as if you were saying that all vectors were covectors. This is confusing and wrong, at least in mathematics. That physicists always make vectors covectors is another subject.

A vector is a tensor, and it is neither a function nor a number.

Repetition of wrong does not change this fact.

weirdoguy
martinbn
Yes, and the projection on the first coordinate is a number, too. The question was "what is a tensor" and not "what can I do to make it a number". To call a tensor a multilinear function into the scalar field is wrong. Your poor construction as an excuse is additional framework. It is as if you were saying that all vectors were covectors. This is confusing and wrong, at least in mathematics. That physicists always make vectors covectors is another subject.

A vector is a tensor, and it is neither a function nor a number.

Repetition of wrong does not change this fact.
This is standard in mathematics. Many define tensors as multilinear maps. It is equavalent to what you consider the only definition and cannot be wrong. It is a matter of personal prefernce.

It is also not hard to see that the space of all linear maps

##V^*\times\cdots\times V^*\times V\times\cdots\times V \rightarrow \mathbb R##

and the space

##V\otimes\cdots\otimes V\otimes V^*\otimes\cdots\otimes V^* ##

are cannonically isomorphic.

fresh_42
Mentor
This is standard in mathematics. Many define tensors as multilinear maps. It is equavalent to what you consider the only definition and cannot be wrong. It is a matter of personal prefernce.

It is also not hard to see that the space of all linear maps

##V^*\times\cdots\times V^*\times V\times\cdots\times V \rightarrow \mathbb R##

and the space

##V\otimes\cdots\otimes V\otimes V^*\otimes\cdots\otimes V^* ##

are cannonically isomorphic.
It would be more accurate to say: all tensors are vectors, scalars and covectors included, than the other way around.

Mathematics distinguishes between scalars, vectors, and covectors. Any isomorphisms come afterward!

If it comes to differential forms or tangent spaces, things become complicated. ##dx## is still a vector, a tangent, but it can be evaluated at a certain point which makes it a number, a slope. So some tangent vectors can be seen as functions that produce numbers, but this is not what a tensor is. It is how it can be used. That is something different.

E.g. I can write ##\mathfrak{su}(2)## as a Lie algebra of differential operators. Its multiplication structure is a bilinear mapping ##\mathfrak{su}(2) \times \mathfrak{su}(2) \longrightarrow \mathfrak{su}(2)## which is a ##(2,1)## tensor. And although the vectors are written as differentials, the tensor itself doesn't become a number-producing machine. It produces differential operators.

To consider curvature tensors or alike, one has to set additional structures upon the tensor. It is what you make out of it, not what it is. A tensor is still a simple vector because it is part of a vector space. And vectors and covectors are different objects, mathematically.

The phrasing in the OP's book is misguiding. Not all vectors are covectors, but all covectors are vectors.

Last edited:
martinbn
It would be more accurate to say: all tensors are vectors, scalars and covectors included, than the other way around.

Mathematics distinguishes between scalars, vectors, and covectors. Any isomorphisms come afterward!

If it comes to differential forms or tangent spaces, things become complicated. ##dx## is still a vector, a tangent, but it can be evaluated at a certain point which makes it a number, a slope. So some tangent vectors can be seen as functions that produce numbers, but this is not what a tensor is. It is how it can be used. That is something different.

E.g. I can write ##\mathfrak{su}(2)## as a Lie algebra of differential operators. Its multiplication structure is a bilinear mapping ##\mathfrak{su}(2) \times \mathfrak{su}(2) \longrightarrow \mathfrak{su}(2)## which is a ##(2,1)## tensor. And although the vectors are written as differentials, the tensor itself doesn't become a number-producing machine. It produces differential operators.

To consider curvature tensors or alike, one has to set additional structures upon the tensor. It is what you make out of it, not what it is. A tensor is still a simple vector because it is part of a vector space. And vectors and covectors are different objects, mathematically.

The phrasing in the OP's book is misguiding. Not all vectors are covectors, but all covectors are vectors.
I am not sure how this relates to what I said. My point is simply that there is nothing wrong, and is often done especially in differential geometry books, to define tensors as multilinear maps. If the OP uses a book that goes that way, does it help to call it "wrong"!?

fresh_42
Mentor
It is wrong to treat all vectors as covectors! You can do it a) the other way around, or b) do not ask a mathematician, or c) learn it wrong.

We would lead the very same discussion the other way around if I would claim that all vectors are covectors and that there is no difference between them. You bet!

martinbn
It is wrong to treat all vectors as covectors! You can do it a) the other way around, or b) do not ask a mathematician, or c) learn it wrong.

We would lead the very same discussion the other way around if I would claim that all vectors are covectors and that there is no difference between them. You bet!
I am still not sure what you mean. What do you mean by "treat all vectors as covectors"?

fresh_42
Mentor
I am still not sure what you mean. What do you mean by "treat all vectors as covectors"?
Sure, that is what you repeatedly and falsely claim. You need covectors to get scalars, and you claim that all parts of a tensor are covectors. Sorry, but this is still wrong. I have provided multiple examples where such a statement is obviously wrong, yet, you insist that vectors are covectors. And, no, do not say no, this is exactly what it means if you want to spit out numbers. Disguise it as you want: Covectors are vectors (but primarily not in this context where both were distinguished already in the definition of ##(n,m)## or ##(r,s)##), but vectors are not covectors, everything else is voodoo.

martinbn
Here is what I said: Given two (any number) vector spaces ##V## and ##W## over a field ##F##, the vector spaces ##T=V\otimes W## and ##L=\text{all multilinear maps from } V^*\times W^*\text{ to }F## are canonically isomorphic. This is TRUE. Therefore one can, and one does sometimes, use the second one as a definition of the first one.

fresh_42
Mentor
Here is what I said: Given two (any number) vector spaces ##V## and ##W## over a field ##F##, the vector spaces ##T=V\otimes W## and ##L=\text{all multilinear maps from } V^*\times W^*\text{ to }F## are canonically isomorphic. This is TRUE. Therefore one can, and one does sometimes, use the second one as a definition of the first one.

Because the definition says ##(r,s)##-tensor. This means ##r## vectors and ##s## covectors.

I am complaining that you basically claim it is a ##(0,r+s)##-tensor, which it is not.
(I know the difference, but that doesn't become clear at all by what you said.)

A ##(r,s)##-tensor ##T## regarded as a function is a multilinear map ##T: V^s \longrightarrow V^{\otimes r}## and not into a field. It is simply wrong and misleading to state otherwise.

Your entire claims are flawed and rely on additional constructions. The ##r## part are vectors and not functions. What makes me angry is, that I know that you know you are talking bs.

weirdoguy and martinbn
martinbn
I give up. It is not my problem that you are stubborn.

weirdoguy, fresh_42 and Gaussian97
fresh_42
Mentor
I give up. It is not my problem that you are stubborn.
Yep, continue to tell that all vectors are covectors. I think you should write a paper. It would a revolution in homological algebra!

Gaussian97
Homework Helper
Yep, continue to tell that all vectors are covectors. I think you should write a paper. It would a revolution in homological algebra!
I also studied tensors as multilinear maps from ##V^{*p} \times V^{q} \to \mathbb{R}## could you explain exactly how this implies that all vectors are covectors?

fresh_42
Mentor
I also studied tensors as multilinear maps from ##V^{*p} \times V^{q} \to \mathbb{R}## could you explain exactly how this implies that all vectors are covectors?
##V^{*p}## eats ##p## many independent vectors and turns them into scalars.
But what do you do with ##V^{q}##? They remain vectors and do not map to ##\mathbb{R}##.
If you want to map them to ##\mathbb{R}##, then you have to make them covectors, i.e. eating vectors to turn them into numbers, or manipulate them otherwise.

Now my questions:
• If we consider this ##(p,q)##-tensor as multilinear mapping into ##\mathbb{R}##, why isn't it a ##(p+q,0)##-tensor to begin with?
• What happened to the vectors?
• Strassen's algorithm for matrix multiplication is a ##(2,1)##-tensor. Where does it end up in ##\mathbb{R}##?
• A vector isn't a function. If we treat it as such, then we must do something, namely setting ##(V^*)^* =V## or something else, e.g. evaluating a differential form at a certain point. These cannot be viewed as automatisms. We actively turn a vector into something else, mostly into ##v \leftrightsquigarrow (w \longmapsto \langle w,v \rangle).## And if there is no inner product?
• One has to be careful with infinite-dimensional spaces:
https://en.wikipedia.org/wiki/Dual_space#Infinite-dimensional_case

martinbn
##V^{*p}## eats ##p## many independent vectors and turns them into scalars.
But what do you do with ##V^{q}##? They remain vectors and do not map to ##\mathbb{R}##.
If you want to map them to ##\mathbb{R}##, then you have to make them covectors, i.e. eating vectors to turn them into numbers, or manipulate them otherwise.
No, you feed them covectors.
Now my questions:
• If we consider this ##(p,q)##-tensor as multilinear mapping into ##\mathbb{R}##, why isn't it a ##(p+q,0)##-tensor to begin with?
• What happened to the vectors?
• Strassen's algorithm for matrix multiplication is a ##(2,1)##-tensor. Where does it end up in ##\mathbb{R}##?
• A vector isn't a function. If we treat it as such, then we must do something, namely setting ##(V^*)^* =V## or something else, e.g. evaluating a differential form at a certain point. These cannot be viewed as automatisms. We actively turn a vector into something else, mostly into ##v \leftrightsquigarrow (w \longmapsto \langle w,v \rangle).## And if there is no inner product?
• One has to be careful with infinite-dimensional spaces:
https://en.wikipedia.org/wiki/Dual_space#Infinite-dimensional_case
##V^{**}## is canonically isomorphic to ##V##. No extra structure needed, no choice of basis needed, no nothing, no matter the dimension.

weirdoguy