# Tensor products: different sizes of apparently the same set

1. Aug 20, 2015

### ELB27

1. The problem statement, all variables and given/known data
Show that the set $\{\mathbf{v}_1⊗\mathbf{v}_2⊗...⊗\mathbf{v}_p:\mathbf{v}_k\in V_k\}$ of tensor products of vectors is strictly less than $V_1⊗V_2⊗...⊗V_p$.

2. Relevant equations

3. The attempt at a solution
Truly, I don't see any difference between the sets. They seem identical to me. It seems to me that the first set can be used as the definition of the second.

If anyone could point me in the correct direction here to spot the difference I would greatly appreciate it!

2. Aug 20, 2015

### HallsofIvy

Staff Emeritus
What do you mean by "strictly less"? What order relation are you using?

3. Aug 20, 2015

### micromass

Staff Emeritus
First, how did you define the tensor product $V_1\otimes...\otimes V_n$. What the question is asking of you is to find an element of $V_1\otimes...\otimes V_n$ which is not of the form $\mathbf{v}_1\otimes...\otimes\mathbf{v}_n$.

4. Aug 21, 2015

### ELB27

Honestly, I don't know. I understood the question as micromass - simply to prove that there exists an element of the first space which is not part of the second. The question is Problem 5.2 from page 244 of Linear Algebra Done Wrong.
I define (as in, according to the definition in the book) the tensor product $V_1\otimes...\otimes V_p$ as the collection of all multilinear functionals depending on $\mathbf{v}_1,...,\mathbf{v}_p ; \mathbf{v}_k\in V_k$.
I know that the collection of functionals $b_{j1}^1\otimes...\otimes b_{jp}^{p} ; 1\leq j1\leq \text{dim} V_k$ is a basis in $V_1\otimes... V_p$ where $\{b_{jk}^k\}$ is a basis in $V_k$. Since the tensor product is linear in each argument, and since any vector $\mathbf{v}_k\in V_k$ can be represented uniquely as a linear combination of $\{b_{jk}^k\}$, we can write
$$\mathbf{v}_1\otimes...\otimes\mathbf{v}_p = \left(\sum_{j1}\alpha_{j1}^1 b_{j1}^1 \right)\otimes...\otimes \left(\sum_{jp}\alpha_{jp}^p b_{jp}^p \right) = \sum_{j1,...,jp} \alpha_{j1}^1\cdot\cdot\cdot\alpha_{jp}^p b_{j1}^1\otimes...\otimes b_{jp}^p$$
so, since an arbitrary element can be expressed as a linear combination of a basis, $$\{\mathbf{v}_1\otimes...\otimes\mathbf{v}_p\} ⊂ V_1\otimes...\otimes V_p$$ A general element of the tensor product of spaces would be written as: $$F = \sum_{j1,...,jp} \beta_{j1,...,jp} b_{j1}^1\otimes...\otimes b_{jp}^p$$
So the question becomes is there a collection of $\beta$ which cannot be consistently expressed as a product of $\alpha$'s. Intuitively, I would say that it is always possible (although the particular vectors forming the functional may not be unique as a product of different numbers can still be equal).

However, I don't see any direction to finding an element of $V_1\otimes...\otimes V_p$ that cannot be written as $\mathbf{v}_1\otimes...\otimes\mathbf{v}_p$...

5. Aug 21, 2015

### micromass

Staff Emeritus
So put $V=\mathbb{R}^2$ and $W=\mathbb{R}^2$. Can you show that $(1,0)\otimes (1,0) + (0,1)\otimes (0,1)$ is a good example?

6. Aug 22, 2015

### andrewkirk

This post of mine needed fixing. Is there a delete button in pf?

Last edited: Aug 22, 2015
7. Aug 23, 2015

### ELB27

Yes,
$$(1,0)\otimes (1,0) + (0,1)\otimes (0,1) = \sum_{j1,j2=1}^2 \beta_{j1,j2} b_{j1}^1\otimes b_{j2}^2$$
where $b_1^{1,2}=(1,0), b_2^{1,2} = (0,1)$, $b^k_{jk} \in V_k$ and $\beta_{1,1}=\beta_{2,2}=1 ; \beta_{1,2}=\beta_{2,1}=0$. To represent this functional as a tensor product of vectors, we want (see my prev. post):
$$\beta_{j1,j2} = \alpha^1_{j1}\alpha^2_{j2}$$ which implies:
$$\alpha_1^1\alpha_1^2 = 1 \qquad \alpha_2^1\alpha_2^2=1$$
So, $$\alpha_1^1, \alpha_1^2, \alpha_2^1, \alpha_2^2 \neq 0$$
But we also must have
$$\alpha_1^1\alpha_2^2=\alpha_2^1\alpha_1^2 =0$$ which is impossible if none of the variables is zero. So we have an inconsistency and such a representation is impossible.

I now see that the product representation $\alpha_{j1}^1\cdot\cdot\cdot \alpha_{jp}^p$ is more restricting than I initially thought because each member of the product depends on one index $jk$ only. In case $\alpha_{jk}^k = 0$ for some $k$, the whole product is zero for any other indexes as long as the particular $jk$ remains fixed and this is not a necessity in general.

Thank you very much for your help!

EDIT: I wonder, is this a formal proof or should I write a more general statement about the class of functionals not included in the first set?

Last edited: Aug 23, 2015
8. Aug 23, 2015

### andrewkirk

To convert it to the general case, we can replace
$$(1,0)\otimes (1,0) + (0,1)\otimes (0,1)$$

$$T=b_1^1\otimes b_1^2\otimes\bigotimes_{k=3}^p b^k_1 + b_2^1\otimes b_2^2\otimes\bigotimes_{k=3}^p b^k_1$$

and assume that there are vectors $v_1,...,v_m$ in $V_1,...,V_m$ respectively such that:
$$T=\bigotimes_{k=1}^m v_{k}$$

Then we can get generalised equivalents of your above equations, as follows:

$$\alpha^1_1\alpha^2_1\prod_{k=3}^m\alpha^k_1=\alpha^1_2\alpha^2_2\prod_{k=3}^m\alpha^k_1=1$$
and
$$\alpha^1_1\alpha^2_2\prod_{k=3}^m\alpha^k_1=\alpha^1_2\alpha^2_1\prod_{k=3}^m\alpha^k_1=0$$

from which we can prove a contradiction using the same arguments as you used above, since the new extra factor $\prod_{k=3}^m\alpha^k_1$ is common to both equations.

9. Aug 23, 2015

### ELB27

Thank you very much!

I was wondering however about whether or not only elements with zeros cannot be represented as a tensor product of vectors. I.e., if your proposed general case is the most general one or are there additional ones?

10. Aug 23, 2015

### andrewkirk

It's not only elements with zeros, because whether the n-tuple representation of a vector is mostly zeros depends on the basis chosen. Any vector that has all zeros except one coordinate in a given basis can be represented as an element with no zeros in another basis.

Consider micromass's example of $(1,0)\otimes(1,0)+(0,1)\otimes(0,1)$. If we switch basis for both $V_1$ and $V_2$ to use basis vectors $(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})$ and $(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$ (ie just rotate the basis axes by 45 degrees) then micromass's tensor is represented in the new basis as $(1,1)\otimes(1,1)+(1,-1)\otimes(1,-1)$, which has no zeros. Yet it is the same tensor.

Conversely, any tensor that has no zeros in its representation in a given basis can be given a representation that has many (1,0,0,...,0) and (0,1,0,0,...,0) elements just by changing to a suitable basis.

11. Aug 23, 2015

### andrewkirk

A little extra thought on this:

The tensor product space is a much 'bigger' set than the set of tensor products. One can get a non-rigorous intuition for this as follows. Let $C$ be the cardinality of the real numbers. Note that the tensor product space is a vector space of dimension $\prod_{k=1}^p n_k$ where $n_k=\dim V_k$. So the tensor product space has cardinality
$$C^{\prod_{k=1}^p n_k}$$
On the other hand the set of tensor products has cardinality
$$\prod_{k=1}^p\left|V_k\right| =\prod_{k=1}^p C^{n_k} =C^{\sum_{k=1}^p n_k}$$

The latter looks like a much bigger cardinality than the former. This demonstration is non-rigorous, because $C^n=C$. But nevertheless I find it helps with my intuition.

I imagine a rigorous demonstration that the set of tensor products is only a small part of the tensor product space would need to use some clever measure theory.

12. Aug 23, 2015

### micromass

Staff Emeritus
Nice, but

$$C^{\prod_{k=1}^p n_k} = C^{\sum_{k=1}^p n_k}$$

13. Aug 23, 2015

### WWGD

I assume that this means the set generated by expressions on the LH side is a strict subset of the RH side, IOW, the LH side does not span the set on the RH side, which is what someone said IOW above. Maybe you (OP) can use the fact that once we have bases for each individual$V_i$ , there is a (somewhat) canonical basis for the tensor product $V_1 \otimes V_2 ...\otimes V_p$.

Last edited: Aug 23, 2015
14. Aug 24, 2015

### ELB27

Thanks again! I think I see it now.
(I was referring to zeros as coefficients of basis vectors. $\alpha_{j_k}^k = 0$ for one or more $j_k$
in post #4. I should have been clearer - sorry for the confusion.)

I think you're referring to the basis I used in post #4 and subsequently: $b_{j_1}^1\otimes...\otimes b_{j_p}^p, \quad 1\leq j_k\leq \text{dim} V_k, \quad 1\leq k\leq p$ where $\{b_{j_k}^k\}_{j_k=1}^{\text{dim} V_k}$ is a basis in $V_k$.

Last edited: Aug 24, 2015