Dimensionality of the sum of subspaces

[PeroK]

You need (6) for the contradiction in (9). But (6) is only true, if the $u_i$ are orthonormal, since for an arbitrary vector as in (9) you need/used $\langle u_i,u_j \rangle = \delta_{ij}$. You can use this by the theorem, but you should mention it, because it is not given by the problem itself. So you have to say why you can make this assumption. Otherwise you only have $\langle u_i,v_j \rangle = 0$ and nothing about the inner product between the $u_i$ or $v_j$.

(6) comes from assuming $\Sigma c_i u_i = 0$

 

[fresh_42]

(6) comes from assuming $\Sigma c_i u_i = 0$

Yes, but in (9) he has an arbitrary vector which is not zero. They are not the same coefficients $c_i$.
In other words: (6) must not be applied in (9) because the conditions of (6) aren't given in (9).

 

[Pushoam]

Let's consider

$B_1 =\{u_1, u_2,...u_p\} ~~~~~~~~~...(1) \\B_2 = \{v_1, v_2, v_3,...v_q\}~~~~~~~...(2) \\p = n_1 , ~ q=n_2 ~~~~~~~~~~~~~~...(3),$where $B_1$ and $B_2$ are basis of $V_1$ and $V_2$.

Taking $B= B_1 \cup B_2 = \{ u_1, u_2,...u_p,v_1, v_2, v_3,...v_q\}$ ...(4)
Considering the following linear combination of basis vectors of $V_1$,
$| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ ...(5)
Taking dot product with $\langle u_1|$ gives,
$c_1 = - \langle u_1 | c_2u_2 +... +c_p u_p\rangle = 0$ ...(6)

A) Let's assume that the dimensionality of $V_1 + V_2 = n \lt p+q$ ...(7)
This means that there exists at least one element say $u_1$ which is not linearly independent of other vectors present in B.
$| c_1 u_1 + c_2u_2 +... +c_p u_p + d_1v_1 +d_2 v_2+...+ d_q v_q\rangle = |0\rangle ~,c_1 \neq 0 ...(8)$
Taking dot product with $\langle u_1|$ gives,
$c_1 = - \langle u_1 | c_2u_2 +... +c_p u_p\rangle \neq 0$ ...(9)
Thus, (9) contradicts (6) ⇒⇒ \Rightarrow there does not exist a single element in B which is linearly dependent on other vectors of B. Thus, all the elements in B is linearly independent. Hence, n is not less than p+q.

My mistake in the above proof is pointed below:
A) Let's assume that the dimensionality of $V_1 + V_2 = n \lt p+q$ ...(7)
This means that there exists at least one element say $u_1$ which is not linearly independent of other vectors present in B.
$| c_1 u_1 + c_2u_2 +... +c_p u_p + d_1v_1 +d_2 v_2+...+ d_q v_q\rangle = |0\rangle ~,c_1 \neq 0 ...(8)$
Taking dot product with $\langle u_1|$ gives,
$c_1 = - \langle u_1 | c_2u_2 +... +c_p u_p\rangle \neq 0$ ...(9)
Now consider $c_1 u_1 + c_2u_2 +... +c_p u_p$ of L.H.S. of (8). As $u_1, u_2 , u_3 ,... u_p$ are basis vectors of $V_1$, these are linearly independent.
So,
$| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ ...(5)
Taking dot product with $\langle u_1|$ gives,
$c_1 = - \langle u_1 | c_2u_2 +... +c_p u_p\rangle = 0$ ...(6) This is what I meant in the earlier calculation.
But the point is eqn (8) does not imply $c_1 u_1 + c_2u_2 +... +c_p u_p = 0$ So, eqn. (5) and (6) are not valid here. Now I see it. Thanks.

But, then I see only one way to prove. Using Gram Schmidt theorem ,I state that $V_1$ and $V_2$ have orthonormal basis. And then use this basis to prove as I did in post ## 10 and 11.

 

[fresh_42]

My mistake in the above proof is pointed below:
A) Let's assume that the dimensionality of $V_1 + V_2 = n \lt p+q$ ...(7)
This means that there exists at least one element say $u_1$ which is not linearly independent of other vectors present in B.
$| c_1 u_1 + c_2u_2 +... +c_p u_p + d_1v_1 +d_2 v_2+...+ d_q v_q\rangle = |0\rangle ~,c_1 \neq 0 ...(8)$
Taking dot product with $\langle u_1|$ gives,
$c_1 = - \langle u_1 | c_2u_2 +... +c_p u_p\rangle \neq 0$ ...(9)

Again. This is NOT true. With $n < p+q$ you may assume w.l.o.g. $c_1=1$ and $u_1 = \sum_{i>1}c_iu_i + \sum_{j\geq 1}d_jv_j\,.$

1. To mention "w.l.o.g." is important. It means that you performed a certain numbering of your basis vectors in order to have the first one as the linear dependent one. It also means, that further renumber will only be allowed, if the first one is affected again. You won't need another renumber here, but it is important that you know what you did and that this is a consequence of it.
2. $\langle u_1\,|\,v_j\rangle = 0$ and so is $$\langle u_1\,|\, u_1\rangle = \langle u_1\,|\,\sum_{i>1}c_iu_i + \sum_{j\geq 1}d_jv_l\rangle = \sum_{i>1}c_i\langle u_1\,|\,u_i\rangle$$ because $u_1\perp v_j\,.$
3. Next you used without mention $\langle u_1\,|\,u_i\rangle = \delta_{1i}\,.$ Why are you allowed to do so? Nowhere up to now, i.e. in this post, is anything written about $u_i\perp u_j \quad (i\neq j)$ and the problem statement doesn't say it either! So until now, there is no argument, why $u_1 \perp \operatorname{span}\{\,u_2,\ldots ,u_p\,\}\,.$
4. IF you assume that the $u_i$ are orthomormal, whereas btw. orthogonal is sufficient, but you don't seem to make a difference, then you immediately have $\langle u_1\,|\,u_1\rangle=0$ which by the properties of the inner product means $u_1=0$ and thus $\dim V_1 < p$ which is the contradiction you need.
5. To summerize: You need to say why you can use the first vector as linear dependent. Then you need to say, why you can assume the $u_i$ be orthogonal, resp. orthonormal which you used. This automatically brings you into problems. Since you first renumbered the basis, will this have an effect on the orthonormalization procedure in Gram-Schmidt? As Gram-Schmidt, as far as I remember, also uses renumbering, both of these could be in conflict and you haven't said why it is none! The trick here is - in case you want to use Gram-Schmidt - to do it first. The first thing in the entire proof should be that you require both basis to be orthonormal. Both basis, as you cannot know at prior whether $V_1$ or $V_2$ causes the defect in dimension. Selecting $u_1$ makes this choice, so that it has to come after the Gram-Schmidt argument. Any other order will make your proof invalid, except you find a different way to deal with those obstacles. Nevertheless, the proof can also be given without the usage of Gram-Schmidt, but then you will not be able to cut out single coefficients by taking the inner product.

Now consider $c_1 u_1 + c_2u_2 +... +c_p u_p$ of L.H.S. of (8). As $u_1, u_2 , u_3 ,... u_p$ are basis vectors of $V_1$, these are linearly independent.
So,
$| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ ...(5)
Taking dot product with $\langle u_1|$ gives,
$c_1 = - \langle u_1 | c_2u_2 +... +c_p u_p\rangle = 0$ ...(6) This is what I meant in the earlier calculation.
But the point is eqn (8) does not imply $c_1 u_1 + c_2u_2 +... +c_p u_p = 0$ So, eqn. (5) and (6) are not valid here. Now I see it. Thanks.

But, then I see only one way to prove. Using Gram Schmidt theorem ,I state that $V_1$ and $V_2$ has orthonormal basis. And then use this basis to prove as I did in post ## 10 and 11.

 

[Mark44]

@Pushoam, back in post #16 you wrote this:

Considering the following linear combination of basis vectors of $V_1$,
$| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ ...(5)

It's not clear to me exactly what PeroK's objection was, but it might be this idea:
Let's look at two sets of vectors:
A: $\{u_1 = <1, 0, 0>, u_2 = <0, 1, 0>, u_3 = <1, 1, 0>\}$ and
B: $\{v_1 = <1, 0, 0>, v_2 = <0, 1, 0>, v_3 = <0, 0, 1>\}$
For A, I write the equation $a_1u_1 + a_2u_2 + a_3u_3 = 0$ (1) , and note that $a_1 = a_2 = a_3 = 0$ is a solution. In other words,
$a_1u_1 + a_2u_2 + a_3u_3 = 0 \Rightarrow a_1 = a_2 = a_3 = 0$.
Can I conclude that the vectors $u_1, u_2, u_3$ are linearly independent? Clearly these vectors are linearly dependent, since $u_3 = u_1 + u_2$. Therefore equation 1 has another solution; namely, $a_1 = 1, a_2 = 1, a_3 = -1$. In fact, there are an infinite number of solutions to equation 1.

For B, I write the equation $b_1v_1 + b_2v_2 + b_3v_3 = 0$ (2), and after a bit of algebra determine that the unique solution for the constants is $b_1 = b_2 = b_3 = 0$, thus concluding that this set of vectors is linearly independent.

Maybe you get the distinction between a linearly independent set of vectors and a linearly dependent set, but it wasn't clear in your equation 5.

 

[Pushoam]

Next you used without mention $\langle u_1\,|\,u_i\rangle = \delta_{1i}\,$. Why are you allowed to do so? Nowhere up to now, i.e. in this post, is anything written about $u_i\perp u_j \quad (i\neq j)$ and the problem statement doesn't say it either! So until now, there is no argument, why $u_1$⊥span{$u_2,…,u_p$}.

Where did I use the above? Please mention this, too.

Thanks for the points.

 

[Pushoam]

@Pushoam, back in post #16 you wrote this:
Maybe you get the distinction between a linearly independent set of vectors and a linearly dependent set, but it wasn't clear in your equation 5.

Basis vectors are linearly independent and definition of linear independent vectors is given as:
$\Sigma_{i=1}^n c_i |u_i \rangle = 0$ means all the coefficients$\{c_i\} = 0.$
This is what I wrote in eqn. (5).
Then why do you say that definition of linearly independent vectors is not clear in eqn. (5)?

 

[Mark44]

Basis vectors are linearly independent and definition of linear independent vectors is given as:
$\Sigma_{i=1}^n c_i |u_i \rangle = 0$ means all the coefficients$\{c_i\} = 0.$
This is what I wrote in eqn. (5).

No, this is what you wrote, which I quoted exactly as you wrote it

Considering the following linear combination of basis vectors of $V_1$,
$| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ ...(5)

My example set A matches exactly in substance what you have here, yet my example is a linearly dependent set. For any set of vectors, if $c_1v_1 + \dots + c_nv_n = 0$, it's always true that $c_1 = c_2 = \dots = c_n = 0$ is a solution, whether the vectors are linearly dependent or linearly independent. The difference is that for linearly independent vectors, the trivial solution is the only solution.

Then why do you say that definition of linearly independent vectors is not clear in eqn. (5)?

See above.

 

[Pushoam]

Aren't $\Sigma_{i=1}^n c_i |u_i \rangle = 0$ means all the coefficients$\{c_i\} = 0.$ and $| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ two expressions for the same thing?
I had this understanding that both expressions define linear independence of vectors.

 

[fresh_42]

Where did I use the above? Please mention this, too.

Taking dot product with $\langle u_1|$ gives $c_1 = - \langle u_1 | c_2u_2 +... +c_p u_p\rangle = 0$ ...(6)

This is not correct without assuming $u_1 \perp u_i \quad (i>1)\text{ and } \langle u_1\,|\,u_1 \rangle=1\,.$ And there was nowhere an argument why it should be correct. And as I elaborated, it's even plain wrong if you apply Gram-Schmidt after the renumbering which leads to the choice of $u_1$, because Gram-Schmidt might also use a renumbering and you have to rule out that both of them doesn't conflict with the other. Gram-Schmidt makes a choice of a basis. After that, another choice must ensure to be allowed. The other way around it is not obvious, that Gram-Schmidt is applicable on a basis, which already has been specified (via numbering).

Aren't $\Sigma_{i=1}^n c_i |u_i \rangle = 0$ means all the coefficients$\{c_i\} = 0.$

Yes, but not for any vectors, only for linear independent ones. And you assumed $u_1$ to be linear dependent at one point.

... and $| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ two expressions for the same thing?

Yes.

I had this understanding that both expressions define linear independence of vectors.

Yes, that is the definition. But as you assumed $u_1 \in \operatorname{span}\{\,u_i,v_j\,\}$ things are not automatically obvious. Your mistake is, that you used the same coefficients $c_i$ to define linear independence, which by the way is no need to do, and next for the expression $u_1=\sum c_iu_i +\sum d_jv_j$ and that is always a bad idea to name two potentially different things by the same name. I wouldn't let you go with it.

 

[Mark44]

... and $| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0 \Rightarrow \{c_i\}= 0, i=1,2,...p$ two expressions for the same thing?

Yes.

I'm going to disagree here, if only because so many students don't grasp the difference between linear independence and linear dependence. The equation above is technically correct, but students commonly don't realize that the equation $| c_1 u_1 + c_2u_2 +... +c_p u_p \rangle =0$ always has at least one solution, whether or not the vectors are linearly independent.
For example, using my earlier example (set A), if we start with this equation, $a_1<1, 0, 0> + a_2<0, 1, 0> + a_3<1, 1, 0> = 0$, by simple substitution we see that there is a solution $a_1 = a_2 = a_3 = 0$. Can you conclude that the three vectors on the left side of this equation are linearly independent? I hope not.

This is why most linear algebra textbooks are careful to define linear independence by saying that there is only the trivial solution and no others.

 

[fresh_42]

Right. The essential part is $\Longrightarrow$ and often gets lost. The entire definition is practically within this arrow.

 

[PeroK]

It's not clear to me exactly what PeroK's objection was, but it might be this idea:

Here's an example to illustrate the logic problem:

Let $\{ u_i \}$ be a basis. Therefore:

$\Sigma c_i u_i = 0 \ \Rightarrow c_i = 0$

And, in particular, $c_1 = 0$ (Equation (1))

Let $u$ be any vector, with $u = \Sigma c_i u_i$

If $c_1 \ne 0$, then we have a contradiction to equation (1).

This sort of thing, in my experience, is a common mistake.

I think @Pushoam understands this now?

 

[Pushoam]

Here's an example to illustrate the logic problem:

Let $\{ u_i \}$ be a basis. Therefore:

$\Sigma c_i u_i = 0 \ \Rightarrow c_i = 0$

And, in particular, $c_1 = 0$ (Equation (1))

Let $u$ be any vector, with $u = \Sigma c_i u_i$

If $c_1 \ne 0$, then we have a contradiction to equation (1).

This sort of thing, in my experience, is a common mistake.

I think @Pushoam understands this now?

Thanks for pointing it out. I shall be careful regarding notation as it might become hard to see notational mistake later.

 

[fresh_42]

Thanks for pointing it out. I shall be careful regarding notation as it might become hard to see notational mistake later.

Thou shalt not use the same letter at different occasions!

 

[Pushoam]

Thanks @ fresh_42.