Cauchy Schwarz equality implies parallel

[Bipolarity]

I'm learning about Support Vector Machines and would like to recap on some basic linear algebra. More specifically, I'm trying to prove the following, which I'm pretty sure is true:
Let $v1$ and $v2$ be two vectors in an inner product space over $\mathbb{C}$.
Suppose that $\langle v1 , v2 \rangle = ||v1|| \cdot ||v2||$, i.e. the special case of Cauchy Schwarz when it is an equality. Then prove that $v1$ is a scalar multiple of $v2$, assuming neither vector is $0$.

I've tried using the triangle inequality and some other random stuff to no avail. I believe there's some algebra trick involved, could someone help me out? I really want to prove this and get on with my machine learning.

Thanks!

BiP

 

[fresh_42]

How <v1,v2> is defined?

 

[Bipolarity]

Proving this should not require the definition of the inner product, only the properties.

 

[fresh_42]

What's the difference? Which properties do you mean?

 

[Bipolarity]

Conjugate symmetry, linearity in the first argument, and positive-definiteness.

 

[fresh_42]

Looks to me as another version of the cosine formula if applied to v1+v2

 

[rs1n]

By definition, \langle v_1, v_2 \rangle = \| v_1 \| \cdot \| v_2 \| \cdot \cos(\theta) where \theta is the angle between vectors v_1 and v_2. If you also additionally know that \langle v_1, v_2 \rangle = \| v_1 \| \cdot \| v_2 \|, then the angle between the two vectors must either be 0 or 180 degrees. So they are parallel; hence one is a scalar multiple of the other.

 

[zinq]

That's the definition? It would be true in a real inner product space, but this one is over ℂ.

 

[rs1n]

That's the definition? It would be true in a real inner product space, but this one is over ℂ.

You are absolutely right! My eyes failed me, somehow.

 

[PeroK]

I'm learning about Support Vector Machines and would like to recap on some basic linear algebra. More specifically, I'm trying to prove the following, which I'm pretty sure is true:
Let $v1$ and $v2$ be two vectors in an inner product space over $\mathbb{C}$.
Suppose that $\langle v1 , v2 \rangle = ||v1|| \cdot ||v2||$, i.e. the special case of Cauchy Schwarz when it is an equality. Then prove that $v1$ is a scalar multiple of $v2$, assuming neither vector is $0$.

I've tried using the triangle inequality and some other random stuff to no avail. I believe there's some algebra trick involved, could someone help me out? I really want to prove this and get on with my machine learning.

Thanks!

BiP

One way to do it is to consider the vector $u = v_2 - \frac{<v1, v2>}{<v1, v1>} v_1$

Look at $<u, u>$ and show that it's zero when you have C-S equality. This also leads to a proof of the C-S inequality.

 

[rs1n]

To get back to the problem, though... over the complex numbers, the inner product is presumably a Hermitian inner product. So

$\begin{align*} \| u + v \|^2 & = \langle u + v, u+v \rangle = \langle u,u \rangle + \langle u,v \rangle + \langle v,u \rangle + \langle v, v \rangle\\ & = \langle u,u \rangle + \langle u,v \rangle + \overline{\langle u,v \rangle} + \langle v, v \rangle \\ & = \langle u,u \rangle + 2 \mathrm{Re}(\langle u,v \rangle) + \langle v, v \rangle\\ & = \| u\|^2 + 2 \mathrm{Re}(\langle u,v \rangle) + \| v\|^2 \end{align*}$

Similarly,

$0 \le \| u + \lambda v \|^2 = \| u\|^2 + 2 \mathrm{Re}(\overline{\lambda} \langle u,v \rangle) + |\lambda|^2 \| v\|^2$

Let $$\lambda = -\frac{\langle u, v\rangle }{\|v \|^2}$$ and the right hand side (above) will simplify to the C.S. inequality. Equality occurs if $$\| u + \lambda v \| = 0$$

 

[Hawkeye18]

There are few possible ways of doing that. The first one is just to follow the proof of the Cauchy--Schwarz. Namely, for real $t$ consider $$\|v_1 - t v_2\|^2 = \|v_1\|^2 +t^2\|v_2\|^2 - 2t (v_1, v_2) = \|v_1\|^2 +t^2\|v_2\|^2 - 2t \|v_1\|\cdot \|v_2\| = (\|v_1\|-t\|v_2\|)^2.$$ The right hand side of this chain of equations is $0$ when $t=\|v_1\|/\|v_2\|$. So for this $t$ you get that $v_1-tv_2=0$, which is exactly what you need.

Another way is more geometric and probably more intuitive. You define $w$ to be the orthogonal projection of $v_2$ onto the one dimensional subspace spanned by $v_1$, $w= \|v_1\|^{-2} (v_2, v_1) v_1$. Then $(v_1, v_2)= (v_1, w)$ (checked by direct calculation) and $v_2-w$ is orthogonal to $v_1$ (and so to $w$).
Therefore $\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2$.

By Cauchy--Schwarz $(v_1, w) \le \|v_1\|\cdot \|w\|$, but on the other hand $(v_1, w) = (v_1, v2) = \|v_1\|\cdot \|v_2\|$, so $\|v_1\|\cdot \|v_2\| \le \|v_1\|\cdot \|w\|$ and therefore $\|v_2\|\le \|w\|$. Comparing this with $\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2$ we conclude that $v_2-w=0$.

The second proof is a bit longer, but it is more intuitive, in a sense that it is a pretty standard reasoning used when one works with orthogonal projections.

 

[PeroK]

Another way is more geometric and probably more intuitive. You define $w$ to be the orthogonal projection of $v_2$ onto the one dimensional subspace spanned by $v_1$, $w= \|v_1\|^{-2} (v_2, v_1) v_1$. Then $(v_1, v_2)= (v_1, w)$ (checked by direct calculation) and $v_2-w$ is orthogonal to $v_1$ (and so to $w$).
Therefore $\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2$.

By Cauchy--Schwarz $(v_1, w) \le \|v_1\|\cdot \|w\|$, but on the other hand $(v_1, w) = (v_1, v2) = \|v_1\|\cdot \|v_2\|$, so $\|v_1\|\cdot \|v_2\| \le \|v_1\|\cdot \|w\|$ and therefore $\|v_2\|\le \|w\|$. Comparing this with $\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2$ we conclude that $v_2-w=0$.

The second proof is a bit longer, but it is more intuitive, in a sense that it is a pretty standard reasoning used when one works with orthogonal projections.

The second method is what I suggested in post #10. And, in fact, you can prove Cauchy Schwartz more intuitively this way.

 

[Bipolarity]

I see! Thank you all for your replies! I knew I had seen it somewhere, little did I know it was right there in the proof of the C-S inequality itself!

BiP