# Cauchy Schwarz equality implies parallel

I'm learning about Support Vector Machines and would like to recap on some basic linear algebra. More specifically, I'm trying to prove the following, which I'm pretty sure is true:
Let ##v1## and ##v2## be two vectors in an inner product space over ##\mathbb{C}##.
Suppose that ## \langle v1 , v2 \rangle = ||v1|| \cdot ||v2|| ##, i.e. the special case of Cauchy Schwarz when it is an equality. Then prove that ##v1## is a scalar multiple of ##v2##, assuming neither vector is ##0##.

I've tried using the triangle inequality and some other random stuff to no avail. I believe there's some algebra trick involved, could someone help me out? I really want to prove this and get on with my machine learning.

Thanks!

BiP

fresh_42
Mentor
How <v1,v2> is defined?

Proving this should not require the definition of the inner product, only the properties.

fresh_42
Mentor
What's the difference? Which properties do you mean?

Conjugate symmetry, linearity in the first argument, and positive-definiteness.

fresh_42
Mentor
Looks to me as another version of the cosine formula if applied to v1+v2

By definition, $\langle v_1, v_2 \rangle = \| v_1 \| \cdot \| v_2 \| \cdot \cos(\theta)$ where $\theta$ is the angle between vectors $v_1$ and $v_2$. If you also additionally know that $\langle v_1, v_2 \rangle = \| v_1 \| \cdot \| v_2 \|$, then the angle between the two vectors must either be 0 or 180 degrees. So they are parallel; hence one is a scalar multiple of the other.

That's the definition? It would be true in a real inner product space, but this one is over ℂ.

rs1n
That's the definition? It would be true in a real inner product space, but this one is over ℂ.

You are absolutely right! My eyes failed me, somehow.

PeroK
Homework Helper
Gold Member
2020 Award
I'm learning about Support Vector Machines and would like to recap on some basic linear algebra. More specifically, I'm trying to prove the following, which I'm pretty sure is true:
Let ##v1## and ##v2## be two vectors in an inner product space over ##\mathbb{C}##.
Suppose that ## \langle v1 , v2 \rangle = ||v1|| \cdot ||v2|| ##, i.e. the special case of Cauchy Schwarz when it is an equality. Then prove that ##v1## is a scalar multiple of ##v2##, assuming neither vector is ##0##.

I've tried using the triangle inequality and some other random stuff to no avail. I believe there's some algebra trick involved, could someone help me out? I really want to prove this and get on with my machine learning.

Thanks!

BiP

One way to do it is to consider the vector ##u = v_2 - \frac{<v1, v2>}{<v1, v1>} v_1##

Look at ##<u, u>## and show that it's zero when you have C-S equality. This also leads to a proof of the C-S inequality.

To get back to the problem, though... over the complex numbers, the inner product is presumably a Hermitian inner product. So

##\begin{align*}
\| u + v \|^2 & = \langle u + v, u+v \rangle = \langle u,u \rangle + \langle u,v \rangle + \langle v,u \rangle + \langle v, v \rangle\\
& = \langle u,u \rangle + \langle u,v \rangle + \overline{\langle u,v \rangle} + \langle v, v \rangle \\
& = \langle u,u \rangle + 2 \mathrm{Re}(\langle u,v \rangle) + \langle v, v \rangle\\
& = \| u\|^2 + 2 \mathrm{Re}(\langle u,v \rangle) + \| v\|^2
\end{align*}##

Similarly,

## 0 \le \| u + \lambda v \|^2 = \| u\|^2 + 2 \mathrm{Re}(\overline{\lambda} \langle u,v \rangle) + |\lambda|^2 \| v\|^2##

Let $$\lambda = -\frac{\langle u, v\rangle }{\|v \|^2}$$ and the right hand side (above) will simplify to the C.S. inequality. Equality occurs if $$\| u + \lambda v \| = 0$$

There are few possible ways of doing that. The first one is just to follow the proof of the Cauchy--Schwarz. Namely, for real ##t## consider $$\|v_1 - t v_2\|^2 = \|v_1\|^2 +t^2\|v_2\|^2 - 2t (v_1, v_2) = \|v_1\|^2 +t^2\|v_2\|^2 - 2t \|v_1\|\cdot \|v_2\| = (\|v_1\|-t\|v_2\|)^2.$$ The right hand side of this chain of equations is ##0## when ##t=\|v_1\|/\|v_2\|##. So for this ##t## you get that ##v_1-tv_2=0##, which is exactly what you need.

Another way is more geometric and probably more intuitive. You define ##w## to be the orthogonal projection of ##v_2## onto the one dimensional subspace spanned by ##v_1##, ##w= \|v_1\|^{-2} (v_2, v_1) v_1##. Then ##(v_1, v_2)= (v_1, w)## (checked by direct calculation) and ##v_2-w## is orthogonal to ##v_1## (and so to ##w##).
Therefore ##\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2##.

By Cauchy--Schwarz ## (v_1, w) \le \|v_1\|\cdot \|w\|##, but on the other hand ##(v_1, w) = (v_1, v2) = \|v_1\|\cdot \|v_2\|##, so ##\|v_1\|\cdot \|v_2\| \le \|v_1\|\cdot \|w\|## and therefore ##\|v_2\|\le \|w\|##. Comparing this with ##\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2## we conclude that ##v_2-w=0##.

The second proof is a bit longer, but it is more intuitive, in a sense that it is a pretty standard reasoning used when one works with orthogonal projections.

PeroK
Homework Helper
Gold Member
2020 Award
Another way is more geometric and probably more intuitive. You define ##w## to be the orthogonal projection of ##v_2## onto the one dimensional subspace spanned by ##v_1##, ##w= \|v_1\|^{-2} (v_2, v_1) v_1##. Then ##(v_1, v_2)= (v_1, w)## (checked by direct calculation) and ##v_2-w## is orthogonal to ##v_1## (and so to ##w##).
Therefore ##\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2##.

By Cauchy--Schwarz ## (v_1, w) \le \|v_1\|\cdot \|w\|##, but on the other hand ##(v_1, w) = (v_1, v2) = \|v_1\|\cdot \|v_2\|##, so ##\|v_1\|\cdot \|v_2\| \le \|v_1\|\cdot \|w\|## and therefore ##\|v_2\|\le \|w\|##. Comparing this with ##\|v_2\|^2 =\|w\|^2+\|v_2-w\|^2## we conclude that ##v_2-w=0##.

The second proof is a bit longer, but it is more intuitive, in a sense that it is a pretty standard reasoning used when one works with orthogonal projections.

The second method is what I suggested in post #10. And, in fact, you can prove Cauchy Schwartz more intuitively this way.

I see! Thank you all for your replies! I knew I had seen it somewhere, little did I know it was right there in the proof of the C-S inequality itself!

BiP