Question about algebraic transformation

[Peter_Newman]

Hello,

I would like to reproduce the following equation, but I don't quite understand how to do the transformation:

$$ \sum_{i=1}^k \left( \frac{\langle y , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \langle y, y \rangle$$

Where $x_1^*,...,x_k^*$, are orthogonalized Gram-Schmidt vectors of $x_1,...,x_k \in \Lambda$ and $y \in span(x_1^*,...,x_k^*) = span(x_1,...,x_k)$.

I would be very grateful for any helpful hints!

 

[pasmith]

I think this reduces to recognising that <br /> y = \sum_{i=1}^k \frac{\langle y, x_i^{*} \rangle}{ \langle x_i^{*}, x_i^{*} \rangle }x_i^{*} <br /> =\sum_{i=1}^k \frac{\langle y, x_i^{*} \rangle}{ \sqrt{\langle x_i^{*}, x_i^{*} \rangle } }\frac{x_i^{*}}{ \sqrt{\langle x_i^{*}, x_i^{*} \rangle } } and noting <br /> \left \langle \frac{x_i^{*}}{ \sqrt{\langle x_i^{*}, x_i^{*} \rangle } }, <br /> \frac{x_j^{*}}{ \sqrt{\langle x_j^{*}, x_j^{*} \rangle } } \right\rangle = \begin{cases} 1 &amp; i = j, \\ 0 &amp; i \neq j.\end{cases}

 

[PeroK]

If you express $y$ in the $\{x_i\}$ basis, then it should be clear.

Edit I meant $\{x_i^*\}$ basis. Apologies for the confusion caused.

 

[Peter_Newman]

Hello @pasmith thank you for your reply!

So if I understand the first part correctly, then that is in words $y$ expressed as its Gram-Schmidt "version"

---

Hello @PeroK thanks also for your answer. YES that was my approach too, but so right I don't see that. Let $y = \sum^k c_i x_i$ in its $x_i$ basis, but then I would have in the big sum something like:

$$\sum_{i=1}^k \left( \frac{\langle y , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{\langle \sum_{j=1}^k c_j x_j , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2$$
then I would use to reduce the sum in the second part this property $\langle x+y, z\rangle = \langle x,z \rangle + \langle y,z \rangle$, but that doesn't really seem to help me?

And from here I do not really see how we achive $\langle y,y\rangle$. It would be nice if you can help me further.

 

[pasmith]

Set y = \sum_{i=1}^k a_ix_i^{*}. The point is that as the x_i^{*} are orthogonal, <br /> \langle y, x_j^{*} \rangle = \sum_{i=1}^k a_i \langle x_i^{*}, x_j^{*} \rangle<br /> = a_j \langle x_j^{*}, x_j^{*} \rangle.

 

[PeroK]

You use the orthogonality of the $x_i$. Note that if use the normalised, orthonormal version of the basis then the result is trivial.

 

[PeroK]

PS that gives another possible approach.

 

[Peter_Newman]

Set y = \sum_{i=1}^k a_ix_i^{*}. The point is that as the x_i^{*} are orthogonal, <br /> \langle y, x_j^{*} \rangle = \sum_{i=1}^k a_i \langle x_i^{*}, x_j^{*} \rangle<br /> = a_j \langle x_j^{*}, x_j^{*} \rangle.

@pasmith I agree with that (and great hint!), but let's go some steps further. If we take this result into the big sum we would have (and recap that we have $\langle y, x_i^{*} \rangle$)

$$\sum_{i=1}^k \left( \frac{\langle y , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{a_i \langle x_i^{*}, x_i^{*} \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( a_i^2 \langle x_i^{*}, x_i^{*} \rangle \right) = \sum_{i=1}^k \left( a_i^2 {x_i^{*}}^2\right) = ||y||^2$$
Right?

---

@PeroK how would you do this with your recommendation using $y$ in the $x_i$ basis?

 

[fresh_42]

@pasmith I agree with that (and great hint!), but let's go some steps further. If we take this result into the big sum we would have (and recap that we have $\langle y, x_i^{*} \rangle$)

$$\sum_{i=1}^k \left( \frac{\langle y , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{a_i \langle x_i^{*}, x_i^{*} \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( a_i^2 \langle x_i^{*}, x_i^{*} \rangle \right) = \sum_{i=1}^k \left( a_i^2 {x_i^{*}}^2\right) = ||y||^2$$
Right?

---

@PeroK how would you do this with your recommendation using $y$ in the $x_i$ basis?

How much time are you willing to invest?

Here is an interesting read about the geometry behind those products:
https://arxiv.org/pdf/1205.5935.pdf

 

[PeroK]

---

@PeroK how would you do this with your recommendation using $y$ in the $x_i$ basis?

Perhaps this should be posted as homework!

 

[Peter_Newman]

Thanks for all the help so far! That is very nice. I currently still have two open points:

1. Is what I have done in post #8 so correct? Can I leave it as it is?

2. I would like to know how to do the representation of $y$ to the base $x_i$. I have already started to do this, see post #4, but I do not really get further. @PeroK it would be nice if you could help me here maybe again, because as simple as you say, I do not find that.

My approach:
Let
$$ y = \sum_{j=1}^k c_j x_j$$
Then
$$ \sum_{i=1}^k \left( \frac{\langle y , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{\langle \sum_{j=1}^k c_j x_j , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{\sum_{j=1}^k c_j \langle x_j , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 $$
But I'am not able to simplify the last expression... I think the idea is to simplify or rewrite $\sum_{j=1}^k c_j \langle x_j , x_i^* \rangle$ but I don't see it...

 

[Hall]

Thanks for all the help so far! That is very nice. I currently still have two open points:

1. Is what I have done in post #8 so correct? Can I leave it as it is?

2. I would like to know how to do the representation of $y$ to the base $x_i$. I have already started to do this, see post #4, but I do not really get further. @PeroK it would be nice if you could help me here maybe again, because as simple as you say, I do not find that.

My approach:
Let
$$ y = \sum_{j=1}^k c_j x_j$$
Then
$$ \sum_{i=1}^k \left( \frac{\langle y , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{\langle \sum_{j=1}^k c_j x_j , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{\sum_{j=1}^k c_j \langle x_j , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 $$
But I'am not able to simplify the last expression... I think the idea is to simplify or rewrite $\sum_{j=1}^k c_j \langle x_j , x_i^* \rangle$ but I don't see it...

Due to policies of the forum I cannot very directly help you, but why, my friend, are you not simplifying that
$\sqrt{ \langle x_i^*, x_i^*\rangle}$ to the norm of $x_i^*$?

And didn’t you yourself write in the original post that $$y \in span\{x_1^*, \cdots, x_k^*\}$$? But you equated $y$ to a span of $x_1, \cdots, x_k$ in your last post.

 

[PeroK]

The $x_i$ are orthogonal. What does that mean when $i \ne j$?

 

[Peter_Newman]

@Hall you mean $\sqrt{ \langle x_i^*, x_i^*\rangle} = ||x_i^*||$ right? And with the idea that $y \in span\{x_1^*, \cdots, x_k^*\}$ I would have suggested that the simplification is then this, what I did in post #8 (but for this I have no final verification from someone here, but for me it looks good). But is this what you mean?:

With:
$$y = \sum_{j=1}^k a_j x_j^*$$

$$ \sum_{i=1}^k \left( \frac{\langle y , x_i^* \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{a_i \langle x_i^{*}, x_i^{*} \rangle}{\sqrt{\langle x_i^*, x_i^* \rangle}} \right)^2 = \sum_{i=1}^k \left( \frac{a_i^2 ||x_i^{*}||^4}{||x_i^*||^2} \right) = \sum_{i=1}^k \left( a_i^2 ||x_i^{*}||^2 \right) = \sum_{i=1}^k \left( ||a_i {x_i^{*}}||^2\right) = ||y||^2 $$

(I'am not sure if the simplification with the norm is correct in the last step(s))

---

@PeroK in the case of $i \ne j$ I would say, that $\langle x_i, x_j \rangle = 0$. But from what comes your assumption that the $x_i$ are orthohonal (that the $x_i^*$'s are pairwise orthogonal is clear because of the Gram Schmidt process)?

 

[Hall]

(I'am not sure if the simplification with the norm is correct in the last step(s))

It is as correct as it can be.

 

[Peter_Newman]

@Hall thank you for your verification that is very good!

---

Now the only missing part is to show the same but expressing $y$ in the $x_i$ basis. I am curious to see what happens next here.

 

[PeroK]

@PeroK in the case of $i \ne j$ I would say, that $\langle x_i, x_j \rangle = 0$. But from what comes your assumption that the $x_i$ are orthohonal (that the $x_i^*$'s are pairwise orthogonal is clear because of the Gram Schmidt process)?

I got the assumption from your original post:

Where $x_1^*,...,x_k^*$, are orthogonalized Gram-Schmidt vectors of $x_1,...,x_k \in \Lambda$ and $y \in span(x_1^*,...,x_k^*) = span(x_1,...,x_k)$.

 

[Peter_Newman]

Hello @PeroK thank you for your reply. I'm actually still interested in a solution via your approach, and would like to work that out (with your help).

The $x_i^*$ are orthogonal to each other that is clear, since these arise from the linearly independent $x_i$'s by the Gram Schmidt procedure. But that does not mean that the $x_i$'s are also orthogonal? With the Gram Schmidt method it is enough that the input vectors are linearly independent, the goal is the orthogonal basis.

 

[Peter_Newman]

A little example let $x_1= (1,0)^T, x_2 = (1,2)^T$ they are linear independent and span the space and let $x_1^* = (1,0)^T , x_2^* = (0,1)^T$ they are linear independent and orthogonal due to the Gram Schmidt process and span $\mathbb{R}^2$. But what I mean is, and that's something I don't quite understand, why one can say that the $x_i$'s are orthogonal.

 

[fresh_42]

A little example let $x_1= (1,0)^T, x_2 = (1,2)^T$ they are linear independent and span the space and let $x_1^* = (1,0)^T , x_2^* = (0,1)^T$ they are linear independent and orthogonal due to the Gram Schmidt process and span $\mathbb{R}^2$. But what I mean is, and that's something I don't quite understand, why one can say that the $x_i$'s are orthogonal.

From that we get $\vec{a}\cdot\vec{b}=|\vec{a}|\cdot|\vec{b}|\cdot\cos\left(\sphericalangle\left(\vec{a},\vec{b}\right)\right).$

 

[Peter_Newman]

@fresh_42 nice plot! But I don't know what you want to say with this... (I know this plot and also the dot product but, I can not relate it to my problem above)

 

[fresh_42]

@fresh_42 nice plot! But I don't know what you want to say with this... (I know this plot and also the dot product but, I can not relate it to my problem obove)

The cosine gets $0$ if and only if the angle gets $90°.$ So the dot product can be used to "detect" right angles. Your question reduces to prove
$$
\sum_{k=1}^na_k\cdot b_k= \sqrt{\sum_{k=1}^n a_k^2}\cdot \sqrt{\sum_{k=1}^nb_k^2}\cdot \cos (\sphericalangle (a_k,b_k))
$$
or
$$
\cos (\sphericalangle (a_k,b_k))=\dfrac{\sum_{k=1}^na_k\cdot b_k}{\left(\sum_{k=1}^na_k^2\right)\left(\sum_{k=1}^nb_k^2\right)}
$$
This is basically the definition of the cosine and the image above illustrates it.

 

[Peter_Newman]

@fresh_42 ok, I understand what you want so say. But I don't know how this exactly helps If I express $y$ in the following way $y = \sum_{i=1}^k a_ix_i$ and then trying to reduce/simplify this $\langle y, x_j^{*} \rangle = \sum_{i=1}^k a_i \langle x_i, x_j^{*} \rangle$ because I don't know the angle between the $x_i$'s and the $x_j^*$. The great thing was if one expresses $y$ in $x_i^*$ basis that one can directly see the orthogonality, but with $y$ in the $x_i$ basis this is much more complex.

 

[fresh_42]

@fresh_42 ok, I understand what you want so say. But I don't know how this exactly helps If I express $y$ in the following way $y = \sum_{i=1}^k a_ix_i$ and then trying to reduce/simplify this $\langle y, x_j^{*} \rangle = \sum_{i=1}^k a_i \langle x_i, x_j^{*} \rangle$ because I don't know the angle between the $x_i$'s and the $x_j^*$. The great thing was if one expresses $y$ in $x_i^*$ basis that one can directly see the orthogonality, but with $y$ in the $x_i$ basis this is much more complex.

You need to write $y = \sum_{i=1}^k a_ix_i^*$ since otherwise, you have to deal with an additional basis transformation $x_k=\sum_{i=1}^k b_ix_i^*$ which complicates the issue unnecessarily. The $\{x_i^*\}$ is a ONB, so why not use it?

 

[Peter_Newman]

I am absolutely d'accord with you. Yes of course you can express the $y$ using the $x_i^*$. This is also the way, which is absolutely plausible (I have shown this here #14, @Hall was so friendly to confirm this ). But @PeroK had here #3 suggested that this also goes if one expresses $y$ in such a way with the $x_i$'s (the way is more difficult that's true, but that interests me )

If you express $y$ in the $\{x_i\}$ basis, then it should be clear.

But I think, this is not so "clear".

 

[PeroK]

Apologies for the confusion. It somehow didn't register that $x_i$ is the original basis and the orthogonalised basis has a $^*$. I can't explain how I missed that!