Optimal approximation and unique representation in Euclidean spaces

[mathmari]

Hey!

I am looking at the following that is related to the existence of the optimal approximation.

$H$ is an euclidean space
$\widetilde{H}$ is a subspace of $H$

We suppose that $dim \widetilde{H}=n$ and $\{x_1,x_2,...,x_n\}$ is the basis of $\widetilde{H}$.

Let $y \in \widetilde{H}$ be the optimal approximation of $x \in H$ from $\widetilde{H}$.
Then $(y,u)=(x,u), \forall u \in \widetilde{H}$.

We take $u=x_i \in \widetilde{H}$, so $(y,x_i)=(x,x_i)$

Since $\{x_1,x_2,...,x_n\}$ is the basis of $\widetilde{H}$, $y$ can be written as followed:
$y=a_1 x_1 + a_2 x_2 +... + a_n x_n$

$\left.\begin{matrix}
(x,x_1)=(y,x_1)=a_1 (x_1,x_1)+a_2 (x_2,x_1)+...+a_n (x_n,x_1)\\
(x,x_2)=(y,x_2)=a_1 (x_1,x_2)+a_2 (x_2,x_2)+...+a_n (x_n,x_2)\\
...\\
(x,x_n)=(y,x_n)=a_1 (x_1,x_n)+a_2 (x_2,x_n)+...+a_n (x_n,x_n)
\end{matrix}\right\}(1)$

So that the optimal approximation exists, I have to be able to write $y$ in an unique way as linear combination of the elements of the basis.

The system $(1)$ has class $n$, since the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$.
So the system has a unique solution.>Why does the optimal approximation only exists when $y$ can be written in an unique way as linear combination of the elements of the basis?

>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?

 

[I like Serena]

Hey! (Blush)

>Why does the optimal approximation only exists when $y$ can be written in an unique way as linear combination of the elements of the basis?

In a linear (sub)space every vector can be written as a unique linear combination of basis vectors.
So if $y$ can be written as 2 different linear combinations, those are really different vectors. In other words: $y$ is not a unique vector, so you cannot call it "the" optimal approximation.

>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?

I'm not aware of a concept named class as related to a system of linear equations. Googling for it gave indeed no hits. As I see it, it is ambiguous in this context. It can either mean $n$ equations or $n$ variables. Luckily, in this particular case it is both. :)

 

[Opalg]

>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?

I have not come across the term "class" in that context. My guess is that what it means is that the matrix of coefficients in the system (1) has rank $n$. That implies that the equations have a unique solution, which is what is wanted here.

 

[mathmari]

In a linear (sub)space every vector can be written as a unique linear combination of basis vectors.
So if $y$ can be written as 2 different linear combinations, those are really different vectors. In other words: $y$ is not a unique vector, so you cannot call it "the" optimal approximation.

A ok! So if $y$ can be written as 2 different linear combinations, that means that there are 2 different approximations, so we do not have the one that is optimal.
I got it!

I'm not aware of a concept named class as related to a system of linear equations. Googling for it gave indeed no hits. As I see it, it is ambiguous in this context. It can either mean $n$ equations or $n$ variables. Luckily, in this particular case it is both. :)

I have not come across the term "class" in that context. My guess is that what it means is that the matrix of coefficients in the system (1) has rank $n$. That implies that the equations have a unique solution, which is what is wanted here.

Aha! Ok!

The system $(1)$ has class $n$, since the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$.

Why do we conclude to that the class of the system is $n$ from the fact that the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$?

 

[Opalg]

Why do we conclude to that the class of the system is $n$ from the fact that the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$?

Good question! We know that $\dim\widetilde H = n$, so the condition for the set $\{x_1, ..., x_n \}$ to be a basis is that it should be linearly independent. Or, to put it negatively, the set will fail to be a basis if and only if it is linearly dependent. That in turn is equivalent to the condition that there should exist scalars $\lambda_1,\ldots,\lambda_n$, not all $0$, such that $\sum \lambda_ix_i = 0.$ But then $\sum \lambda_i\langle x_i,x_j \rangle = 0$ for all $j$. That says that the rows of the matrix $A = (\langle x_i,x_j \rangle)$ are linearly dependent, which means that the rank of $A$ is less than $n$.

Conversely, if the rank of $A$ is less than $n$, then its rows are linearly dependent. So there exist scalars $\lambda_1,\ldots,\lambda_n$, not all $0$, such that $\sum \lambda_i\langle x_i,x_j \rangle = 0$ for all $j$. This says that $\sum \lambda_ix_i$ is orthogonal to each $x_j$. Since the $x_j$ form a basis, it follows that $\sum \lambda_ix_i = 0$ and so $\{x_1, ..., x_n \}$ is not a basis for $\widetilde H$.