MHB Some questions about the existence of the optimal approximation

AI Thread Summary
The discussion centers on the existence of an optimal approximation in a Euclidean space, specifically within a subspace defined by a basis. It emphasizes that for an optimal approximation \( y \) of a vector \( x \) from the subspace \( \widetilde{H} \) to exist uniquely, \( y \) must be expressible as a unique linear combination of the basis vectors. The concept of "class" in the context of the linear system is debated, with participants suggesting it refers to having \( n \) equations and \( n \) unknowns, leading to a unique solution. The linear independence of the basis vectors is crucial, as any dependence would imply multiple representations of \( y \), negating its status as the optimal approximation. The conclusion drawn is that the rank of the coefficient matrix must equal \( n \) to ensure the basis is valid and the approximation is unique.
mathmari
Gold Member
MHB
Messages
4,984
Reaction score
7
Hey! :o

I am looking at the following that is related to the existence of the optimal approximation.

$H$ is an euclidean space
$\widetilde{H}$ is a subspace of $H$

We suppose that $dim \widetilde{H}=n$ and $\{x_1,x_2,...,x_n\}$ is the basis of $\widetilde{H}$.

Let $y \in \widetilde{H}$ be the optimal approximation of $x \in H$ from $\widetilde{H}$.
Then $(y,u)=(x,u), \forall u \in \widetilde{H}$.

We take $u=x_i \in \widetilde{H}$, so $(y,x_i)=(x,x_i)$

Since $\{x_1,x_2,...,x_n\}$ is the basis of $\widetilde{H}$, $y$ can be written as followed:
$y=a_1 x_1 + a_2 x_2 +... + a_n x_n$

$\left.\begin{matrix}
(x,x_1)=(y,x_1)=a_1 (x_1,x_1)+a_2 (x_2,x_1)+...+a_n (x_n,x_1)\\
(x,x_2)=(y,x_2)=a_1 (x_1,x_2)+a_2 (x_2,x_2)+...+a_n (x_n,x_2)\\
...\\
(x,x_n)=(y,x_n)=a_1 (x_1,x_n)+a_2 (x_2,x_n)+...+a_n (x_n,x_n)
\end{matrix}\right\}(1)$

So that the optimal approximation exists, I have to be able to write $y$ in an unique way as linear combination of the elements of the basis.

The system $(1)$ has class $n$, since the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$.
So the system has a unique solution.>Why does the optimal approximation only exists when $y$ can be written in an unique way as linear combination of the elements of the basis?

>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?
 
Mathematics news on Phys.org
Hey! (Blush)

>Why does the optimal approximation only exists when $y$ can be written in an unique way as linear combination of the elements of the basis?

In a linear (sub)space every vector can be written as a unique linear combination of basis vectors.
So if $y$ can be written as 2 different linear combinations, those are really different vectors. In other words: $y$ is not a unique vector, so you cannot call it "the" optimal approximation.
>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?

I'm not aware of a concept named class as related to a system of linear equations. Googling for it gave indeed no hits. As I see it, it is ambiguous in this context. It can either mean $n$ equations or $n$ variables. Luckily, in this particular case it is both. :)
 
mathmari said:
>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?
I have not come across the term "class" in that context. My guess is that what it means is that the matrix of coefficients in the system (1) has rank $n$. That implies that the equations have a unique solution, which is what is wanted here.
 
I like Serena said:
In a linear (sub)space every vector can be written as a unique linear combination of basis vectors.
So if $y$ can be written as 2 different linear combinations, those are really different vectors. In other words: $y$ is not a unique vector, so you cannot call it "the" optimal approximation.

A ok! So if $y$ can be written as 2 different linear combinations, that means that there are 2 different approximations, so we do not have the one that is optimal.
I got it!
I like Serena said:
I'm not aware of a concept named class as related to a system of linear equations. Googling for it gave indeed no hits. As I see it, it is ambiguous in this context. It can either mean $n$ equations or $n$ variables. Luckily, in this particular case it is both. :)

Opalg said:
I have not come across the term "class" in that context. My guess is that what it means is that the matrix of coefficients in the system (1) has rank $n$. That implies that the equations have a unique solution, which is what is wanted here.

Aha! Ok!

The system $(1)$ has class $n$, since the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$.
Why do we conclude to that the class of the system is $n$ from the fact that the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$?
 
mathmari said:
Why do we conclude to that the class of the system is $n$ from the fact that the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$?
Good question! We know that $\dim\widetilde H = n$, so the condition for the set $\{x_1, ..., x_n \}$ to be a basis is that it should be linearly independent. Or, to put it negatively, the set will fail to be a basis if and only if it is linearly dependent. That in turn is equivalent to the condition that there should exist scalars $\lambda_1,\ldots,\lambda_n$, not all $0$, such that $\sum \lambda_ix_i = 0.$ But then $\sum \lambda_i\langle x_i,x_j \rangle = 0$ for all $j$. That says that the rows of the matrix $A = (\langle x_i,x_j \rangle)$ are linearly dependent, which means that the rank of $A$ is less than $n$.

Conversely, if the rank of $A$ is less than $n$, then its rows are linearly dependent. So there exist scalars $\lambda_1,\ldots,\lambda_n$, not all $0$, such that $\sum \lambda_i\langle x_i,x_j \rangle = 0$ for all $j$. This says that $\sum \lambda_ix_i$ is orthogonal to each $x_j$. Since the $x_j$ form a basis, it follows that $\sum \lambda_ix_i = 0$ and so $\{x_1, ..., x_n \}$ is not a basis for $\widetilde H$.
 
Last edited:
Suppose ,instead of the usual x,y coordinate system with an I basis vector along the x -axis and a corresponding j basis vector along the y-axis we instead have a different pair of basis vectors ,call them e and f along their respective axes. I have seen that this is an important subject in maths My question is what physical applications does such a model apply to? I am asking here because I have devoted quite a lot of time in the past to understanding convectors and the dual...
Insights auto threads is broken atm, so I'm manually creating these for new Insight articles. In Dirac’s Principles of Quantum Mechanics published in 1930 he introduced a “convenient notation” he referred to as a “delta function” which he treated as a continuum analog to the discrete Kronecker delta. The Kronecker delta is simply the indexed components of the identity operator in matrix algebra Source: https://www.physicsforums.com/insights/what-exactly-is-diracs-delta-function/ by...

Similar threads

Replies
1
Views
626
Replies
1
Views
4K
Replies
17
Views
3K
Replies
9
Views
5K
Replies
21
Views
3K
Replies
0
Views
461
Replies
1
Views
2K
Back
Top