MHB Some questions about the existence of the optimal approximation

AI Thread Summary
The discussion centers on the existence of an optimal approximation in a Euclidean space, specifically within a subspace defined by a basis. It emphasizes that for an optimal approximation \( y \) of a vector \( x \) from the subspace \( \widetilde{H} \) to exist uniquely, \( y \) must be expressible as a unique linear combination of the basis vectors. The concept of "class" in the context of the linear system is debated, with participants suggesting it refers to having \( n \) equations and \( n \) unknowns, leading to a unique solution. The linear independence of the basis vectors is crucial, as any dependence would imply multiple representations of \( y \), negating its status as the optimal approximation. The conclusion drawn is that the rank of the coefficient matrix must equal \( n \) to ensure the basis is valid and the approximation is unique.
mathmari
Gold Member
MHB
Messages
4,984
Reaction score
7
Hey! :o

I am looking at the following that is related to the existence of the optimal approximation.

$H$ is an euclidean space
$\widetilde{H}$ is a subspace of $H$

We suppose that $dim \widetilde{H}=n$ and $\{x_1,x_2,...,x_n\}$ is the basis of $\widetilde{H}$.

Let $y \in \widetilde{H}$ be the optimal approximation of $x \in H$ from $\widetilde{H}$.
Then $(y,u)=(x,u), \forall u \in \widetilde{H}$.

We take $u=x_i \in \widetilde{H}$, so $(y,x_i)=(x,x_i)$

Since $\{x_1,x_2,...,x_n\}$ is the basis of $\widetilde{H}$, $y$ can be written as followed:
$y=a_1 x_1 + a_2 x_2 +... + a_n x_n$

$\left.\begin{matrix}
(x,x_1)=(y,x_1)=a_1 (x_1,x_1)+a_2 (x_2,x_1)+...+a_n (x_n,x_1)\\
(x,x_2)=(y,x_2)=a_1 (x_1,x_2)+a_2 (x_2,x_2)+...+a_n (x_n,x_2)\\
...\\
(x,x_n)=(y,x_n)=a_1 (x_1,x_n)+a_2 (x_2,x_n)+...+a_n (x_n,x_n)
\end{matrix}\right\}(1)$

So that the optimal approximation exists, I have to be able to write $y$ in an unique way as linear combination of the elements of the basis.

The system $(1)$ has class $n$, since the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$.
So the system has a unique solution.>Why does the optimal approximation only exists when $y$ can be written in an unique way as linear combination of the elements of the basis?

>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?
 
Mathematics news on Phys.org
Hey! (Blush)

>Why does the optimal approximation only exists when $y$ can be written in an unique way as linear combination of the elements of the basis?

In a linear (sub)space every vector can be written as a unique linear combination of basis vectors.
So if $y$ can be written as 2 different linear combinations, those are really different vectors. In other words: $y$ is not a unique vector, so you cannot call it "the" optimal approximation.
>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?

I'm not aware of a concept named class as related to a system of linear equations. Googling for it gave indeed no hits. As I see it, it is ambiguous in this context. It can either mean $n$ equations or $n$ variables. Luckily, in this particular case it is both. :)
 
mathmari said:
>What does it mean that the system $(1)$ has class $n$? That it has $n$ equations and $n$ unknown variabes?
I have not come across the term "class" in that context. My guess is that what it means is that the matrix of coefficients in the system (1) has rank $n$. That implies that the equations have a unique solution, which is what is wanted here.
 
I like Serena said:
In a linear (sub)space every vector can be written as a unique linear combination of basis vectors.
So if $y$ can be written as 2 different linear combinations, those are really different vectors. In other words: $y$ is not a unique vector, so you cannot call it "the" optimal approximation.

A ok! So if $y$ can be written as 2 different linear combinations, that means that there are 2 different approximations, so we do not have the one that is optimal.
I got it!
I like Serena said:
I'm not aware of a concept named class as related to a system of linear equations. Googling for it gave indeed no hits. As I see it, it is ambiguous in this context. It can either mean $n$ equations or $n$ variables. Luckily, in this particular case it is both. :)

Opalg said:
I have not come across the term "class" in that context. My guess is that what it means is that the matrix of coefficients in the system (1) has rank $n$. That implies that the equations have a unique solution, which is what is wanted here.

Aha! Ok!

The system $(1)$ has class $n$, since the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$.
Why do we conclude to that the class of the system is $n$ from the fact that the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$?
 
mathmari said:
Why do we conclude to that the class of the system is $n$ from the fact that the $\{x_1, ..., x_n \}$ consist the basis of $\widetilde{H}$?
Good question! We know that $\dim\widetilde H = n$, so the condition for the set $\{x_1, ..., x_n \}$ to be a basis is that it should be linearly independent. Or, to put it negatively, the set will fail to be a basis if and only if it is linearly dependent. That in turn is equivalent to the condition that there should exist scalars $\lambda_1,\ldots,\lambda_n$, not all $0$, such that $\sum \lambda_ix_i = 0.$ But then $\sum \lambda_i\langle x_i,x_j \rangle = 0$ for all $j$. That says that the rows of the matrix $A = (\langle x_i,x_j \rangle)$ are linearly dependent, which means that the rank of $A$ is less than $n$.

Conversely, if the rank of $A$ is less than $n$, then its rows are linearly dependent. So there exist scalars $\lambda_1,\ldots,\lambda_n$, not all $0$, such that $\sum \lambda_i\langle x_i,x_j \rangle = 0$ for all $j$. This says that $\sum \lambda_ix_i$ is orthogonal to each $x_j$. Since the $x_j$ form a basis, it follows that $\sum \lambda_ix_i = 0$ and so $\{x_1, ..., x_n \}$ is not a basis for $\widetilde H$.
 
Last edited:
Seemingly by some mathematical coincidence, a hexagon of sides 2,2,7,7, 11, and 11 can be inscribed in a circle of radius 7. The other day I saw a math problem on line, which they said came from a Polish Olympiad, where you compute the length x of the 3rd side which is the same as the radius, so that the sides of length 2,x, and 11 are inscribed on the arc of a semi-circle. The law of cosines applied twice gives the answer for x of exactly 7, but the arithmetic is so complex that the...
Is it possible to arrange six pencils such that each one touches the other five? If so, how? This is an adaption of a Martin Gardner puzzle only I changed it from cigarettes to pencils and left out the clues because PF folks don’t need clues. From the book “My Best Mathematical and Logic Puzzles”. Dover, 1994.

Similar threads

Replies
1
Views
681
Replies
1
Views
4K
Replies
17
Views
3K
Replies
9
Views
5K
Replies
21
Views
3K
Replies
0
Views
509
Replies
1
Views
2K
Back
Top