Proof of differentiability for <x,x>

[JulienB]

Homework Statement

Hi everybody! I'm struggling to solve the following problem:

Let $< \cdot, \cdot >$ be an inner product on the vector space $X$, and $|| \cdot ||$ is the norm generated by the inner product. Prove that the function $x \in X \mapsto ||x||^2 \in \mathbb{R}$ is differentiable.

Homework Equations

I copy here what I have in my script:

$\exists L \in \mathbb{L} (X;Y): \lim\limits_{y \to 0} \frac{f(x + y) - f(x) - L y}{||y||} = 0 \implies f$ is differentiable at $x$ and $L$ is unique. When $\exists f'(x) \forall x \in M \implies f$ is differentiable and $f': M \to \mathbb{L} (X;Y)$ is the derivative ($M \subseteq X$ is an open set, $\mathbb{L} (X;Y)$ is the set of all multilinear continuous functions mapping from $X$ to $Y$).

The Attempt at a Solution

Well... This is very abstract to me. I assume that the norm generated by the inner product is:

$||x|| := \sqrt{<x,x>}$

which leads me to

$||x||^2 = <x,x>$.

Inserting this in the definition for differentiability gives me:

$\lim\limits_{y \to 0} \frac{f(x + y) - f(x) - L y}{||y||} = \lim\limits_{y \to 0} \frac{<x+y,x+y> - <x,x> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{<x,x+y> + <y,x+y> - <x,x> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{<x,x> + <x,y> + <y,x> + <y,y> - <x,x> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{2<x,y> + <y,y> - L y}{\sqrt{<y,y>}}$

To begin, I'm not sure that's the way to go. I didnt get any other bright idea though. Notice that in the last line, I used the property of the scalar product $<x,y> = <y,x>$ which should hold if we are in $\mathbb{R}^n$. It is not clear to me if that is the case, since no information is given about $X$. What would you guys say?

Anyway I am kind of stuck then. I kind somewhat rewrite the expression I got as:

$\lim\limits_{y \to 0} \frac{2<x,y> + <y,y> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \sqrt{<y,y>} + \frac{2<x,y> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{2<x,y> - L y}{\sqrt{<y,y>}}$

and then I am quite stuck again. Can I somehow differentiate using L'Hopital? That seems pretty unlikely.Thank you guys in advance for your answers, I appreciate it.Julien.

 

[Ray Vickson]

Homework Statement

Hi everybody! I'm struggling to solve the following problem:

Let $< \cdot, \cdot >$ be an inner product on the vector space $X$, and $|| \cdot ||$ is the norm generated by the inner product. Prove that the function $x \in X \mapsto ||x||^2 \in \mathbb{R}$ is differentiable.

Homework Equations

I copy here what I have in my script:

$\exists L \in \mathbb{L} (X;Y): \lim\limits_{y \to 0} \frac{f(x + y) - f(x) - L y}{||y||} = 0 \implies f$ is differentiable at $x$ and $L$ is unique. When $\exists f'(x) \forall x \in M \implies f$ is differentiable and $f': M \to \mathbb{L} (X;Y)$ is the derivative ($M \subseteq X$ is an open set, $\mathbb{L} (X;Y)$ is the set of all multilinear continuous functions mapping from $X$ to $Y$).

The Attempt at a Solution

Well... This is very abstract to me. I assume that the norm generated by the inner product is:

$||x|| := \sqrt{<x,x>}$

which leads me to

$||x||^2 = <x,x>$.

Inserting this in the definition for differentiability gives me:

$\lim\limits_{y \to 0} \frac{f(x + y) - f(x) - L y}{||y||} = \lim\limits_{y \to 0} \frac{<x+y,x+y> - <x,x> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{<x,x+y> + <y,x+y> - <x,x> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{<x,x> + <x,y> + <y,x> + <y,y> - <x,x> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{2<x,y> + <y,y> - L y}{\sqrt{<y,y>}}$

To begin, I'm not sure that's the way to go. I didnt get any other bright idea though. Notice that in the last line, I used the property of the scalar product $<x,y> = <y,x>$ which should hold if we are in $\mathbb{R}^n$. It is not clear to me if that is the case, since no information is given about $X$. What would you guys say?

Anyway I am kind of stuck then. I kind somewhat rewrite the expression I got as:

$\lim\limits_{y \to 0} \frac{2<x,y> + <y,y> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \sqrt{<y,y>} + \frac{2<x,y> - L y}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{2<x,y> - L y}{\sqrt{<y,y>}}$

and then I am quite stuck again. Can I somehow differentiate using L'Hopital? That seems pretty unlikely.Thank you guys in advance for your answers, I appreciate it.Julien.

I think you have forgotten that the "differential" $df$ is a linear functional that depends on $x$, so its value at $y$ is $L_x(y) \in \mathbb{R}$, not just $L(y)$.

 

[JulienB]

Hi @Ray Vickson and thank you for your answer. I would assume that $X$ is finite-dimensional since that's what we were working with the whole semester. Not sure how to make use of your suggestion about expanding $||x||^2$. Also not quite sure I understand it: is the inner product in finite-dimensional always defined as $<x,x> = x_1^2 + ... + x_n^2$? I was under the impression that it could be defined differently as long as its properties ($<x,x>\ > 0\ \forall x \neq 0$, symmetry, bilinear form as far as I can remember, maybe others such as commutativity) are respected. I thought that's what the "an" in "an inner product" was referring to, if you see what I'm saying.Julien.

 

[Ray Vickson]

Hi @Ray Vickson and thank you for your answer. I would assume that $X$ is finite-dimensional since that's what we were working with the whole semester. Not sure how to make use of your suggestion about expanding $||x||^2$. Also not quite sure I understand it: is the inner product in finite-dimensional always defined as $<x,x> = x_1^2 + ... + x_n^2$? I was under the impression that it could be defined differently as long as its properties ($<x,x>\ > 0\ \forall x \neq 0$, symmetry, bilinear form as far as I can remember, maybe others such as commutativity) are respected. I thought that's what the "an" in "an inner product" was referring to, if you see what I'm saying.Julien.

Yes, I see. So, use whatever definition of $\langle x,x \rangle$ applies to your problem.

By the way: it is fairly typical to start with a definition of $\langle x,x \rangle$, and---if it possesses certain properties---go on to define the inner product $\langle x,y \rangle$ in terms of it.

 

[JulienB]

@Ray Vickson Thank you for your answer. Here is the definition I have of the inner product (just to make sure we talk about the same thing :) ):

The inner product is a function $(x,y) \in X \times X \mapsto <x,y> \in \mathbb{R}$ so that $\forall x,y \in \mathbb{R}$ and $\lambda \in \mathbb{R}$ the following properties are valid:
1. Positive definite: $<x,x> > 0\ \forall x \neq 0$
2. Symmetry: $<x,y> = <y,x>$
3. Bilinear form: $<\lambda x + \mu y, z> = \lambda <x,z> + \mu <y,z>$

I'll try your suggestion nevertheless:

$\lim\limits_{y \to 0} \frac{f(x+y) - f(x) - Ly}{||y||} = \lim\limits_{y \to 0} \frac{<x+y,x+y> - <x,x> - Ly}{\sqrt{<y,y>}}$
$= \lim\limits_{y \to 0} \frac{(x_1+y_1)^2 + ... + (x_n+y_n)^2 - x_1^2 - ... - x_n^2 - Ly}{\sqrt{y_1^2 + ... + y_n^2}}$
$= \lim\limits_{y \to 0} \frac{x_1^2 + y_1^2 + 2x_1 y_1 + ... + x_n^2 + y_n^2 + 2x_n y_n - x_1^2 - ... - x_n^2 - Ly}{\sqrt{y_1^2 + ... + y_n^2}}$
$= \lim\limits_{y \to 0} \frac{y_1^2 + ... + y_n^2 + 2x_1 y_1 + ... + 2 x_n y_n - Ly}{\sqrt{y_1^2 + ... + y_n^2}}$
$= \lim\limits_{y \to 0} \frac{y_1^2 + ... + y_n^2 + 2x_1 y_1 + ... + 2 x_n y_n - Ly}{\sqrt{y_1^2 + ... + y_n^2}}$

which looks an awful lot like what I had in post #1... I'm going to try L'Hopital (not sure I use it correctly with the components though...):

$= \lim\limits_{y \to 0} \frac{2y_1 + ... + 2y_n + 2x_1 + ... + 2 x_n - L}{y_1 + ... + y_n} \sqrt{y_1^2 + ... + y_n^2}$
$= \lim\limits_{y \to 0} 2\sqrt{y_1^2 + ... + y_n^2} + \frac{(2x_1 + ... + 2 x_n - L) \sqrt{y_1^2 + ... + y_n^2}}{y_1 + ... + y_n}$
$= \lim\limits_{y \to 0} \frac{(2x_1 + ... + 2 x_n - L) \sqrt{y_1^2 + ... + y_n^2}}{y_1 + ... + y_n}$

...and a second time...

$= \lim\limits_{y \to 0} \frac{(2x_1 + ... + 2x_n - L) (y_1 + ... + y_n)}{(y_1 + ... + y_n) \sqrt{y_1^2 + ... y_n^2}}$
$= \lim\limits_{y \to 0} \frac{(2x_1 + ... + 2x_n - L)}{\sqrt{y_1^2 + ... y_n^2}}$

...which is a complete disaster. Well. I guess I can't use L'Hopital that way.

What am I doing wrong here? This proof can't be so difficult :(Thanks a lot for your help.Julien.

 

[Ray Vickson]

@Ray Vickson Thank you for your answer. Here is the definition I have of the inner product (just to make sure we talk about the same thing :) ):

The inner product is a function $(x,y) \in X \times X \mapsto <x,y> \in \mathbb{R}$ so that $\forall x,y \in \mathbb{R}$ and $\lambda \in \mathbb{R}$ the following properties are valid:
1. Positive definite: $<x,x> > 0\ \forall x \neq 0$
2. Symmetry: $<x,y> = <y,x>$
3. Bilinear form: $<\lambda x + \mu y, z> = \lambda <x,z> + \mu <y,z>$

Thanks a lot for your help.Julien.

Using only those three properties above you can show that $\langle x,x \rangle = \sum_{i=1}^n c_{ii} x_i^2 +2 \sum_{1 \leq i < j \leq n} c_{ij} x_i x_j$, where $c_{ij} = \langle e_i,e_j \rangle$, with $c_{ij} = c_{ji}$ if $i \neq j$. Here, $e_1 = (1,0,0,\ldots,0)$, $e_2 = (0,1,0 \ldots, 0)$, $\ldots$, $e_n = (0,0,0 \ldots, 1)$ is the standard basis for ${\mathbb{R}}^n$. That follows from writing $x = \sum_i x_i e_i$ and expanding $\| x \|^2$ using properties 2--3.

Now all you need to know is how to find the derivatives of a quadratic function.

 

[JulienB]

Hi @Ray Vickson and thanks again for your answer. After some trials and errors I've been able to come up with the same expression as you in your last post. But something is troubling me: isn't $<e_i, e_j> = 0$? That would make sense since $<x,x> = x_1^2 + ... + x_n^2$. I assume you left it there in case we are not working with the standard basis, is that right?

About the derivatives of a quadratic function, not sure what you mean. Are you referring to something like $\frac{\partial}{\partial (x_1, ..., x_n)} <x,x> = \sum_{i=1}^{n} 2 <\vec{e_i},\vec{e_i}> x_i^2 + 2 \sum_{1 \leq i < j \leq n} <\vec{e_i}, \vec{e_j}> (x_i + x_j)$? Maybe I just wrote crap, I must admit I'm getting quite confused about this problem. In the standard basis that would be equivalent to writing $\frac{\partial}{\partial\vec{x}} <\vec{x},\vec{x}> = \frac{\partial}{\partial (x_1, ..., x_n)} \vec{x}^2 = 2 \sum_{i=1}^n x_i^2$ I suppose. Apologies in advance if that makes no sense. :)By the way I dig the photo of your dog :)Julien.

 

[Ray Vickson]

Hi @Ray Vickson and thanks again for your answer. After some trials and errors I've been able to come up with the same expression as you in your last post. But something is troubling me: isn't $<e_i, e_j> = 0$? That would make sense since $<x,x> = x_1^2 + ... + x_n^2$. I assume you left it there in case we are not working with the standard basis, is that right?

About the derivatives of a quadratic function, not sure what you mean. Are you referring to something like $\frac{d}{dx} <x,x> = \sum_{i=1}^{n} 2 <\vec{e_i},\vec{e_i}> x_i + 2 \sum_{1 \leq i < j \leq n} <\vec{e_i}, \vec{e_j}> (x_i + x_j)$? Maybe I just wrote crap, I must admit I'm getting quite confused about this problem. In the standard basis that would be equivalent to writing $\frac{d}{d\vec{x}} <\vec{x},\vec{x}> = \frac{d}{d\vec{x}} \vec{x}^2 = 2\vec{x}$ I suppose. Apologies in advance if that makes no sense. :)By the way I dig the photo of your dog :)Julien.

You said in post #3 that you were unhappy assuming that $\|x\|^2=x_1^2+x_2^2+ \cdots + x_n^2$, which would necessarily mean you would be unhappy assuming that $\langle e_i, e_j \rangle = 0$ for $j \neq i$, and perhaps also unhappy assuming $\|e_i\|^2 = 1$ for all $i$. However, you wanted to assume three fundamental properties of $\langle , \rangle$. So, the new form of $\|x\|^2$ applies to any type of inner product, even one in which the standard basis is not orthonormal.

At this point I do not understand what are your difficulties. Learning how to differentiate quadratics was the very first thing you ever did when you started learning calculus.

 

[JulienB]

Hi @Ray Vickson and sorry for the misunderstanding. I would love indeed to solve the problem using the properties of the scalar product, I was just checking (for myself, so to say) how that would work with a standard basis. I guess I didn't express it correctly.

My problem is that I am not sure what I should be doing. I want to prove that $||x||^2$ is differentiable, can I do that by simply differentiating your expression in post #6? I thought the proof was going to make use of the limit definition. Sorry, maybe I've lost your train of thoughts somewhere :/

Thank you for your help.Julien.

 

[Ray Vickson]

Hi @Ray Vickson and sorry for the misunderstanding. I would love indeed to solve the problem using the properties of the scalar product, I was just checking (for myself, so to say) how that would work with a standard basis. I guess I didn't express it correctly.

My problem is that I am not sure what I should be doing. I want to prove that $||x||^2$ is differentiable, can I do that by simply differentiating your expression in post #6? I thought the proof was going to make use of the limit definition. Sorry, maybe I've lost your train of thoughts somewhere :/

Thank you for your help.Julien.

Of course when we first introduce concepts related to "differentiability" we need to begin with a definition that typically (but not always) involves a limit.

However, always using the definition over and over again in every problem is a real pain, so instead we start proving some theorems to help us out. Typically, such theorems will tell us other, easier, conditions for differentiability, so we can most of the time avoid going back to the "limit" definition. For multivariable functions those alternative tests will involve properties of the partial derivatives, and because we have previously proved lots of properties of derivatives (including derivative formulas for many standard functions) we can by-pass a lot of work and just use what we have learned already.

That is my very last word on this topic,

 

[JulienB]

@Ray Vickson Okay thank you for your help, I will think about it.Julien.