Derivative of a Real-Valued Function of Several Variables....

[Math Amateur]

I am reading Hugo D. Junghenn's book: "A Course in Real Analysis" ...

I am currently focused on Chapter 9: "Differentiation on \mathbb{R}^n"

I need some help with an aspect of Definition 9.1.3 ...

Definition 9.1.3 and the relevant accompanying text read as follows:

At the top of the above text, in Definition 9.1.3 we read the following text:

" ... ... there exists a vector $f'(a)$ in $\mathbb{R}^n$ ... ... "My question is as follows:

How (arguing from the definition of derivative) do we indicate\demonstrate\prove that $f'(a) \in \mathbb{R}^n$ ... ...?Hope someone can help ... ...

Peter

 

[Math Amateur]

<Error filled quote deleted>

Thanks Mark44 ...

Just now reflecting on what you have written ...

Struggling a bit ...

If $f'(\vec a)$ is a scalar ... that is $f'(\vec a) \in \mathbb{R}$ then how do we interpret the term $f'(\vec a) \cdot \vec h$ in equation (9.1) ... it looks as if to make sense of such an expression $f'(\vec a)$ has to be a vector in $\mathbb{R}^n$ ...

I am assuming that $\vec h \in \mathbb{R}^n$ and that $\vec h = (h_1, h_2, \ ... \ ... \ , h_n)$

Can you clarify ...Thanks again for your help ...

Peter

 

[fresh_42]

The author uses the following notations for the derivative of the in $\vec{x}=\vec{a}$ differentiable function $f \, : \,\mathbb{R}^n\longrightarrow \mathbb{R}\,$:
\begin{align*}
f\,'(a) &= df_a = (\nabla f)(a) = \nabla f (a) \\
&= df_{\vec{a}}= (\nabla f)(\vec{a}) = \nabla f(\vec{a}) = f\,'(\vec{a}) = \left. \dfrac{d}{d\vec{x}}\right|_{\vec{x}=\vec{a}} f(\vec{x})\\
&= (\partial_1f(a),\ldots ,\partial_n f(a)) \\
& = (\partial_1f(\vec{a}),\ldots ,\partial_n f(\vec{a})) = \left(\left. \dfrac{\partial}{\partial x_1}\right|_{\vec{x}=\vec{a}}f(\vec{x}),\ldots ,\left. \dfrac{\partial}{\partial x_n}\right|_{\vec{x}=\vec{a}}f(\vec{x})\right)
\end{align*}
which is a linear map from $\mathbb{R}^n$ to $\mathbb{R}$ which is represented by a $(1 \times n)-$matrix, i.e. a vector, if the coordinates $x_1,\ldots ,x_n$ are given, resp. the basis vectors $e^1,\ldots ,e^n$. The mapping instruction is thus:
$$
f\,'(\vec{a})(\vec{h}) = f'(\vec{a})\cdot \vec{h} = \sum_{i=1}^n \partial_i f(\vec{a}) \cdot h_i =\sum_{i=1}^n \left. \dfrac{\partial}{\partial x_i}\right|_{\vec{x}=\vec{a}}f(\vec{x}) \cdot h_i \in \mathbb{R}
$$

I think it could really help you to read https://www.physicsforums.com/insights/the-pantheon-of-derivatives-i/ (part 1), in order to get a feeling of what is going on.

If you do so, please do the exercise and find out why my equation (1) is exactly the definition in 9.1 above and which letters translate to which here.

 

[Mark44]

<Error filled quote deleted>

Thanks Mark44 ...

Just now reflecting on what you have written ...

Struggling a bit ...

If $f'(\vec a)$ is a scalar

That was a mistake on my part. I mistakenly thought that since f was a map from $\mathbb R^n$ to $\mathbb R$, then f' would be, as well. The differential is a scalar, but the derivative is a vector. Apologies for any misunderstanding I caused.

... that is $f'(\vec a) \in \mathbb{R}$ then how do we interpret the term $f'(\vec a) \cdot \vec h$ in equation (9.1) ... it looks as if to make sense of such an expression $f'(\vec a)$ has to be a vector in $\mathbb{R}^n$ ...

I am assuming that $\vec h \in \mathbb{R}^n$ and that $\vec h = (h_1, h_2, \ ... \ ... \ , h_n)$

Can you clarify ...

See above.

Thanks again for your help ...

Peter

 

[Math Amateur]

The author uses the following notations for the derivative of the in $\vec{x}=\vec{a}$ differentiable function $f \, : \,\mathbb{R}^n\longrightarrow \mathbb{R}\,$:
\begin{align*}
f\,'(a) &= df_a = (\nabla f)(a) = \nabla f (a) \\
&= df_{\vec{a}}= (\nabla f)(\vec{a}) = \nabla f(\vec{a}) = f\,'(\vec{a}) = \left. \dfrac{d}{d\vec{x}}\right|_{\vec{x}=\vec{a}} f(\vec{x})\\
&= (\partial_1f(a),\ldots ,\partial_n f(a)) \\
& = (\partial_1f(\vec{a}),\ldots ,\partial_n f(\vec{a})) = \left(\left. \dfrac{\partial}{\partial x_1}\right|_{\vec{x}=\vec{a}}f(\vec{x}),\ldots ,\left. \dfrac{\partial}{\partial x_n}\right|_{\vec{x}=\vec{a}}f(\vec{x})\right)
\end{align*}
which is a linear map from $\mathbb{R}^n$ to $\mathbb{R}$ which is represented by a $(1 \times n)-$matrix, i.e. a vector, if the coordinates $x_1,\ldots ,x_n$ are given, resp. the basis vectors $e^1,\ldots ,e^n$. The mapping instruction is thus:
$$
f\,'(\vec{a})(\vec{h}) = f'(\vec{a})\cdot \vec{h} = \sum_{i=1}^n \partial_i f(\vec{a}) \cdot h_i =\sum_{i=1}^n \left. \dfrac{\partial}{\partial x_i}\right|_{\vec{x}=\vec{a}}f(\vec{x}) \cdot h_i \in \mathbb{R}
$$

I think it could really help you to read https://www.physicsforums.com/insights/the-pantheon-of-derivatives-i/ (part 1), in order to get a feeling of what is going on.

If you do so, please do the exercise and find out why my equation (1) is exactly the definition in 9.1 above and which letters translate to which here.

Thanks fresh_42 ...

You write:

" ... ... I think it could really help you to read https://www.physicsforums.com/insights/the-pantheon-of-derivatives-i/ (part 1), in order to get a feeling of what is going on.

If you do so, please do the exercise and find out why my equation (1) is exactly the definition in 9.1 above and which letters translate to which here. ... ... "Taking up your exercise ... ... we wish to show that equation (9.1) in Junghenn is equivalent to equation 1 in "Differentiation in a Nutshell" ... ..

Now ... Definition 9.1.3 in Junghenn (where we find (9.1) ) defines differentiation of a real valued function of several variables ...

... while equation 1 in "Differentiation in a Nutshell" defines the differentiation of a real-valued function of a single real variable ...

... but ... amending equation (1) for a real valued function of several variables ... we have

$f( x_0 + v) = f(x_0) + J(v) + r(v)$ ... ... ... ... ... (1)

and

$\lim_{ v \rightarrow 0 } \frac{ r(v) }{ \| v \| } = 0$ ... ... ... ... ... (1a)

where $x_0, v$ are vectors in $\mathbb{R}^n$ ... ...Equation (9.1) (with variables amended appropriately to match equation (1) ) effectively defines the derivative $f'( x_0 )$ as follows:

$\lim_{ v \rightarrow 0 } \frac{ f( x_0 + v ) - f ( x_0 ) - f' ( x_0 ) \cdot v }{ \| v \| } = 0$ ... ... ... ... ... (9.1)Now ... (9.1) $\Longrightarrow$

$\frac{ f( x_0 + v ) - f ( x_0 ) - f' ( x_0 ) \cdot v }{ \| v \| } = \frac{ \epsilon (v) }{ \| v \| }$ ... ... ... ... ... (2)

where $\epsilon (v)$ is the "error" in using $f' ( x_0 ) \cdot v$ to estimate $f( x_0 + v ) - f ( x_0 )$ ... ...(2) $\Longrightarrow$

$f( x_0 + v ) - f ( x_0 ) - f' ( x_0 ) \cdot v = \epsilon (v)$

... and ... we require $\epsilon (v)$ to converge faster to zero than linear which means:

$\lim_{ v \rightarrow 0 } \frac{ \epsilon (v) }{ \| v \| } = 0$ ...... then ... rename $\epsilon (v)$ as $r(v)$ ... and we get (1) ... assuming, of course, that $f' ( x_0 ) \cdot v = J(v)$ ... ...

... so (9.1) is equivalent to (1) ...... BUT ... unsure if the above is rigorous and correct ... and, further ... VERY unsure how to justify that (9.1) implies that $\epsilon (v)$ converges faster to zero than linear ... ...Can you please confirm that the above is correct and/or point out errors and shortcomings ...

Can you please also explain how to justify that (9.1) implies that \epsilon (v) converges faster to zero than linear ... ...

Peter

 

[fresh_42]

Yes, that's what I meant. More basically though, as $x_0=a\; , \;h=v$ and what you wrote $J(v)=f\,'(a)\cdot h$. I took $J$ to stand for the Jacobian matrix evaluated at $x_0=a$ applied to direction $v$. If we simply take $r(v)=f(x_0+v)-f(x_0)-J(v)$ form (1) and put it in (1a) then we get
$$
\lim_{v\to 0} \dfrac{r(v)}{||v||} = \lim_{v \to 0} \dfrac{f(x_0+v)-f(x_0)-J(v)}{||v||} = \lim_{v \to 0} \dfrac{f(x_0+v)-f(x_0)-f\,'(a)(v)}{||v||} = \lim_{h \to 0} \dfrac{f(a+h)-f(a)-f\,'(a)(h)}{||h||} = 0
$$
which is exactly (9.1). and your $\varepsilon(v)=r(v)$. The faster than linear aspect comes from, although we divide by the norm of $||v||=||h||$ which is basically the linear component, we're still running towards zero. That's what is meant by faster than linear. This is usually written as $r(v)=o(||v||)$ or simply by $f\,'(a)(h)=f(a+h)-f(a)+o(||h||)$ instead of explicitly naming the error function $\varepsilon\,,$ resp. the remainder function $r\,,$ plus the limit, see https://en.wikipedia.org/wiki/Big_O_notation#Little-o_notation. In the Wiki article they say "$||v||$ grows much faster than $r(v)$" which is the same, just expressed the other way around: If $||v||$ overtakes $r(v)$, then $\dfrac{r(v)}{||v||}$ still runs towards zero for $v \to 0$.