Understanding of Higher-Order Derivatives

[shaggymoods]

Hey guys, so this may be a really silly question, but I'm trying to grasp a subtle point about higher-order derivatives of multivariable functions. In particular, suppose we have an infinitely differentiable function

$$f: \mathbb{R}^{n} \rightarrow \mathbb{R}$$

I know that the first derivative of this function is a linear map $$\lambda: \mathbb{R}^{n}\rightarrow\mathbb{R}$$. However, when we take the second-derivative of $$\lambda$$, some questions arise for me:

1.) If we are taking this derivative when considering $$\lambda$$ as a linear function, then we'd just get back $$\lambda$$, which isn't the case. So how are we interpreting the first derivative when taking a second?

2.) In general, why do we say that $$D^{k}f:\mathbb{R}^{n^{k}}\rightarrow\mathbb{R}$$ and not $$D^{k}f:\mathbb{R}^{n}\rightarrow\mathbb{R}$$ ??

Thanks in advance.

 

[VKint]

If $$f : \mathbb{R}^n \to \mathbb{R}$$, then the derivative of $$f$$ is a linear map $$\lambda : \mathbb{R}^n \to \mathbb{R}$$ at each point in $$\mathbb{R}^n$$. That is to say, the derivative of $$f$$, properly considered, is a map $$Df : \mathbb{R}^n \to L(\mathbb{R}^n, \mathbb{R})$$, where $$L(\mathbb{R}^n, \mathbb{R})$$ denotes the space of all linear maps $$\lambda : \mathbb{R}^n \to \mathbb{R}$$, which is just the dual of $$\mathbb{R}^n$$ (and is thus isomorphic to $$\mathbb{R}^n$$). The second derivative of $$f$$ is then a map $$D^2 f : \mathbb{R}^n \to L(\mathbb{R}^n, L(\mathbb{R}^n, \mathbb{R})) \cong L(\mathbb{R}^n, \mathbb{R}^n)$$, where $$L(\mathbb{R}^n, \mathbb{R}^n)$$ is the space of all $$n \times n$$ matrices, and is isomorphic to $$\mathbb{R}^{n^2}$$. (The output of the second derivative is usually called the Hessian matrix of $$f$$.) Continuing in this vein, you can show that $$D^k f$$ is a map from $$\mathbb{R}^n$$ to $$\mathbb{R}^{n^k}$$, not a map from $$\mathbb{R}^{n^k} \to \mathbb{R}$$ as you suggest in #2.

Basically, what's going on here is that a derivative, properly defined, is a best linear approximation to a function. Thus, at some point $$\mathbf{p} \in \mathbb{R}^n$$, the derivative $$Df$$ takes the value of the linear map $$\lambda : \mathbb{R}^n \to \mathbb{R}$$ which most closely resembles $$f$$ near $$\mathbf{p}$$. Thus, $$Df$$ is actually a map from $$\mathbb{R}^n$$ into the space of all possible such approximations, and $$D^k f$$ is a map from $$\mathbb{R}^n$$ into some higher tensor product of $$\mathbb{R}^n$$ and its dual space. Your answer to #1 is thus that, while elements of the range of $$Df$$ must be linear maps and have trivial derivatives, $$Df$$ itself is not necessarily linear. This is why it is necessary to specify two arguments when evaluating $$Df$$: a location $$\mathbf{a}$$, and a direction $$\mathbf{h}$$. The location specifies a linear map, i.e., there is some linear map $$\lambda$$ for which $$Df : \mathbf{a} \mapsto \lambda$$. The direction then serves as the argument for $$\lambda$$, and, in a slight abuse of notation, we usually write $$\lambda(\mathbf{h}) \equiv Df(\mathbf{a})(\mathbf{h})$$ or $$Df(\mathbf{a}, \mathbf{h})$$.