Partial derivative with respect to a vector

[TheOldHag]

I've come across using partial derivative notation for taking the partial derivative of a function f with respect to a vector x. I've never seen this before. It is also being referred to as a gradient. However, I have only seen gradients where all variables in the space are featured in the result vector. In this case, the result is a vector but not with components representing each dimension in the space. On wikipedia I've seen this referred to as matrix calculus notation. I would like to know a bit more about this in broad terms. For instance, for a space x1, x2, x3, x4 if I take the partial derivative with respect to a vector x1,x2 is that result vector valued function pointing in the direction of steepest ascent similar to a gradient but only for x1 and x2? Any other pointers appreciated.

 

[HallsofIvy]

If f is a function from R^n to R^m then we can always write it, around x= p, as a linear function plus a non-linear function: f(x)= f(p)+ L(x- p)+ N(x) where L is linear and N is non-linear. Of course, if f is continuous at x= p, we must have \lim_{x\to p} N(x)= 0. In addition, we say that f is "differentiable at x= p" if and only N(p) goes to zero "faster than linearly"- specifically that \lim_{x\to p}N(x)/|x-p|= 0.

In that case we say that L(p) is the "derivative of f(x) at x= p". If, for example, m= n= 1, the usual "real valued function of a single variable", f(x)= f'(p)(x- p) is the tangent line approximation to f- and "L" is f'(p). In the case that n= 1 and m> 1, we can think of L as being a vector which, multiplied by x- p, gives the vector value approximating f(p). That is, L is the vector of derivatives of the coordinate functions. If f(x) is a real valued function of several variables: n> 1 and m=1, L is the vector whose dot product with the variable gives f(x). That would be \nabla f.

More generally, If f is from R^n to R^m, L is a linear transform that maps the n-vector x- p to an m-vector: it can be represented (in a given coordinate system) by a matrix with n columns and m rows.