The Gradient and the Hessian of a Function of Two Vectors

[EngWiPy]

Hi,

Suppose we have a function of two n-dimensional vectors $$f(\mathbf{x},\mathbf{y})$$. How can we find the gradient and Hessian of this function?

Regards

 

[mathwonk]

i would just treat it as a function of 2n variables and take the vector of 1st, then the matrix of 2nd derivatives.

 

[EngWiPy]

i would just treat it as a function of 2n variables and take the vector of 1st, then the matrix of 2nd derivatives.

So, the gradient will be 1-by-2n vector, and the Hessian will be 2n-by-2n matrix?

 

[mathwonk]

yep. basically the gradient is a linear map approximating the original map. so you could also view it as a linear map from R^n x R^n -->R, and the Hessian I suppose as a bilinear such map.

I.e. if R^n = V, the 1st derivative is a linear map VxV-->R, and the second derivative is a symmetric bilinear map (VxV)x(VxV)-->R.

So if you really want to break up your source space into a pair of n vectors, then
you get two "partial" gradients, each a 1byn matrix, and you get a 2 by 2 Hessian matrix, where each block is an nbyn matrix, i.e. 4 nbyn matrices of vector second derivatives

 

[EngWiPy]

yep. basically the gradient is a linear map approximating the original map. so you could also view it as a linear map from R^n x R^n -->R, and the Hessian I suppose as a bilinear such map.

I.e. if R^n = V, the 1st derivative is a linear map VxV-->R, and the second derivative is a symmetric bilinear map (VxV)x(VxV)-->R.

So if you really want to break up your source space into a pair of n vectors, then
you get two "partial" gradients, each a 1byn matrix, and you get a 2 by 2 Hessian matrix, where each block is an nbyn matrix, i.e. 4 nbyn matrices of vector second derivatives

Ok, I see. Thank you.

Regards