Vector/Matrix Differentiation: Where to Put the Transpose?

[daviddoria]

if you want to find the derivative (gradient) of f(x)^2 when f is a vector, you would get

2*f(x)*del(f(x))

I never know where to put the transpose! sometimes its clear because another term in the equation will be a scalar, so you know an inner product is needed, but if you don't have a hint like that, how do you know if you should put the transpose on the f(x) or the del(f(x))? I suppose it depends on if f is a column or row vector, but a lot of times this is not given in the statement of the problem.

Any thoughts on this? Does anyone have a good online tutorial on vector/matrix differentiation?

Thanks!

Dave

 

[Peeter]

In terms of geometric algebra (since f is a vector grad and the the vector don't commute) one has:

$$\nabla f(x)^2 = (\nabla f(x)) f(x) + f(x) \nabla(f(x)) = 2 f(x) \cdot \nabla f(x)$$

(this works for a scalar f too since the dot product will just be scalar multiplication and you get $$2 f(x) \nabla f(x)$$.

Since you are talking about transposes, I'm assuming you've taken coordinate for your f and grad in some basis, in which case you can do it either way:

$$2 f(x)^\text{T}\nabla f(x)$$

or:
$$2 ({\nabla{f(x)}})^\text{T} f(x)$$

but in the second case you have to restrict the gradient to operating just on the first f(x).

 

[Peeter]

ps. I don't know of an online tutorial. I'm learning about this now from two places:

Hestenes's New Foundations for Classical Mechanics.
Doran/Lasenby's Geometric Algebra for Physicists.

 

[Peeter]

pps. If you choose to use coordinate vectors, and tranposition I think that basis also has to be orthonormal. Better to express using the dot product directly.

 

[Peeter]

It occurred to me that what I initially wrote is wrong (has to be since it was a scalar result when it should be a vector).

Using ticks to mark what the grad is operating on when separated one can write:

$$\nabla \lvert f \rvert ^2 = \nabla f^2 = \nabla f f = \acute{\nabla}f\acute{f} + (\nabla f)f$$

(again using the geometric product to multiply the two vectors).

Expanding this I get:

$$\nabla \lvert f \rvert ^2 = f (\nabla \cdot f) + (f \cdot \nabla) f - f \cdot (\nabla \wedge f) - (f \wedge \nabla) \cdot f$$

Which in the three dimensional case can be written in terms of the "normal" vector products.

$$\nabla \lvert f \rvert ^2 = f (\nabla \cdot f) + (f \cdot \nabla) f + f \times (\nabla \times f) + (f \times \nabla) \times f$$

Here we have a divergence, curl, directional derivative, and a ``normal'' directional derivative term (name?).

In my original post I allowed grad to commute with the vector, which means they are colinear (not generally true). Note that with the correction I don't really see how one would express this naturally with matrixes at all, so I've no idea now to answer your question of the where to put the transpose except for the colinear case.