Fisher Information Matrix: Equivalent Expressions

weetabixharry

I don't understand the following step regarding the [itex](i,j)^{th}[/itex] element of the Fisher Information Matrix, [itex]\textbf{J}[/itex]:
[tex]J_{ij}\triangleq\mathcal{E}\left\{ \frac{\partial}{\partial\theta_{i}}L_{\textbf{x}}(\textbf{θ})\frac{\par tial}{\partial\theta_{j}}

L_{\mathbf{x}}(\textbf{θ})\right\}
\\
=-\mathcal{E}\left\{ \frac{\partial^{2}}{\partial\theta_{i} \partial \theta_{j}}L_{\textbf{x}}(\textbf{θ})\right\}[/tex]

which is given in (Eq. 8.26, on p. 926 of) "Optimum Array Processing" by Harry van Trees. I don't know if the details matter, but [itex]L_{\textbf{x}}[/itex] is the log-likelihood function and he is looking at the problem of estimating the non-random real vector, [itex]\textbf{θ}[/itex], from discrete observations of a complex Gaussian random vector, [itex]\textbf{x}[/itex].

Am I missing something obvious? I'm not very sharp on partial derivatives.

DrDu

L_X is minus the log of the pdf. Write down the explicit expression for the expectation values in terms of L_X and its derivatives and use partial integration.

weetabixharry

The expressions are rather intimidating functions of complex matrices. I don't think I want to try partial integration. I tried just evaluating the derivatives to see if they would come out the same, but I ran out of paper before I had even scratched the surface.

If the result is specific to this problem, then I would be willing to take it on face value. It's just frustrating that I keep seeing the result stated without proof. It's as though it's too obvious to warrant a formal proof.

DrDu

The problem is that I am too lazy to tex something which can be found in any text on statistics or google, e.g.:

http://mark.reid.name/iem/fisher-inf...ikelihood.html

weetabixharry

I see.

Thanks - this was useful; its author claims to have spent 6 months pondering this problem, so I'm glad I'm not the only one to have had difficulty finding the solution.

DrDu

6 month is completely inacceptable. As I said, this topic is contained in every book on introductory statistics.
Elsewise it is a good idea to search for some lecture notes containing the problem: I used "fisher information lecture notes" and almost every lecture note contained a proof of the statement, e.g. the first one i got:
http://ocw.mit.edu/courses/mathemati...s/lecture3.pdf

weetabixharry

This is obviously incorrect.

This is no longer necessary. The topic is resolved.