Register to reply 
Fisher Information Matrix: Equivalent Expressionsby weetabixharry
Tags: cramerrao bound, fisher information, log likelihood, normal distribution, partial derivative 
Share this thread: 
#1
Jul612, 01:49 PM

P: 108

I don't understand the following step regarding the [itex](i,j)^{th}[/itex] element of the Fisher Information Matrix, [itex]\textbf{J}[/itex]:
[tex]J_{ij}\triangleq\mathcal{E}\left\{ \frac{\partial}{\partial\theta_{i}}L_{\textbf{x}}(\textbf{θ})\frac{\par tial}{\partial\theta_{j}} L_{\mathbf{x}}(\textbf{θ})\right\} \\ =\mathcal{E}\left\{ \frac{\partial^{2}}{\partial\theta_{i} \partial \theta_{j}}L_{\textbf{x}}(\textbf{θ})\right\}[/tex] which is given in (Eq. 8.26, on p. 926 of) "Optimum Array Processing" by Harry van Trees. I don't know if the details matter, but [itex]L_{\textbf{x}}[/itex] is the loglikelihood function and he is looking at the problem of estimating the nonrandom real vector, [itex]\textbf{θ}[/itex], from discrete observations of a complex Gaussian random vector, [itex]\textbf{x}[/itex]. Am I missing something obvious? I'm not very sharp on partial derivatives. 


#2
Jul612, 02:36 PM

Sci Advisor
P: 3,633

L_X is minus the log of the pdf. Write down the explicit expression for the expectation values in terms of L_X and its derivatives and use partial integration.



#3
Jul712, 11:14 PM

P: 108

If the result is specific to this problem, then I would be willing to take it on face value. It's just frustrating that I keep seeing the result stated without proof. It's as though it's too obvious to warrant a formal proof. 


#4
Jul912, 03:24 AM

Sci Advisor
P: 3,633

Fisher Information Matrix: Equivalent Expressions
The problem is that I am too lazy to tex something which can be found in any text on statistics or google, e.g.:
http://mark.reid.name/iem/fisherinf...ikelihood.html 


#5
Jul1112, 03:45 AM

P: 108




#6
Jul1112, 06:05 AM

Sci Advisor
P: 3,633

6 month is completely inacceptable. As I said, this topic is contained in every book on introductory statistics.
Elsewise it is a good idea to search for some lecture notes containing the problem: I used "fisher information lecture notes" and almost every lecture note contained a proof of the statement, e.g. the first one i got: http://ocw.mit.edu/courses/mathemati...s/lecture3.pdf 


#7
Jul1412, 05:47 AM

P: 108




Register to reply 
Related Discussions  
Invariance of the Fisher matrix  Set Theory, Logic, Probability, Statistics  2  
Equivalent expressions  Quantum Physics  5  
Equivalent expressions?  Classical Physics  1  
Fisher information  General Math  0 