Fisher matrix for multivariate normal distribution

In summary, the Fisher information matrix for multivariate normal distribution is said at many places to be simplified as:\mathcal{I}_{m,n} = \frac{\partial \mu^\mathrm{T}}{\partial \theta_m} \Sigma^{-1} \frac{\partial \mu}{\partial \theta_n}.\even on http://en.wikipedia.org/wiki/Fisher_information#Multivariate_normal_distribution.
  • #1
hdb
3
0
The fisher information matrix for multivariate normal distribution is said at many places to be simplified as:
[tex]\mathcal{I}_{m,n} = \frac{\partial \mu^\mathrm{T}}{\partial \theta_m} \Sigma^{-1} \frac{\partial \mu}{\partial \theta_n}.\ [/tex]
even on
http://en.wikipedia.org/wiki/Fisher_information#Multivariate_normal_distribution"
I am trying to come up with the derivation, but no luck so far. Does anyone have any ideas / hints / references, how to do this?

Thank you
 
Last edited by a moderator:
Physics news on Phys.org
  • #2
Using matrix derivatives one has [tex] D_x(x^T A x) = x^T(A+A^T) [/tex] from which it follows that [tex] D_{\theta} \log p(z ; \mu(\theta) , \Sigma) = (z-\mu(\theta))^T \Sigma^{-1} D_{\theta} \mu(\theta) [/tex] For simplicity let's write [tex] D_{\theta} \mu(\theta) = H [/tex] The FIM is then found as [tex] J = E[ ( D_{\theta} \log p(z ; \mu(\theta) , \Sigma))^T D_{\theta} \log p(z ; \mu(\theta) , \Sigma)] = E[ H^T R^{-1} (z - \mu(\theta))^T (z - \mu(\theta)) R^{-1} H] = H^T R^{-1} R R^{-1} H = H^T R^{-1} H [\tex] which is equivalent to the given formula. Notice that this formula only is valid as long as [tex] \Sigma [\tex] does not depend on [tex] \theta [\tex]. I'm still struggling to find a derivation of the more general case where also [tex] \Sigma [\tex] depends on [tex] \theta [\tex].

For some reason my tex code is not correctly parsed. I cannot understand why.
 
  • #3
Actually the general proof can apparently be found in Porat & Friedlander: Computation of the Exact Information Matrix of Gaussian Time Series with Stationary Random Components, IEEE Transactions on Acoustics, Speech and Signal Processing, Vol ASSP-34, No. 1, Feb. 1986.
 
  • #4
edmundfo said:
R^{-1} H] = H^T R^{-1} R R^{-1} H = H^T R^{-1} H [\tex]

For some reason my tex code is not correctly parsed. I cannot understand why.

For one thing, you're using the back slash [\tex] instead of the forward slash [/tex] at the end of your code.
 
  • #5
edmundfo said:
Actually the general proof can apparently be found in Porat & Friedlander: Computation of the Exact Information Matrix of Gaussian Time Series with Stationary Random Components, IEEE Transactions on Acoustics, Speech and Signal Processing, Vol ASSP-34, No. 1, Feb. 1986.
Thank you for the answers, in between I have found an another reference, which is a direct derivation of the same result, for me this one seems to be easier to interpret:

Klein, A., and H. Neudecker. “A direct derivation of the exact Fisher information matrix of Gaussian vector state space models.” Linear Algebra and its Applications 321, no. 1-3
 

1. What is the Fisher matrix for a multivariate normal distribution?

The Fisher matrix for a multivariate normal distribution is a matrix that contains information about the parameters of the distribution, specifically the mean and variance. It is used in statistical inference to estimate the parameters of the distribution based on a sample of data.

2. How is the Fisher matrix calculated?

The Fisher matrix is calculated by taking the negative second derivative of the log-likelihood function with respect to each parameter. This involves taking the partial derivatives of the log-likelihood function with respect to each parameter and then plugging them into a matrix.

3. What is the significance of the Fisher matrix?

The Fisher matrix is significant because it provides a way to quantify the amount of information contained in a sample of data about the parameters of a multivariate normal distribution. It also allows for the calculation of confidence intervals for the estimated parameters.

4. Can the Fisher matrix be used for any multivariate distribution?

No, the Fisher matrix is specific to the multivariate normal distribution. It cannot be used for other distributions, such as the multivariate t-distribution or the multivariate exponential distribution.

5. How is the Fisher information matrix related to the Fisher matrix?

The Fisher information matrix is the inverse of the Fisher matrix. It is used to calculate the standard errors of the estimated parameters and plays a crucial role in statistical inference for the multivariate normal distribution.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
6
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
Replies
0
Views
181
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
6
Views
3K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
916
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
995
  • Beyond the Standard Models
Replies
1
Views
1K
Back
Top