How to derive the multivariate normal distribution

In summary, the density function for a multivariate normal distribution with an invertible covariance matrix \mathbf{\Sigma} can be derived using a change of variables and differentiation. The transformation of the standard i.i.d. Gaussian vector \mathbf{Y} using \mathbf{X} = \mathbf{A}\mathbf{Y} + \mathbf{\mu} leads to a determinant of \det(\mathbf{A}) in the denominator instead of \det(\mathbf{AA^T}), but this can be solved by using \mathbf{\Sigma} = \mathbf{A}\mathbf{A}^T as the covariance matrix.
  • #1
jone
6
0
If the covariance matrix [itex]\mathbf{\Sigma}[/itex] of the multivariate normal distribution is invertible one can derive the density function:

[itex]f(x_1,...,x_n) = f(\mathbf{x}) = \frac{1}{(\sqrt(2\pi))^n\sqrt(\det(\mathbf{\Sigma)}}\exp(-\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\mathbf{\Sigma}^{-1}(\mathbf{x}-\mathbf{\mu}))[/itex]

So, how do I derive the above?
 
Physics news on Phys.org
  • #2
Start with a normal distribution where all the variables are independent and then do a change of variables.
 
  • #3
I was on that track before, make use of the CDF and then differentiate back to get the PDF. This is how far I get: Let Y be a standard i.i.d. Gaussian vector. Then use the transformation

[itex]
\mathbf{X} = \mathbf{A}\mathbf{Y} + \mathbf{\mu}
[/itex]

[itex]
P(\mathbf{X} < \mathbf{x}) = P(\mathbf{A}\mathbf{Y} + \mathbf{\mu} < \mathbf{x}) = P(\mathbf{Y} < \mathbf{A}^{-1}(\mathbf{x}-\mathbf{\mu}))
[/itex]
Now I differentiate this to get the PDF

[itex]
f_{\mathbf{X}}(\mathbf{x}) = f_{\mathbf{Y}}(\mathbf{A}^{-1}\mathbf{x-\mu})\det(\mathbf{A}^{-1}) = f_{\mathbf{Y}}(\mathbf{A}^{-1}\mathbf{x-\mu})\frac{1}{\det(\mathbf{A})} = \frac{1}{(2\pi)^{n/2}\det(A)}\exp\left(\frac{1}{2}(\mathbf{x-\mu})^{T}(\mathbf{AA^T})^{-1}(\mathbf{x-\mu})\right)
[/itex]

So [itex]\det(\mathbf{A})}[/itex] pops out in the denominator, instead of [itex]\det(\mathbf{AA^T})}[/itex] it as it should be. Something is wrong in my differentiation here but I can't figure it out.
 
  • #4
jone said:
So [itex]\det(\mathbf{A})}[/itex] pops out in the denominator, instead of [itex]\det(\mathbf{AA^T})}[/itex] it as it should be. Something is wrong in my differentiation here but I can't figure it out.

Why do you think the denominator should be [itex]\det(\mathbf{AA^T})}[/itex].

That would give you something analogies to the variance while the denominator of the Gaussian function is the standard deviation.

You want:

[itex]\sqrt{|\mathbf{AA^T}|}=\sqrt{|\mathbf{A}|}\sqrt{|\mathbf{A^T}|}=|\mathbf{A}|[/itex]
 
  • #5
Ok, so now it works out. [itex]\mathbf{\Sigma} = \mathbf{A}\mathbf{A}^T[/itex] is the covariance matrix. Thank you for your help!
 
  • #6
jone said:
Ok, so now it works out. [itex]\mathbf{\Sigma} = \mathbf{A}\mathbf{A}^T[/itex] is the covariance matrix. Thank you for your help!

exactly! And, your welcome :)
 

FAQ: How to derive the multivariate normal distribution

1. What is the multivariate normal distribution?

The multivariate normal distribution is a probability distribution that describes the joint behavior of multiple random variables. It is often used in statistics and data analysis to model data that has more than one variable.

2. How is the multivariate normal distribution derived?

The multivariate normal distribution is derived by considering a linear transformation of multiple independent standard normal random variables. This transformation allows for the variables to be correlated and results in a multivariate normal distribution.

3. What are the properties of the multivariate normal distribution?

The multivariate normal distribution has the properties of symmetry, unimodality, and elliptical shape. It is also fully defined by its mean vector and covariance matrix, and the marginal distributions of each variable are normal.

4. What are some applications of the multivariate normal distribution?

The multivariate normal distribution is commonly used in statistical modeling and data analysis, particularly in fields such as finance, economics, and engineering. It is also used in machine learning for tasks such as classification and regression.

5. Are there any limitations to using the multivariate normal distribution?

While the multivariate normal distribution is a useful tool, it does have some limitations. It assumes that the variables are normally distributed and that there is a linear relationship between them. Additionally, it is not suitable for modeling data with outliers or extreme values.

Back
Top