Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

How to derive the multivariate normal distribution

  1. Jul 18, 2009 #1
    If the covariance matrix [itex]\mathbf{\Sigma}[/itex] of the multivariate normal distribution is invertible one can derive the density function:

    [itex]f(x_1,...,x_n) = f(\mathbf{x}) = \frac{1}{(\sqrt(2\pi))^n\sqrt(\det(\mathbf{\Sigma)}}\exp(-\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\mathbf{\Sigma}^{-1}(\mathbf{x}-\mathbf{\mu}))[/itex]

    So, how do I derive the above?
  2. jcsd
  3. Jul 18, 2009 #2
    Start with a normal distribution where all the variables are independent and then do a change of variables.
  4. Jul 18, 2009 #3
    I was on that track before, make use of the CDF and then differentiate back to get the PDF. This is how far I get: Let Y be a standard i.i.d. Gaussian vector. Then use the transformation

    \mathbf{X} = \mathbf{A}\mathbf{Y} + \mathbf{\mu}

    P(\mathbf{X} < \mathbf{x}) = P(\mathbf{A}\mathbf{Y} + \mathbf{\mu} < \mathbf{x}) = P(\mathbf{Y} < \mathbf{A}^{-1}(\mathbf{x}-\mathbf{\mu}))
    Now I differentiate this to get the PDF

    f_{\mathbf{X}}(\mathbf{x}) = f_{\mathbf{Y}}(\mathbf{A}^{-1}\mathbf{x-\mu})\det(\mathbf{A}^{-1}) = f_{\mathbf{Y}}(\mathbf{A}^{-1}\mathbf{x-\mu})\frac{1}{\det(\mathbf{A})} = \frac{1}{(2\pi)^{n/2}\det(A)}\exp\left(\frac{1}{2}(\mathbf{x-\mu})^{T}(\mathbf{AA^T})^{-1}(\mathbf{x-\mu})\right)

    So [itex]\det(\mathbf{A})}[/itex] pops out in the denominator, instead of [itex]\det(\mathbf{AA^T})}[/itex] it as it should be. Something is wrong in my differentiation here but I can't figure it out.
  5. Jul 19, 2009 #4
    Why do you think the denominator should be [itex]\det(\mathbf{AA^T})}[/itex].

    That would give you something analogies to the variance while the denominator of the Gaussian function is the standard deviation.

    You want:

  6. Jul 19, 2009 #5
    Ok, so now it works out. [itex]\mathbf{\Sigma} = \mathbf{A}\mathbf{A}^T[/itex] is the covariance matrix. Thank you for your help!
  7. Jul 19, 2009 #6
    exactly! And, your welcome :)
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook