1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Marginal distribution of a Gaussian

  1. Oct 1, 2012 #1

    fluidistic

    User Avatar
    Gold Member

    1. The problem statement, all variables and given/known data
    The random variables X and Y have a joint probability distribution of [tex]f_{XY}(x,y)= \frac{1}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}} \exp \{ \left [ -\frac{1}{2(1-\rho ^2)} \right ] \left [ \left ( \frac{x-\mu _X}{\sigma _X} \right )^2 + \left ( \frac{y-\mu _Y}{\sigma _Y} \right )^2 -\frac{2\rho (x-\mu _X)(y-\mu _Y)}{\sigma _X \sigma _Y} \right ] \}[/tex].
    Where rho, sigma's and mu's are constants.
    Show that the marginal distributions are also Gaussians. Calculate their mean value and their variance.
    2. Relevant equations
    a)The marginal distribution of the variable X is [itex]f_ X (x)=\int _{-\infty}^{\infty} f_{XY}(x,y)dy[/itex].
    b)The marginal distribution of the variable Y is [itex]f_Y(y)=\int _{-\infty}^{\infty} f_{XY}(x,y)dx[/itex].
    c)[tex]\int _{-\infty}^{\infty} e^{-ax^2+bx }dx=\sqrt{\frac{\pi}{a}}e^{\frac{b^2}{4a}}[/tex].
    d)For the mean values I need to calculate [itex]\int f_X(x)xdx[/itex] and [itex]\int f_Y(y)ydy[/itex] which in my case is so horrible that I've lost some hope in it.
    3. The attempt at a solution
    So my problem resides in calculating that huge integral. I've done it 3 times, each time reaching a different result, but I've spotted an error in my first 2 attempts and didn't in my last attempt though the result is so ugly I can't believe I've done it right.
    So in order to perform [itex]\int _{-\infty}^{\infty} f_{XY}(x,y)dy[/itex], I factorize outside the integral the constant [itex]\frac{1}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}}\exp \{ \left [ - \frac{1}{2(1-\rho ^2)} \right ] \left ( \frac{x-\mu _X}{\sigma _X} \right ) \}[/itex].
    Therefore I am left to calculate the integral [tex]\int _{-\infty}^{\infty} \exp \{ \left [ - \frac{1}{2(1-\rho ^2)} \right ] \left [ \left ( \frac{y-\mu _Y}{\sigma _Y} \right ) ^2 - \frac{2\rho (x-\mu _X )(y- \mu _Y)}{\sigma _X \sigma _Y} \right ] \} dy[/tex].
    I skip many steps (latex is very long to write!) but what I did was rewrote the argument of the exponential into a form [itex]-ax^2+bx+c[/itex] and where [itex]a = \frac{1}{2(1-\rho ^2) \sigma _Y ^2}[/itex], [tex]b= \frac{\mu _Y}{\sigma _Y ^2(1-\rho ^2)} - \frac{\rho \mu _X}{\sigma _X \sigma _Y (1-\rho ^2)} + \frac{2\rho x}{\sigma _X \sigma Y}[/tex] and [tex]c= \left [ \frac{1}{2(1-\rho ^2)} \right ] \left [ \frac{2\rho \mu _X \mu _Y}{\sigma _X \sigma _Y}-\frac{2\rho x \mu _Y}{\sigma _X \sigma _Y} -\frac{\mu _Y ^2}{\sigma _Y ^2} \right ][/tex].
    This gave me as a final result, [tex]f_X(x)= \frac{1}{\sqrt {2\pi} \sigma _X} \exp \{ x^2 \left [ \frac{1}{2(1-\rho ^2)} \right ] \left ( \frac{4 \rho ^2 -1 }{\sigma _X ^2} \right ) +x \left [ \frac{1}{2(1-\rho ^2)} \left ( \frac{2\rho \mu _Y }{\sigma _X \sigma _Y}-\frac{4 \rho ^2 \mu _X}{\sigma _X ^2}+\frac{2\mu _X}{\sigma _X ^2} \right ) \right ] +\left [ \frac{1}{2(1-\rho ^2)} \right ] \left ( \frac{\rho ^2 \mu _X ^2}{\sigma _X ^2}- \frac{\mu _X ^2}{\sigma _X ^2} \right ) \}[/tex]. This would be my answer to the first question. For the marginal distribution of the random variable Y, I'd have to change all x's by y's and all X's by Y's in that result, because the joint distribution is totally symmetric with respect to x-y/X-Y.
    As you can see my answer is under the form [itex]Ae^{ax^2+bx+c}[/itex] so I think this implies it's a Gaussian?
    I would like to know whether my result is good and how can I calculate the mean of that function; I mean there must be a trick or something because the integral to perform looks much uglier than the first!
    Thanks for any help!
     
  2. jcsd
  3. Oct 1, 2012 #2

    jbunniii

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    Yikes, I haven't read through all of your equations to check their validity because, as you said, it's pretty ugly. I think you can simplify this problem quite a bit using this change of variables:

    [tex]U = \frac{X - \mu_X}{\sigma_X}[/tex]
    [tex]V = \frac{Y - \mu_Y}{\sigma_Y}[/tex]

    Then it's easy to find the marginal distributions, means, and variances for X and Y if you have found these things for U and V.
     
  4. Oct 1, 2012 #3

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    I don't want to spoil your fun, but the mere fact that the parameters have names like μX and σX, etc., should be a hint that the final answer is pretty simple. All I can say is that if you do the y-integration very carefully and completely, the formula for fX(x) comes out pleasantly uncomplicated.

    RGV
     
  5. Oct 2, 2012 #4

    fluidistic

    User Avatar
    Gold Member

    Wow. Wow. Wow. Worked like a charm, thanks a lot!!!!
    Haha, "my fun". Yeah you are right, I was execting a friendler result that the one I obtained, but I must say it's even simpler than what I thought. Thanks a lot guys, problem solved (99% sure). I'm too exhausted to post the full answer right now. If anyone care, just ask me and I'll post it.
     
  6. Oct 2, 2012 #5

    jbunniii

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    Nice! Glad it worked out. I still remember the day as an undergraduate when I had sweated through some similarly horrible thing, and then the teaching assistant goes over the solution and says "well, first of all, let's make this problem easier by changing the variables..."
     
  7. Oct 2, 2012 #6

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Yes, please post it.

    BTW: your initial post gave a wrong formula for fX(x).

    RGV
     
  8. Oct 3, 2012 #7

    fluidistic

    User Avatar
    Gold Member

    Here are my results: [itex]f_X(x)=\frac{1}{\sqrt{2\pi} \sigma _X} \exp \left [ -\frac{1}{2} \left ( \frac{x-\mu _X}{\sigma _X} \right )^2 \right ][/itex].
    [itex]f_Y(y)=\frac{1}{\sqrt{2\pi} \sigma _Y} \exp \left [ -\frac{1}{2} \left ( \frac{y-\mu _Y}{\sigma _Y} \right )^2 \right ][/itex].
    Mean of [itex]f_X(x)[/itex], [itex]\langle X \rangle = \mu _X[/itex].
    Mean of [itex]f_Y(y)[/itex], [itex]\langle Y \rangle = \mu _Y[/itex].
    Furthermore, [itex]\langle X ^2 \rangle = \mu _X ^2+ \sigma _X ^2[/itex] and [itex]\langle Y ^2 \rangle = \mu _Y ^2+ \sigma _Y ^2[/itex] so that the variances are [itex]\text{var} (f_X(x))= \sigma _X^2[/itex] and [itex]\text{var} (f_Y(y))= \sigma _Y^2[/itex].
    I must also get the covariance matrix which involves calculating the term [itex]\langle XY\rangle = \int _{-\infty}^{\infty} \int _{-\infty}^{\infty} f(x,y)xydxdy[/itex]. I guess I must look at an integral table to do [itex]\int f_{XY} (x,y)xdx[/itex].
     
  9. Oct 3, 2012 #8

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    These are the correct marginals, etc.

    To calculate ## E[(X-\mu_X)(Y - \mu_Y)]## you do not need to use integration tables. The first step would be to simplify everything by following previous suggestions, and use variables
    [tex] Z_1 = \frac{X - \mu_X}{\sigma_X}, \; Z_2 = \frac{Y - \mu_Y}{\sigma_Y}.[/tex] The function
    [itex]f_{Z_1 Z_2}(z1,z2) \equiv g(z_1,z_2)[/itex] is much, much simpler than the original, and then all you need to do is calculate
    [tex] \int_{-\infty}^{\infty}\, \int_{-\infty}^{\infty} z_1 z_2 g(z_1,z_2) \, dz_1 \, dz_2.[/tex] It is fairly easy to do the z2 integral first, then do the z1 integral. Both integrals just rely on already-known values of gaussian integrals.

    RGV
     
    Last edited: Oct 3, 2012
  10. Oct 5, 2012 #9

    fluidistic

    User Avatar
    Gold Member

    Thanks for helping me again. I don't really understand how the change of variable could be helpful nor why what you wrote holds. Nor do I understand why the z_2 integral is easier than the z_1 first, to me they look absolutely symmetric, so the same algebra. I also do not see how we get rid of an integral of the type [itex]\int x e^{x^2}dx[/itex].
     
  11. Oct 5, 2012 #10

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Do whichever one you want first, but pick one and do it first. Why change variables? Because it is 100 times easier to do it in z1,z2 than in x,y. However, it is absolutely NOT NECESSARY: just use the original form if you like doing lots of work and have lots of spare time.

    RGV
     
  12. Oct 6, 2012 #11

    fluidistic

    User Avatar
    Gold Member

    Ok I don't have that much time and I'd rather use the change of variables.
    Please correct me where I'm wrong:
    [tex]f_{Z_1Z_2}(z_1,z_2)=\frac{1}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}}\exp \left [ -\frac{1}{2 (1-\rho ^2)} (z_1 ^2 +z_2 ^2-2\rho z_1 z_2 ) \right ][/tex].
    The first integral would be [tex]\int _{-\infty}^{\infty} \frac{z_1}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}}\exp \left [ -\frac{1}{2 (1-\rho ^2)} (z_1 ^2 +z_2 ^2-2\rho z_1 z_2 ) \right ] dz_1[/tex]. Is this correct so far?
     
  13. Oct 6, 2012 #12

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    It is OK if you eliminate σX and σY in the denominator. The change of variable gets rid of the σ, because dz1 = dx/σX, etc.

    RGV
     
    Last edited: Oct 6, 2012
  14. Oct 6, 2012 #13

    fluidistic

    User Avatar
    Gold Member

    Ok I understand, this is what I thought at first but then... I don't understand the integral.
    Since [itex]z_1=\frac{x-\mu _X}{\sigma _X}[/itex] the integral [tex]\int _{-\infty}^{\infty} \frac{z_1}{2\pi \sigma _Y \sqrt{1-\rho ^2}}\exp \left [ -\frac{1}{2 (1-\rho ^2)} (z_1 ^2 +z_2 ^2-2\rho z_1 z_2 ) \right ] dz_1[/tex] (1) is not equivalent to the integral [tex]\int _{-\infty}^{\infty }\frac{x}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}} \exp \{ \left [ -\frac{1}{2(1-\rho ^2)} \right ] \left [ \left ( \frac{x-\mu _X}{\sigma _X} \right )^2 + \left ( \frac{y-\mu _Y}{\sigma _Y} \right )^2 -\frac{2\rho (x-\mu _X)(y-\mu _Y)}{\sigma _X \sigma _Y} \right ] \} dx[/tex] (2).
    Oh wait, so it would mean that (2) is equivalent to [itex](1) -\frac{f_Y(y)\mu_X}{x}[/itex]?
     
  15. Oct 6, 2012 #14

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    What on Earth are you talking about? Just get rid of the denominator ##\sigma_X \sigma_Y## in your (z1,z2) integral. Have you never studied changes of variables in integration?

    RGV
     
  16. Oct 6, 2012 #15

    fluidistic

    User Avatar
    Gold Member

    I have studied change of variables in integration but I'm stuck at understanding the change of variable you use in post #8.
    From what I understand you are saying that to simplify the computation of [tex]\int _{-\infty}^{\infty} \int _{-\infty}^{\infty} \frac{xy}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}} \exp \{ \left [ -\frac{1}{2(1-\rho ^2)} \right ] \left [ \left ( \frac{x-\mu _X}{\sigma _X} \right )^2 + \left ( \frac{y-\mu _Y}{\sigma _Y} \right )^2 -\frac{2\rho (x-\mu _X)(y-\mu _Y)}{\sigma _X \sigma _Y} \right ] \}dxdy[/tex] it is equivalant to calculate [itex]\int _{-\infty}^{\infty} \int _{-\infty}^{\infty} z_1z_2g(z_1,z_2)dz_1dz_2[/itex] but if you replace [itex]z_1[/itex] and [itex]z_2[/itex] by [itex]\frac{x-\mu _X}{\sigma _X}[/itex] and [itex]\frac{y-\mu _Y}{\sigma _Y}[/itex] respectively, then you get [tex]\int _{-\infty}^{\infty} \int _{-\infty}^{\infty} z_1z_2g(z_1,z_2)dz_1dz_2= \int _{-\infty}^{\infty} \int _{-\infty}^{\infty} \left ( \frac{x-\mu _X}{\sigma _X} \right ) \left ( \frac{y-\mu _Y}{\sigma _Y} \right ) \cdot \frac{1}{2 \pi \sqrt{1-\rho ^2}} \cdot \exp \{ \left [ -\frac{1}{2(1-\rho ^2)} \right ] \left [ \left ( \frac{x-\mu _X}{\sigma _X} \right )^2 + \left ( \frac{y-\mu _Y}{\sigma _Y} \right )^2 -\frac{2\rho (x-\mu _X)(y-\mu _Y)}{\sigma _X \sigma _Y} \right ] \}dxdy[/tex]. Clearly this is not equal to what we started with, namely [tex]\int _{-\infty}^{\infty} \int _{-\infty}^{\infty} \frac{xy}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}} \exp \{ \left [ -\frac{1}{2(1-\rho ^2)} \right ] \left [ \left ( \frac{x-\mu _X}{\sigma _X} \right )^2 + \left ( \frac{y-\mu _Y}{\sigma _Y} \right )^2 -\frac{2\rho (x-\mu _X)(y-\mu _Y)}{\sigma _X \sigma _Y} \right ] \}dxdy[/tex].
    I know I'm missing something but I don't know what.
     
  17. Oct 7, 2012 #16

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    This will be absolutely the last time I respond.
    (1) What is the relation between Cov(X,Y) and Cov(Z1,Z2)?
    (2) What is the formula for Cov(Z1,Z2)?
    (3) The density ##f_{Z_1,Z_2}(z_1,z_2) \equiv g(z_1,z_2)## should not have any parameters ##\sigma_X, \; \sigma_Y## in it, but it will still contain ##\rho.##

    RGV
     
  18. Oct 21, 2012 #17

    fluidistic

    User Avatar
    Gold Member

    Hey guys I'm still stuck on this problem; if someone could push me a little further I'd be happy.
    Okay no problem
    I don't know. :frown:
    I don't know either. I'm asked to find the covariance matrix, namely [itex]\begin{bmatrix} <X^2>-<X>^2 &<YX>-<Y><X> \\ <XY>-<X><Y> & <Y^2>-<Y>^2 \end{bmatrix}[/itex]; I've all the terms except <XY>=<YX> and this is where I'm stuck.
    Yes I noticed this, that's because the [itex]dz_i[/itex]'s contain a [itex]\sigma _x dx[/itex] and [itex]\sigma _y dy[/itex].
     
  19. Oct 21, 2012 #18

    fluidistic

    User Avatar
    Gold Member

    Ok I've made more progresses thanks to Ray Vickson's last post.
    I didn't realize that the coefficient I'm looking for in the matrix, namely [itex]<XY>-<X><Y>[/itex] is the covariance between X and Y (yes I'm a total newbie I know).
    More than this, the covariance between X and Y is definied as [itex]cov(x,y)=\iint dxdy (x-\mu _X)(y-\mu _Y)f_{XY}(x,y)[/itex]. I was not taught this, that's why nothing clicked in my mind.
    So now I UNDERSTAND why the change of variable [itex]z_i=\frac{x_i-\mu _i}{\sigma _i}[/itex] works!!!
    I can answer your question now Ray Vickson: [itex]cov(x,y)=\sigma _X \sigma _Y cov(z_1,z_2)[/itex].
    I have that [itex]cov(z_1, z_2)=\iint \sigma _X \sigma _Y dz_1 dz_2 z_1 z_2 \sigma _X \sigma _Y \cdot \frac{1}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}}\exp \{ \left [ -\frac{1}{2(1-\rho^2)} \right ] (z_1^2+z_2^2-2 \rho z_1 z_2) \}[/itex].
    So that [itex]cov(z_1,z_2)=\frac{\sigma _X \sigma _Y}{2\pi \sqrt{1-\rho ^2}} \int _{-\infty}^{\infty}\int _{-\infty}^{\infty} z_1 z_2 \exp \{ \left [ -\frac{1}{2(1-\rho ^2)} \right ] (z_1^2+z_2^2 -2\rho z_1 z_2 \}[/itex].
    So I can concentrate on the integral of [itex]z_1[/itex] first.
    I have it under the form [itex]\int _{-\infty}^{\infty} z_1 \exp (-az_1^2+bz_1)dz_1[/itex] where [itex]a=\frac{1}{2(1-\rho ^2 )}[/itex] and [itex]b=\frac{\rho z_2}{1-\rho ^2}[/itex].
    I'm having troubles solving this integral. I tried by parts by calling [itex]u=z_1[/itex], [itex]v'=\exp (-az_1^2+bz_1)[/itex] so that [itex]u'=1[/itex] and [itex]v=\frac{1}{2} \sqrt {\frac{\pi}{a}} erf \left ( \frac{2a z_1 -b}{2 \sqrt a} \right ) e^{b^2/(4a)}[/itex].
    And so that integral is worth [itex]uv \big | _{-\infty}^{\infty} - \int _{-\infty}^{\infty} vdu[/itex].
    First huge problem: uv evaluated in infinities is infinite, because the erf function is worth -1 at -infinity and 1 at infinity but the u part ([itex]z_1[/itex]) is not finite there. Hence the integral diverges... which is I know, wrong.
    Second big problem, I don't know a primitive of the erf function.

    Ray Vickson said there is no need to look at integral tables and that it's fairly easy to solve (post #8). I do not see any substitution that could help me and integral by parts shows that it's divergent (I don't know where lies my error) while it's not.
    Any help would be appreciated.
     
  20. Oct 22, 2012 #19

    fluidistic

    User Avatar
    Gold Member

    I'm totally lost guys and exhausted.
    I could solve the integral of the last post.
    At the end I reached that [itex]\iint \sigma _X \sigma _Y dz_1 dz_2 z_1 z_2 \sigma _X \sigma _Y \cdot \frac{1}{2\pi \sigma _X \sigma _Y \sqrt{1-\rho ^2}}\exp \{ \left [ -\frac{1}{2(1-\rho^2)} \right ] (z_1^2+z_2^2-2 \rho z_1 z_2) \} = \sigma _X \sigma _Y \rho \frac{(1-\rho ^2 )^{3/2}}{[1-/(1-\rho ^2) \rho ^2]^{3/2}}[/itex]. It's almost obvious I made at least one error somewhere and that I should have reached [itex]\sigma _X \sigma _Y \rho[/itex]. I know this is worth cov(X,Y) and I thought it was worth cov (Z_1, Z_2) but I don't think it's true now.
    I don't know the relation between cov (X, Y) and cov (Z_1, Z_2).
     
  21. Oct 27, 2012 #20

    fluidistic

    User Avatar
    Gold Member

    Hi guys! I've finnally reached the desired result!!!! (1 min ago!)
    I reached [itex]cov(x,y)=\sigma _X \sigma _Y \rho[/itex]. I just had to be extremely careful and solve 2 "nasty" integrals (one with a very "tough" trick, i.e. rewriting of the integral into something to make a u substitution and the other by parts).
    Problem solved.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Marginal distribution of a Gaussian
Loading...