Bivariate normal distribution

[learner928]

Quick question on bivariate normal distribution please:

I know for a bivariate normal distribution, the two variables are marginally normal and all the conditional distributions are also normal.

Is the reverse true?

i.e. if you have two random variables that are marginally normal themselves and all the conditional distributions of one variable given a value of the other are also normal, does this result in the joint distribution has to be "bivariate normal" or it can still be another type of joint distribution?

thanks.

 

[viraltux]

Quick question on bivariate normal distribution please:

I know for a bivariate normal distribution, the two variables are marginally normal and all the conditional distributions are also normal.

Is the reverse true?

i.e. if you have two random variables that are marginally normal themselves and all the conditional distributions of one variable given a value of the other are also normal, does this result in the joint distribution has to be "bivariate normal" or it can still be another type of joint distribution?

thanks.

In fact as long as they both are normal (and independent) you are done; the joined distribution will be bivariate normal.

 

[mathman]

In fact as long as they both are normal (and independent) you are done; the joined distribution will be bivariate normal.

Independence is unnecessary.

 

[viraltux]

Independence is unnecessary.

I didn't say it is mandatory, but if it is there then you are sure you have a jointly bivariate normal, if it is not then you have to prove it.

 

[learner928]

thanks guys, just to be clear,

even if they are not independent, is the joint distribution has to be bivariate normal?

no need for proof.

 

[viraltux]

thanks guys, just to be clear,

even if they are not independent, is the joint distribution has to be bivariate normal?

no need for proof.

Well, as I said, if X and Y are normally distributed but not independent, they might be or might be not jointly normally distributed; you need to prove it per each case.

 

[learner928]

on top of X and Y are normally distributed but not independent,

if all the conditional distributions are also normal, does that mean X and Y are definitely jointly normal?

 

[viraltux]

on top of X and Y are normally distributed but not independent,

if all the conditional distributions are also normal, does that mean X and Y are definitely jointly normal?

I don't think so, the moment you are allowed to create dependencies between X and Y you can always look for a freak relationship that breaks the definition of the bivariate jointly distribution.

 

[learner928]

thanks,

so you don't think the fact not only all conditional distributions are normal, X and Y are also marginally normal, this is not enough to restrict the joint distribution to be bivariate normal?

 

[viraltux]

thanks,

so you don't think the fact not only all conditional distributions are normal, X and Y are also marginally normal, this is not enough to restrict the joint distribution to be bivariate normal?

Nah, I don't think so. The definition of a multivariate normal distribution is not simple, one of the condition it has to follow (among other more complex than this one) is that every linear combination of its components is also normally distributed .

You could try to further constraint X and Y to behave in such a way that a particular linear combination of their values would not behave normally even if for every particular value of X and Y they do (which is basically what your restriction does). And if that does not work you could still try to mess with the other conditions.

 

[learner928]

thanks,

i don't have the proof, but I think the fact because X and Y are also marginally normal (on top of all conditional distributions are normal), this extra condition does make every linear combination of X and Y normally distributed hence bivariate normal, you don't think that is the case?

any idea how to proof?

 

[learner928]

Mathman do you know who is correct?

 

[viraltux]

thanks,

i don't have the proof, but I think the fact because X and Y are also marginally normal (on top of all conditional distributions are normal), this extra condition does make every linear combination of X and Y normally distributed hence bivariate normal, you don't think that is the case?

any idea how to proof?

It really looks like that, does it? Your restrictions enforce a good amount of independence between X and Y and make it difficult to find dependencies between X and Y to break the bivariate normality but, anyway, how about this one:

Imagine that given X=x the variance of the normal conditional distribution of Y is inversely proportional to x, and also imagine that the variance of X condition to Y=y is also inversely proportional to y.

Now you have X,Y, X|Y=y and Y|X=x following normal distributions, but you are getting in the bivariate distribution a contour line that looks nothing like an ellipse which is what you would expect if it would follow a bivariate joint normal distribution... yeah? I let for you the fun to do the formal proof though :tongue:

 

[mathman]

Mathman do you know who is correct?

I haven't looked at the question in detail, but my instinct tells me that with all the conditions that are imposed the joint distribution should be normal.

I know for a bivariate normal distribution, the two variables are marginally normal and all the conditional distributions are also normal.

 

[SW VandeCarr]

With the bivariate normal PDF, the variables may be correlated. Jointly normal random variables are uncorrelated and are a special case of the bivariate normal PDF.

http://athenasc.com/Bivariate-Normal.pdf

 

[alan2]

I've been reading this thread for a couple of days and I also found the above referenced article. As I recall learning once, the missing condition is that the conditional variances must be constant in addition to the conditions given in the original post. I can't find any reference to this and now I'm wondering if it equivalent to the description given in this article or I'm just wrong. Any ideas?

 

[learner928]

viraltux, your example doesn't work as it breaks one of the restrictions.
In your case, as soon as correlation is not zero (the mean of the normal conditional distributions shift), the resultant marginal distribution is no longer normal.

In fact, the more I think about it, it does seem the only way of keeping marginal distribution of Y normal is for the normal conditional distributions of Y for each value of X follow the bivariate normal formula, with the mean shifting slightly and variance the same.
As soon as you change the variance of these conditional distributions or shift the mean in a different way, the resultant marginal distribution of Y is no longer normal.

Therefore I do still think that given the restriction of both X and Y being marginally normal and all conditional distributions being also normal, it does mean that X and Y has to be jointly normal and the joint distribution can not be anything else.

Would be great if someone can confirm this or convince otherwise.

Thanks.

 

[viraltux]

viraltux, your example doesn't work as it breaks one of the restrictions.
In your case, as soon as correlation is not zero (the mean of the normal conditional distributions shift), the resultant marginal distribution is no longer normal.

...

As soon as you change the variance of these conditional distributions or shift the mean in a different way, the resultant marginal distribution of Y is no longer normal.
Thanks.

I disagree...

First, two variables can be correlated and still behave normally, so I am not sure what you mean by "as soon as correlation is not zero... resultant marginal distribution is no longer normal."

Second, the example I gave results into a symmetric bivariate distribution, if you accept X|Y=y as normal (and it seems you do) you have to accept Y|X=x as normal too.

 

[learner928]

In your example, I believe you are saying the conditional normal distributions having different variance? If that is the case, I don't see how the resultant marginal distribution (from adding all the conditional ones) can still be normal.

Doesn't the conditional distributions need to have the same variance to be able to add up to a normal distribution?

 

[viraltux]

In your example, I believe you are saying the conditional normal distributions having different variance? If that is the case, I don't see how the resultant marginal distribution (from adding all the conditional ones) can still be normal.

Doesn't the conditional distributions need to have the same variance to be able to add up to a normal distribution?

Yes, that is what I am saying... The marginal of Y is still normal because I also force the variances of its conditional distributions to change likewise. To understand how the marginal distributions are still normal think about the symmetry of the bivariate distribution.

You could make up more convoluted examples but in this one you only need to see the symmetry, turn the plot 90º and you'll have exactly the same bivariate plot, so you see, if you have no problem accepting X|Y=y as normal, then you should not have any problem accepting Y|X=x as normal too.

 

[learner928]

Thanks viraltux, just to confirm my understanding, in your example:

"Imagine that given X=x the variance of the normal conditional distribution of Y is inversely proportional to x, and also imagine that the variance of X condition to Y=y is also inversely proportional to y.
Now you have X,Y, X|Y=y and Y|X=x following normal distributions, but you are getting in the bivariate distribution a contour line that looks nothing like an ellipse which is what you would expect if it would follow a bivariate joint normal distribution... yeah? I let for you the fun to do the formal proof though"

Two things I am not sure about:

1) by setting the variance of the normal conditional distribution of Y inversely proportional to x, is the resultant marginal distribution of Y (by adding up the conditional ones) really still normal despite the changing variance?

2) in your setting, are you sure the joint distribution is no longer bivariate normal, ie. the distribution of Z=X+Y is now not normal?

also since you didn't mention about the mean of the conditional normal distributions, guess your example is only dealing with correlation of zero, I think to create non-zero correlation, we need to shift the mean of the conditional distribution right?

thanks

 

[viraltux]

Thanks viraltux, just to confirm my understanding, in your example:

Two things I am not sure about:

1) by setting the variance of the normal conditional distribution of Y inversely proportional to x, is the resultant marginal distribution of Y (by adding up the conditional ones) really still normal despite the changing variance?

2) in your setting, are you sure the joint distribution is no longer bivariate normal, ie. the distribution of Z=X+Y is now not normal?

also since you didn't mention about the mean of the conditional normal distributions, guess your example is only dealing with correlation of zero, I think to create non-zero correlation, we need to shift the mean of the conditional distribution right?

thanks

Hi learner,

1) For every Y regardless the value of X, Y is normal, and the same goes for X. So you would be adding up normal distributions all along.

2) Yes, I am sure. For a particular linear combination like X+Y we should check, but I am sure because in a bivariate normal distribution you would expect elliptical contour lines, and the example I gave you has a star-like shape. I made a simulation for you to see the shape and the symmetries I am talking about.

In this example all normal distributions are independent from each other, have μ=0 and their variances change along the axis as described.

 

[learner928]

Hi Viraltux,

Can you please tell me within the sample simulation you made, what are the formulae for the change of the variance of the conditional normal distributions?

Thanks a lot!

 

[viraltux]

Hi Viraltux,

Can you please tell me within the sample simulation you made, what are the formulae for the change of the variance of the conditional normal distributions?

Thanks a lot!

It was something like N(0,1/(0.1+abs(x))) , but if you make it proportional N(0,0.1+abs(x)) you also get non elliptical contour lines.

 

[learner928]

If you use that for the change of variance, I think the resultant marginal distribution of Y is no longer normal and would have very high kurtosis (more squeezed around the mean than a normal).

reason is:

If you think in the case of a proper bivariate normal distribution, for each X=x, only by keeping the conditional normal distribution of Y all have the same variance, adding these up produces a marginal distribution of Y which is normal.
Now the only thing we are changing to arrive at your case is to reduce the variance as X goes away from 0 (on both positive and negative side), then naturally the end marginal distribution of Y would be more squashed towards the middle and not be normal anymore?

 

[viraltux]

If you use that for the change of variance, I think the resultant marginal distribution of Y is no longer normal and would have very high kurtosis (more squeezed around the mean than a normal).

reason is:

If you think in the case of a proper bivariate normal distribution, for each X=x, only by keeping the conditional normal distribution of Y all have the same variance, adding these up produces a marginal distribution of Y which is normal.
Now the only thing we are changing to arrive at your case is to reduce the variance as X goes away from 0 (on both positive and negative side), then naturally the end marginal distribution of Y would be more squashed towards the middle and not be normal anymore?

Why would adding Normal distributions would give you anything but another Normal distribution? And why would the kurtosis change at all if for any Normal distribution, regardless its variance, the kurtosis is 0?

 

[learner928]

Hi Viraltux,

I think in this case we are not actually doing a sum of normal distributions of independent random variables (which I agree would be normal).

What we are doing is superimposing the conditional normal distributions on each other to get to the marginal distribution of Y.

Let's use an example,

Say X has a probability of 0.5 equal to either x1 or x2.

For X=x1 and X=x2, the conditional distribution of Y is normal with different volatility of say 50% and 10%.

Clearly you can see by superimposing these two conditional distributions of Y, the resultant marginal distribution of Y is clearly not normal as the smaller vol one increases the probability around the mean by more than a normal distribution would.
Marginal distribution of Y would only be normal if the conditional distributions for X=x1 and X=x2 have the same volatility.

hence i think in your case of conditional distributions having different volatilities, the resultant marginal distribution of Y is no longer normal.

Please let me know what you think is wrong above?

Thank you very much again.

 

[viraltux]

Hi Viraltux,

I think in this case we are not actually doing a sum of normal distributions of independent random variables (which I agree would be normal).

What we are doing is superimposing the conditional normal distributions on each other to get to the marginal distribution of Y.

Let's use an example,

Say X has a probability of 0.5 equal to either x1 or x2.

For X=x1 and X=x2, the conditional distribution of Y is normal with different volatility of say 50% and 10%.

Clearly you can see by superimposing these two conditional distributions of Y, the resultant marginal distribution of Y is clearly not normal as the smaller vol one increases the probability around the mean by more than a normal distribution would.
Marginal distribution of Y would only be normal if the conditional distributions for X=x1 and X=x2 have the same volatility.

hence i think in your case of conditional distributions having different volatilities, the resultant marginal distribution of Y is no longer normal.

Please let me know what you think is wrong above?

Thank you very much again.

Ah, OK, I see what you mean, yeah, the conditional distributions would be normal but the marginal would
not. Yet, if you rotate the star 45º you solve these problems, check this:

http://demonstrations.wolfram.com/MarginalNormalityDoesNotImplyBivariateNormality/

So the bottom line is that normal marginal distributions does not necessarily mean bivariate normal.

Edit: Before you ask, I don't know any counter example when normal conditional distributions and normal marginal distribution occur together... sooo :tongue:

 

[learner928]

cool Viraltux, just so that we are on the same page,

we agree that marginal normal distributions as a condition alone do not have to result in only bivariate normal.

but if we add an extra restriction that all conditional distributions are also normal, you are also not sure whether this will now limit the joint distribution has to be bivariate normal or it can still be something else?

 

[viraltux]

cool Viraltux, just so that we are on the same page,

we agree that marginal normal distributions as a condition alone do not have to result in only bivariate normal.

but if we add an extra restriction that all conditional distributions are also normal, you are also not sure whether this will now limit the joint distribution has to be bivariate normal or it can still be something else?

Yeah, we are on the same page now, but not sure 100% that these two conditions together means bivariate normal, I feel tempted to say yes though, but unless I see the formal prove showing that these two conditions imply every other condition in the definition of bivariate normal I'd be cautious; for instance, singular matrices are allowed in the definition in which case you would have distributions without density, so there might always be a counter example around the corner.

Anyway if someone has the proof that these two conditions are an equivalent definition of the bivariate normal I want to know too

 

[learner928]

Hi Viraltux,

the more I think about it, it does seem the only way of keeping marginal distribution of Y normal is for the normal conditional distributions of Y for each value of X follow the bivariate normal formula, with the mean shifting slightly and keeping variance the same.
As soon as you change the variance of these conditional distributions or shift the mean in a different way, the resultant marginal distribution of Y is no longer normal which would break our condition.

do you think so?

 

[Dickfore]

Hey, OP, if the joint probability density function is $\phi(x, y)$, then, how would you express the conditional probability for x, assuming $Y = y_{0}$?

 

[viraltux]

Hi Viraltux,

the more I think about it, it does seem the only way of keeping marginal distribution of Y normal is for the normal conditional distributions of Y for each value of X follow the bivariate normal formula, with the mean shifting slightly and keeping variance the same.
As soon as you change the variance of these conditional distributions or shift the mean in a different way, the resultant marginal distribution of Y is no longer normal which would break our condition.

do you think so?

Maybe... or Maybe you can figure out a way to change the variance through the axis in a way that keeps the marginal normal. For instance, increasing the variance is going to create kurtosis in the marginals as we previously discussed, but how about if you increase the variance for a while and then you reduce it back in a way to amount for the increased kurtosis, would you get a normal marginal? So yeah, until I don't see a formal prove I'll keep my healthy 'I don't know' flashing