Uncorrelated Vs. Independent variables

[musicgold]

Hi,

I am confused with respect to these two terms. In a book on regression analysis, I read the following statements.

1. For two normally distributed variables, zero covariance / correlation means independence of the two variables.

2. With the normality assumption, the following equation means that $$\mu_i$$ and $$\mu_j$$ are NOT ONLY uncorrelated BUT ALSO independently distributed.


$$\left \mu_i - N (0, \sigma^2 \right)$$

Not able to get the wiggly line (~) after ui

I am trying to understand if it is possible to have two variables that are
(a) uncorrelated, and not-independent.
(b) uncorrelated and independent
(c) correlated and not-independent
(d) correlated and independent

I would appreciate it if you could explain each type with one example.

Thanks

MG.

 

[statdad]

If the variables are normally distributed, then correlation is zero if and only if they are independent. (By the way, instead of not-independent you should say dependent .

In general, if $$X, Y$$ are independent, their correlation is zero, since

$$E[(X-\mu_X)(Y-\mu_Y)] = E[X-\mu_X] \cdot E[Y - \mu_Y] = 0$$

so the correlation will be zero.

For uncorrelated but dependent, consider this somewhat classic example. Assume $$X$$ has a standard normal distribution, let $$W$$ be independent of $$X$$ and $$P(W=1) = 1/2 = P(W = -1)$$. Set

$$Y = W X$$

With a little work you can find that

a) $$Y$$ and $$X$$ are not correlated

b) $$Y$$ has a standard normal distribution (calculate $$P(Y \le y) = E[P(Y \le y \mid W)] =E[P(X \le y \mid W)]$$, and use both the definition of W and the fact that W, X are independent


For correlated and dependent - look at any multivariate normal distribution with non-zero correlations.

Correlated and independent. Let $$X$$ be uniformly distributed on $$[-1, 1]$$ and let $$Y = X^2$$.

These two variables are not independent, since $$Y$$ is determined by $$X$$, but they are uncorrelated.
c) $$X$$ and $$Y$$ are dependent.

 

[g_edgar]

Summary ... (d) is impossible. If X and Y are independent, then X and Y are uncorrelated.

The other three are all possible.

However, when the RVs are normal, (a) is also impossible. For normal random variables X and Y, we have: X and Y are independent if and only if X and Y are uncorrelated.

 

[musicgold]

statdad and g_edgar,

Thanks.

I thought the term 'independent' here was the opposite of 'joint', as in 'jointly distributed'.

Also, in terms of examples, I was looking for more simple explanations. For example, can we say
the Height and Weight variables for a certain population are correlated but independent?

I found some discussion at the end of http://www.ccl.rutgers.edu/~ssi/thesis/thesis-node53.html" web page, but it is not very clear to me.

Thanks,

MG.

 

[g_edgar]

For example, can we say
the Height and Weight variables for a certain population are correlated but independent?

I would not expect them to be independent, since taller people tend to weigh more than shorter people.

 

[statdad]

Some return comments.

statdad and g_edgar,

Thanks.

I thought the term 'independent' here was the opposite of 'joint', as in 'jointly distributed'.

No, variables that are jointly distributed may or may not be independent.

Also, in terms of examples, I was looking for more simple explanations. For example, can we say
the Height and Weight variables for a certain population are correlated but independent?

No - if you take look at a group of people, and measure (say) each person's height and weight, those measured variables will be correlated - as another says, taller people tend to weigh more, but the more central point is that the measurements are taken from the same person.

I found some discussion at the end of http://www.ccl.rutgers.edu/~ssi/thesis/thesis-node53.html" web page, but it is not very clear to me.

Those are good notes, but seem to be (may be - I'm not sure of your mathematical background) more advanced than your current investigations.

 

[gel]

I am confused with respect to these two terms. In a book on regression analysis, I read the following statements.

1. For two normally distributed variables, zero covariance / correlation means independence of the two variables.

No, that's not right. It is not necessary for two uncorrelated and normal variables to be independent. I added a counterexample myself to planetmath website a while ago, http://planetmath.org/encyclopedia/SumsOfNormalRandomVariablesNeedNotBeNormal.html" . Are you sure that your book doesn't add an extra requirement that they are "joint normal"? (which is more than just saying that they are normal.)

Edit: I see statdad's example also showed this, but his post started with "If the variables are normally distributed, then correlation is zero if and only if they are independent." which is wrong, unless by 'normally distributed' he meant 'joint normal'.

 

[musicgold]

Statdad and gel,

Thanks a lot. I think I will do more reading on this topic and come back with my questions, if any.

MG.