Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Uncorrelated Vs. Independent variables

  1. Jun 29, 2009 #1
    Hi,

    I am confused with respect to these two terms. In a book on regression analysis, I read the following statements.

    1. For two normally distributed variables, zero covariance / correlation means independence of the two variables.

    2. With the normality assumption, the following equation means that [tex] \mu_i [/tex] and [tex] \mu_j [/tex] are NOT ONLY uncorrelated BUT ALSO independently distributed.


    [tex] \left \mu_i - N (0, \sigma^2 \right) [/tex]

    Not able to get the wiggly line (~) after ui

    I am trying to understand if it is possible to have two variables that are
    (a) uncorrelated, and not-independent.
    (b) uncorrelated and independent
    (c) correlated and not-independent
    (d) correlated and independent

    I would appreciate it if you could explain each type with one example.


    Thanks

    MG.
     
    Last edited: Jun 29, 2009
  2. jcsd
  3. Jun 29, 2009 #2

    statdad

    User Avatar
    Homework Helper

    If the variables are normally distributed, then correlation is zero if and only if they are independent. (By the way, instead of not-independent you should say dependent .

    In general, if [tex] X, Y [/tex] are independent, their correlation is zero, since

    [tex]
    E[(X-\mu_X)(Y-\mu_Y)] = E[X-\mu_X] \cdot E[Y - \mu_Y] = 0
    [/tex]

    so the correlation will be zero.

    For uncorrelated but dependent, consider this somewhat classic example. Assume [tex] X [/tex] has a standard normal distribution, let [tex] W [/tex] be independent of [tex] X [/tex] and [tex] P(W=1) = 1/2 = P(W = -1) [/tex]. Set

    [tex] Y = W X
    [/tex]

    With a little work you can find that

    a) [tex] Y [/tex] and [tex] X [/tex] are not correlated

    b) [tex] Y [/tex] has a standard normal distribution (calculate [tex] P(Y \le y) = E[P(Y \le y \mid W)] =E[P(X \le y \mid W)] [/tex], and use both the definition of W and the fact that W, X are independent


    For correlated and dependent - look at any multivariate normal distribution with non-zero correlations.

    Correlated and independent. Let [tex] X [/tex] be uniformly distributed on [tex] [-1, 1] [/tex] and let [tex] Y = X^2 [/tex].

    These two variables are not independent, since [tex] Y [/tex] is determined by [tex] X [/tex], but they are uncorrelated.
    c) [tex] X [/tex] and [tex] Y [/tex] are dependent.
     
  4. Jun 30, 2009 #3
    Summary ... (d) is impossible. If X and Y are independent, then X and Y are uncorrelated.

    The other three are all possible.

    However, when the RVs are normal, (a) is also impossible. For normal random variables X and Y, we have: X and Y are independent if and only if X and Y are uncorrelated.
     
  5. Jun 30, 2009 #4
    statdad and g_edgar,

    Thanks.

    I thought the term 'independent' here was the opposite of 'joint', as in 'jointly distributed'.

    Also, in terms of examples, I was looking for more simple explanations. For example, can we say
    the Height and Weight variables for a certain population are correlated but independent?

    I found some discussion at the end of http://www.ccl.rutgers.edu/~ssi/thesis/thesis-node53.html" web page, but it is not very clear to me.

    Thanks,

    MG.
     
    Last edited by a moderator: Apr 24, 2017
  6. Jun 30, 2009 #5
    I would not expect them to be independent, since taller people tend to weigh more than shorter people.
     
  7. Jun 30, 2009 #6

    statdad

    User Avatar
    Homework Helper

    Some return comments.

    No, variables that are jointly distributed may or may not be independent.
    No - if you take look at a group of people, and measure (say) each person's height and weight, those measured variables will be correlated - as another says, taller people tend to weigh more, but the more central point is that the measurements are taken from the same person.
    Those are good notes, but seem to be (may be - I'm not sure of your mathematical background) more advanced than your current investigations.
     
    Last edited by a moderator: Apr 24, 2017
  8. Jun 30, 2009 #7

    gel

    User Avatar

    No, that's not right. It is not necessary for two uncorrelated and normal variables to be independent. I added a counterexample myself to planetmath website a while ago, http://planetmath.org/encyclopedia/SumsOfNormalRandomVariablesNeedNotBeNormal.html" [Broken]. Are you sure that your book doesn't add an extra requirement that they are "joint normal"? (which is more than just saying that they are normal.)

    Edit: I see statdad's example also showed this, but his post started with "If the variables are normally distributed, then correlation is zero if and only if they are independent." which is wrong, unless by 'normally distributed' he meant 'joint normal'.
     
    Last edited by a moderator: May 4, 2017
  9. Jul 2, 2009 #8
    Statdad and gel,

    Thanks a lot. I think I will do more reading on this topic and come back with my questions, if any.

    MG.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Uncorrelated Vs. Independent variables
Loading...