Statistical independence, orthogonality and uncorrelatedness

[dexterdev]

A doubt on statistical independence , orthogonality and uncorrelatedness ?

Hi friends,
I wanted to make my concepts on statistical independence, uncorrelatedness and orthogonality clear. Suppose I have 2 random variables x and y. I have 2 pictures on the above concepts, is it correct ? which is more general picture? If any mistake is there please point it out.


What is statistical independence and linear independent independence means? Are they same?

Do pdf(x,y)=pdf(x)*pdf(y) always imply E(XY)=E(X)E(Y). Can anyone please explain that?

-Devanand T

 

[Stephen Tashi]

You'd get better answers if you only asked one important question per thread!

What do you mean by "independent independence"? Are you referring to indpendence of vectors in a vector space? That type of indpendence is not identical to statistical indpendence merely because independence in vector spaces is defined without any reference to probability or expected values.

It is possible for a person to view a set of random variables as vectors in various ways.

So the relevant question is "Can I view random variables as vectors in such a way that
a set of random variables is statistically independent if and only if the set is independent as a set of vectors?". I can't answer that. I'll have to think about it.

It is more common to view random variables as vectors in such as way that two random variables are uncorrelated if and only if they are orthogonal as vectors. I think the general idea is that you view a set of random variables as forming, not only a vector space, but a vector space with an inner product. I think the inner product is the covariance. If you do things this way, the "natural" addition of random variables becomes the way you add them as vectors. The "natural" way you multiply a random variable by a constant becomes the way you multiply it as a vector by a scalar.

There could be other ways ot viewing random variables as vectors where addition and scalar multiplication in the vector space have a different definition that the "natural" one. That's why it's hard to answer the question "Can I view random variables as vectors in such a way that
a set of random variables is statistically independent if and only if the set is independent as a set of vectors?". You can't automatically say "no".

 

[haruspex]

In stats, orthogonal generally means the same as uncorrelated, but only used in the context of two r.v.s which affect a third. The point is that if they're uncorrelated you can run the two regressions independently. See http://en.wikipedia.org/wiki/Orthogonality#Statistics.2C_econometrics.2C_and_economics

If pdf(x,y)=pdf(x)*pdf(y), E(XY)=∫∫xy.pdf(x,y).dx.dy=∫∫xy.pdf(x)*pdf(y).dx.dy=∫∫x.pdf(x).dx*y.pdf(y).dy=∫x.pdf(x).dx*∫y.pdf(y).dy=E(X)E(Y).

 

[dexterdev]

Thanks guys... will ask one question per thread