Decomposition of a R.V. into dependent and independent parts

[mXSCNT]

Given random variables X and Y, which are not independent, is it always possible to find a random variable W which is independent from X, such that Y = f(X,W), for some function f?

Example: let the joint distribution of X and Y be

 Y  0  1
X+-------
0|1/3 1/6
1|1/6 1/3

Then if we let W have the Bernoulli distribution with p = 1/3, and f(x,w) = x XOR w, we have
y = f(X,W).

 

[gel]

yes. You can prove it as follows.

Define,
g_X(y) = P(Y<=y|X)
which gives the cumulative distribution function of Y conditional on X.
Define the variable
U=g_X(Y)
As long as g_X is continuous (so Y has continuous distribution) then U will be uniformly distributed on [0,1] independently of X.

Setting f(x,u)=g_X^{-1}(u) gives Y=f(X,U).

If g_X is not continuous then you can still set Y=f(X,U) for a r.v. U uniform on [0,1] independently of X, and X,Y will have the correct joint distribution. Note that this can require enlarging the probability space in order to introduce U.

Hopefully you can fill in the gaps in that brief proof.

 

[quadraphonics]

Given random variables X and Y, which are not independent, is it always possible to find a random variable W which is independent from X, such that Y = f(X,W), for some function f?

Depends on what you mean by "=" in the above equation. You can definitely find such a W and f such that f(X,W) has the same distribution as Y (in fact, you don't even need an X here at all), or even such that (X,Y) and (X,f(X,W)) have the same joint distribution.

But if by "=" you mean "almost surely equal," then, no, such a result would generally have to depend on X. I.e., you'd use W = Y-X, and f(X,W) would be X+W. But that is clearly NOT independent of X. Simply put, there's generally no way that you can replace the outcome of a random variable with the outcome of an independent random variable, and hope to get the same answer all of the time.

 

[mXSCNT]

For the discrete case: Let A be the domain of X and B be the domain of Y. Let W be a random variable defined on B^A, where P(W[x] = y) = P(Y=y | X=x). Let f(x,w) = w[x]. Then P(f(X,W) = y) = P(W[X] = y) = P(Y=y | X), which is the distribution of Y.