Calculating Conditional Expectation for IID Normal Variables

[redflame34]

If I have x1,x2 iid normal with N(0,1)

and I want to find E(x1*x2 | x1 + x2 = x)
Can I simply say: x1 = x - x2 and thus
E(x1*x2 | x1 + x2 = x) =
E[ (x - x2)*x2) = E[ (x * x2) - ((x2)^2) ] <=>

x*E[x2] - E[x2^2] =
0 - 1 =
-1?

 

[mathman]

No.
You need to transform the problem to one where x1 + x2 is a random variable.
Let u=(x1 + x2)/√2, v=(x1-x2)/√2. (I am using √2 so that u and v will be iid N(0,1))
Then x1=(u+v)/√2 and x2=(u-v)/√2

Your original problem is now E((u² - v²)/2 |u=x/√2)
Since u and v are independent, the result is x²/4 -1/2.

 

[kasraa]

I'm confused. Can you please explain more why the first solution (-1) is wrong?
X_1 and X_2 are 2 RVs, and we're told that X_1+X_2=y.

1) I understand that sum of two RVs will be a RV, but here, when we are told the sum is y, it is something like a constant? (a realization of Y that is Y=y I mean.)

2) if I'm right about (1), then when we are told the sum is y, it means we can focus only on one RV (accurately, I mean we can think we have two RVs: y-X_1 and X_1). Now the solution as redflame said will be:

\int_{x_1} (y-x_1)(x_1) p(x_1) \ud x_1 = \ldots = -1
(I'm not sure if the TeX code above is shown correctly, it is (\int_{x_1} (y-x_1)(x_1) p(x_1) \ud x_1 = \ldots = -1), (the dx_1 is not shown correctly for me).

Thanks in advance. (;

 

[mathman]

The argument against this approach is complicated, but a simple illustration will show that it is wrong.
Let x1 be N(0,a) and x2 be N(0,b), where a and b are different. Then we have the following:
E(x1*x2|x1+x2=x)=E(x*x2)-E(x2^2)=-b
E(x1*x2|x1+x2=x)=E(x1*x)-E(x1*2)=-a
This is obviously incorrect!

 

[kasraa]

Thanks.
I'm really confused!
Can you please suggest a solution through the definition of E{x1*x2|x1+x2=x}?
I want to correct my beliefs about this kind of problems (working with sum of RVs and ...).
I think E{x1*x2|x1+x2=x}=double integral on x1 and x2 { x1 * x2 * p(x1,x2|x1+x2=x) *dx1 *dx2}

(BTW, is it true?)
Can you continue it?

 

[kasraa]

and please tell me if you know a good reference for understanding these subjects. I am trying to fix my probability/statistics background knowledge in order to understand stochastic processes and estimation theory deeply.

 

[mathman]

The basic problem in your original approach is that by using x2=x-x1, you were redefining x2. This new x2 is N(x,1) not N(0,1) and is no longer independent of x1.

As for texts I haven't been keeping up (I'm retired). I hope someone else can help.

 

[mathman]

I finally understand the underlying problem with your original approach. x1 and x2 are N(0,1) when there is no condition. However their distributions change when you impose the condition x1 + x2=x.

I can give you a very simple example to show what happens. Toss a fair coin twice, and for this analysis, let heads be called +1 and tails -1. Then each toss has mean 0 and variance 1.
Now impose the condition the sum of the tosses = 2. The resulting conditional distribution for both tosses is now +1, so the conditional means are now 1 and the conditional variances are 0.