How to view conditional variance intuitively?

[yamata1]

We have a sample of X, a Normalized Gaussian random variable.We divide the data into positive and negative.
Each will have a conditional variance of $1−\frac{2}{π}$ .
Can someone show how to get this result ?

I found this problem here (page 3) : https://www.dropbox.com/s/18pjy7gmz0hl6q7/Correlation.pdf?dl=0

Thank you.

 

[Stephen Tashi]

The title of the thread speaks of viewing something intuitively but your question

Can someone show how to get this result ?

seems ask for a mathematical derivation.

The relevant passage in the document concerns forming a correct intuition about conditional variances - the correct intuition being that they need not be larger (or smaller) than the variance of the original distribution that is constrained.

The problem becomes much easier when we consider the behavior in lower dimensions for Gaussian variables. The intuition is as follows. Take a sample of X, a Normalized Gaussian random variable. Verify that the variance is 1. Divide the data into positive and negative. Each will have a conditional variance of 1−2/π=≈0.363. Divide the segments further, and there will be additional drop in variance. And, although one is programmed to think that the tail should be more volatile, it isn’t so; the segments in the tail have an increasingly lower variance as one gets further away, see in Fig.4.

 

[yamata1]

The title of the thread speaks of viewing something intuitively but your question

seems ask for a mathematical derivation.

The relevant passage in the document concerns forming a correct intuition about conditional variances - the correct intuition being that they need not be larger (or smaller) than the variance of the original distribution that is constrained.

I am failing to understand the way to get this answer . Is there a simple computation or property that easily gives this answer ? I tried the formula for $v_x(Q)$ without success.

 

[Stephen Tashi]

Consider the case when the random variable $X$ has a probability density given by $g(x) = 2 \frac{1}{\sqrt{2 \pi} } e^{-{x^2/2}}$ for $x \ge 0$.

The variance of $X$ is $\sigma^2_X = \int_0^\infty x^2 g(x) dx -( \int_0^{\infty} x g(x) dx)^2$

$\int_0^\infty x^2 g(x) dx = \int_0^\infty x^2 2 \frac{1}{\sqrt{2 \pi}} e^{-{x^2/2}} dx$
$= 2 \int_0^\infty x^2 \frac{1}{\sqrt{2 \pi}} e^{-{x^2/2}} dx$
$= \int_{-\infty}^{\infty} x^2 \frac{1}{\sqrt{2 \pi}} e^{-{x^2/2}} dx = 1$
since the last integral is the same as computing the variance of a normal distribution that has mean zero and variance 1.

$\int_0^\infty x g(x) dx = \int_0^\infty x (2 \frac{1}{\sqrt{2 \pi}} e^{-{x^2/2}}) dx$
$= ( -2 \frac{1}{\sqrt{2 \pi}} e^{-{x^2/2}}) |_0^\infty$
$= 0 - ( -2 \frac{1}{\sqrt{2 \pi}})$
$= \frac{\sqrt{2}}{\sqrt{\pi}}$

So $\sigma^2_X = 1 - (\frac{\sqrt{2}}{\sqrt{\pi}})^2 = 1 - 2/\pi$