Solving Difficult Expectation with Normal Distribution

[Hejdun]

Hi everyone,

I have been struggling with an expectation for a while. It is seems very difficult (if not impossible) to find an analytical expression, but all hints and suggestions would be most appreciated. Here goes. I want to find an analytical expression (i.e. solve the integral) for the expectation

E[ X^2*exp(X)/(1+exp(X))^2 ], where X~N(0,σ^2)

That is, X is a r.v. that follows a normal distribution with mean 0 and variance sigma^2, and is then transformed according to the formula in the expectation above. Any hints or suggestions?

Best regards
Hejdun

 

[csopi]

I haven't worked this out fully, but the following might work:

1. the function you integrate is symmetric, so write it as 2* integrate from 0..infinity

2. notice, that exp(x)/(1+exp(x))^2 is just the derivative of -1/(1+exp(x)), and integrate by parts

3. you will end up with something like a polynomial * exp(-x^2/2)/(1+exp(x)). On [0..infinity] you can expand 1/(1+exp(x)) as a geometric series, you get terms like exp(k*x)*exp(-x^2)*x^m, which you can work out easily (k and m are integers).

4. sum it up, you should get pi^2/3. This strange result is due to that after summing up, you should end up with a Riemann zeta function.

 

[mathman]

I haven't worked this out fully, but the following might work:

1. the function you integrate is symmetric, so write it as 2* integrate from 0..infinity

exp(x) is not symmetric.

 

[csopi]

but exp(x)/(1+exp(x))^2 is symmetric...

and just to make it cleaner: in 3. you must do some algebra before expanding:

1/(1+exp(x))=exp(-x) * 1/(1+exp(-x)), and now you can expand the latter term

 

[Hejdun]

Thanks for the replies! I am still working on this integral, but it seems that the expectation is infinite. I will be back giving a more thorough explanation.

 

[mfb]

I would be really surprised if the expectation value is not finite. The denominator is bounded below by 1, so the expectation value is smaller than E(X^2*exp(X)), which gives an integral over x^2*e^(-x^2+x) (neglecting prefactors), and that is finite.

 

[csopi]

It is finite of course , but I was wrong with the pi^2 / 3. If i got time for tomorrow, I'll give it a try.

 

[Hejdun]

Alright, I was wrong regarding the expectation not being finite. Still, I have tried various things but are not able to solve it. Not even with the hints above, trying to do integration by parts.

Due to the function being even, I used focused on the positive part of the domain and instead of the normal pdf, I used the pdf of the half-normal distrubution. Plugging this into Mathematica gave however an expression consisting of hypergeometric functions which looked rather messy.

Do you think it is possible to approximate the function X^2*exp(X)/(1+exp(X))^2 (which looks pretty nice when only considering the negative or the positive part of the domain) with some other function to make the analytical results more convenient to work with?

I have also been plotting the expectation as a function of the variance (maybe other higher moments should be used instead). Anyway, this function is unimodal, with the left part being concave and the right part first being concave and then after the inflexion point being convex for the rest of the domain when the variance grows large.

Edit: It seems that the pdf of normal distribution with mean 0 and variance 1.6 (approximately) approximates the function exp(X)/(1+exp(X))^2 quite well. Can this be used somehow?

 

[Hejdun]

I tried the pdf of a normal distribution instead of exp(X)/(1+exp(X))^2 and it seems to work quite well.

 

[Herick]

Hello Hejdun:

Just by curiosity, this inquiry follows from a model that you are working with? What is the interpretation of the x^2*exp(x)/(1+exp(x))^2?

 

[Hejdun]

Hello Hejdun:

Just by curiosity, this inquiry follows from a model that you are working with? What is the interpretation of the x^2*exp(x)/(1+exp(x))^2?

It is a component in the covariance matrix of a logistic regression following a specific data generating process.

Anyway, I solved the problem somehow by approximating exp(x)/(1+exp(x))^2 with a mixed normal distribution. This works terrific actually! The "only" issue now is how to select the specific parameter values for this mixed normal so that exp(x)/(1+exp(x))^2 is approximated good enough. But that is a question for the Calculus section, since this was not trivial.

 

[Herick]

Just wondering, did you get a result near
$$ \frac { \sqrt { 2 } { e }^{ \frac { { \pi }^{ 2 } }{ 2\quad { \sigma }^{ 2 } } }\pi \left( { \pi }^{ 2 }+2{ \sigma }^{ 2 } \right) { \pi }^{ 2 } }{ { \left( { \sigma }^{ 2 }\pi \right) }^{ \frac { 3 }{ 2 } } } $$?

 

[Hejdun]

Just wondering, did you get a result near
$$ \frac { \sqrt { 2 } { e }^{ \frac { { \pi }^{ 2 } }{ 2\quad { \sigma }^{ 2 } } }\pi \left( { \pi }^{ 2 }+2{ \sigma }^{ 2 } \right) { \pi }^{ 2 } }{ { \left( { \sigma }^{ 2 }\pi \right) }^{ \frac { 3 }{ 2 } } } $$?

Not exactly the same but an expression somewhat similar. Does your solution work? This is when substituing exp(x)/(1+exp(x))^2 with normal pdf?

 

[Herick]

Dear Hejdun

This is kind of a guess solution by using residue methods (integration in the complex plane). I am not sure about how to select the right contour. First, I transformed your initial integral to a form with a rational expression inside. The resulting integrand has a 'pole' of order two and then proceed to take the limit as usual in this technique. Please, check numerically with yours to see how close it is with my solution. By the way, I noticed that it is almost unreadable the numerator in the exponent...it is \pi^2. If the solutions are close, I will make this more rigorous and send you a complete explanation. My guess is that I am not selecting correctly the contour in the complex plane. This expression has a square of a Log! And Log is multivariate in the complex plane. Anyway, here is how I started...

Let $$\quad y={ e }^{ x }\quad$$ then $$\quad dy={ e }^{ x }dx\quad and\quad I=\int _{ 0 }^{ \infty }{ \frac { { \left( \ln { y } \right) }^{ 2 }{ e }^{ \frac { -{ \left( \ln { y } \right) }^{ 2 } }{ 2\quad { \sigma }^{ 2 } } } }{ { \left( 1+y \right) }^{ 2 }\sqrt { 2\pi { \sigma }^{ 2 } } } dy }$$ .

So, I though that at z=-1, the integrand has a removable singularity.