Simplifying this expression for large values of D

[CuppoJava]

Homework Statement

Show that:
p(r'+e) \cong p(r')*exp( -3e² / 2s² ) for large D

given that:
p(r) = r^D-1 * exp( -r² / 2s² )
r' = sqrt(D) * s

Homework Equations

This arises out of a D-dimensional gaussian distribution and r' is the radius at which the probability is maximized.

The Attempt at a Solution

I've gotten to the point:

p(r'+e) \propto exp( -(r'+e)² / 2s² + (D-1)ln(r'+e) )

And the solution manual says that using the Taylor expansion for ln, will help everything reduce down, but all I get is a mess. What really baffles me is where the dependence on D went.

Thanks very much for your help
-Patrick

 

[foxjwill]

According to Wolfram Alpha, this is true for large s but not for large D.

 

[CuppoJava]

Thanks for helping foxjwill. I'm almost certain that the result is correct though. There's an probabilistic interpretation of it, and it requires that this holds true for large D not s.

Wolfram Alpha, though, seems like an extremely useful tool. I haven't heard of it before. May I ask exactly what you typed into the query box?

Thanks
-Patrick

 

[gabbagabbahey]

For large D, what can you say about the size of \frac{e}{r'}? Use that to Taylor expand the logarithm:

\ln(r'+e)=\ln(r'(1+\frac{e}{r'}))=\ln(r')+\ln(1+\frac{e}{r'})

 

[foxjwill]

First, I typed in

(exp(-3epsilon^2/(2s^2)) *r^(D-1)*exp(-r^2/(2s^2)) /  (r+epsilon)^(D-1)*exp(-(r+epsilon)^2/(2s^2)) )

Then I copied the result given in the Alternate form box:

r^(D-1) (r+epsilon)^(1-D) e^(-(r^2+r epsilon+2 epsilon^2)/s^2)

Finally, I pasted that into the input box, replaced each instance of r with "(s*sqrt(D))", then prepended "limit" and appended "as D->infinity":

 limit (s*sqrt(D))^(D-1) ((s*sqrt(D))+epsilon)^(1-D) e^(-((s*sqrt(D))^2+(s*sqrt(D)) epsilon+2 epsilon^2)/s^2) as D->infinity

Wolfram Alpha says this equals 0.

 

[Dick]

Something in this isn't quite right. If you work out the expansion around sqrt(D)*s you get a linear term in e. That's because the maximum value of your p(r) is at sqrt(D-1)*s NOT sqrt(D)*s, isn't it? Why don't you expand around the maximum? And I also don't get exp(-3*e^2/s^2) for the correction factor several different ways. I get just exp(-e^2/s^2).

 

[CuppoJava]

Thank you very much for your help.
I haven't gotten then answer actually. It's a textbook question that asks me to prove the statement. (Exercise 1.20 from Pattern Recognition and Machine Learning by Bishop).

You're right. The maximum is actually at sqrt(D-1)*s, which the solution manual says can be approximated to sqrt(D)*s for large D. I will try expanding around sqrt(D-1)*s and see if I can get it.

Thank you for helping me.
-Patrick

 

[CuppoJava]

Hi Dick,
Would you mind showing me how to arrived at exp(-e^2/s^2)?
I've tried expanding around r = sqrt(D-1)*s
and by noticing that (D-1)*s^2 = r^2

I simplified the equation to
p(r+e) approx. = exp( -(r+e)^2 + r^2*[ r+e-1 - ((r+e-1)^2)/2 ] )

By the way: is there a general rule for deciding when some approximations are valid and when some aren't?
eg. for large D: can I say e/r -> 0? or that r+e -> r?

 

[Dick]

Hi Dick,
Would you mind showing me how to arrived at exp(-e^2/s^2)?
I've tried expanding around r = sqrt(D-1)*s
and by noticing that (D-1)*s^2 = r^2

I simplified the equation to
p(r+e) approx. = exp( -(r+e)^2 + r^2*[ r+e-1 - ((r+e-1)^2)/2 ] )

By the way: is there a general rule for deciding when some approximations are valid and when some aren't?
eg. for large D: can I say e/r -> 0? or that r+e -> r?

Where did the '(r+e-1)' stuff come from? Start with your expression in the first post exp(-(r'+e)^2/(2s^2)+(D-1)ln(r'+e)). Expand the (r'+e)^2, expand ln(r'+e) using fabbagabbahey's suggestion. Keep the linear and quadratic terms in the ln series. Collect like powers of e. What do you get?

 

[CuppoJava]

Hi Dick,
Thank you very much for your help and patience.

Here's my work so far, I have an extra term at the end that I cannot get rid of

numerator:
-(r+e)^2+2(D-1)s^2(ln(r)+ln(1+e/r))
=-(r+e)^2 + 2r^2(ln(r)+ln(1+e/r))
=-r^2-2re-e^2+2r^2(r-1-(r-1)^2/2+e/r-(e/r)^2/2)
=-r^2-2re-e^2+2r^2(r-1)(1-(r-1)/2)+2re-e^2
=-r^2-2e^2+r^2(r-1)(1-r)

Thanks
-Patrick

 

[Dick]

You do not replace ln(r) with (r-1)-(r-1)^2/2. You can only series expand small quantities. r (or rather you should be writing r') is not necessarily small. Just leave that term as ln(r'). Remember you want f(r')*exp(...). The ln(r') is part of the f(r').

 

[CuppoJava]

Of course I can't expand around a large number. I should know that. Thanks again Dick.

I left ln(r') as it is, and finally I'm left with:

-r'^2+2r'^2ln(r') - 2e^2 in the numerator.

The first two terms are part of f(r'), and the last term is the correction term. I get a different coefficient (-2) in front of e^2 though. It's different from both your answer and the textbook answer. I'll keep on looking for what I did wrong.

Thank you very much for your help.
-Patrick

 

[Dick]

You multiplied the quantity you are expanding by 2*s^2. You should divide it back out and get f(r')exp(-e^2/s^2).

 

[CuppoJava]

Right. I forgot I did that.
Thanks again Dick.