# Homework Help: Simplifying this expression for large values of D

1. Aug 28, 2009

### CuppoJava

1. The problem statement, all variables and given/known data
Show that:
p(r'+e) $$\cong$$ p(r')*exp( -3e2 / 2s2 ) for large D

given that:
p(r) = rD-1 * exp( -r2 / 2s2 )
r' = sqrt(D) * s

2. Relevant equations
This arises out of a D-dimensional gaussian distribution and r' is the radius at which the probability is maximized.

3. The attempt at a solution
I've gotten to the point:

p(r'+e) $$\propto$$ exp( -(r'+e)2 / 2s2 + (D-1)ln(r'+e) )

And the solution manual says that using the Taylor expansion for ln, will help everything reduce down, but all I get is a mess. What really baffles me is where the dependance on D went.

Thanks very much for your help
-Patrick

2. Aug 28, 2009

### foxjwill

According to Wolfram Alpha, this is true for large s but not for large D.

3. Aug 28, 2009

### CuppoJava

Thanks for helping foxjwill. I'm almost certain that the result is correct though. There's an probabilistic interpretation of it, and it requires that this holds true for large D not s.

Wolfram Alpha, though, seems like an extremely useful tool. I haven't heard of it before. May I ask exactly what you typed into the query box?

Thanks
-Patrick

4. Aug 28, 2009

### gabbagabbahey

For large $D$, what can you say about the size of $\frac{e}{r'}$? Use that to Taylor expand the logarithm:

$$\ln(r'+e)=\ln(r'(1+\frac{e}{r'}))=\ln(r')+\ln(1+\frac{e}{r'})$$

5. Aug 28, 2009

### foxjwill

First, I typed in

Code (Text):
(exp(-3epsilon^2/(2s^2)) *r^(D-1)*exp(-r^2/(2s^2)) /  (r+epsilon)^(D-1)*exp(-(r+epsilon)^2/(2s^2)) )
Then I copied the result given in the Alternate form box:
Code (Text):
r^(D-1) (r+epsilon)^(1-D) e^(-(r^2+r epsilon+2 epsilon^2)/s^2)
Finally, I pasted that into the input box, replaced each instance of r with "(s*sqrt(D))", then prepended "limit" and appended "as D->infinity":
Code (Text):
limit (s*sqrt(D))^(D-1) ((s*sqrt(D))+epsilon)^(1-D) e^(-((s*sqrt(D))^2+(s*sqrt(D)) epsilon+2 epsilon^2)/s^2) as D->infinity
Wolfram Alpha says this equals 0.

6. Aug 28, 2009

### Dick

Something in this isn't quite right. If you work out the expansion around sqrt(D)*s you get a linear term in e. That's because the maximum value of your p(r) is at sqrt(D-1)*s NOT sqrt(D)*s, isn't it? Why don't you expand around the maximum? And I also don't get exp(-3*e^2/s^2) for the correction factor several different ways. I get just exp(-e^2/s^2).

Last edited: Aug 28, 2009
7. Aug 29, 2009

### CuppoJava

Thank you very much for your help.
I haven't gotten then answer actually. It's a textbook question that asks me to prove the statement. (Exercise 1.20 from Pattern Recognition and Machine Learning by Bishop).

You're right. The maximum is actually at sqrt(D-1)*s, which the solution manual says can be approximated to sqrt(D)*s for large D. I will try expanding around sqrt(D-1)*s and see if I can get it.

Thank you for helping me.
-Patrick

8. Aug 29, 2009

### CuppoJava

Hi Dick,
Would you mind showing me how to arrived at exp(-e^2/s^2)?
I've tried expanding around r = sqrt(D-1)*s
and by noticing that (D-1)*s^2 = r^2

I simplified the equation to
p(r+e) approx. = exp( -(r+e)^2 + r^2*[ r+e-1 - ((r+e-1)^2)/2 ] )

By the way: is there a general rule for deciding when some approximations are valid and when some aren't?
eg. for large D: can I say e/r -> 0? or that r+e -> r?

9. Aug 29, 2009

### Dick

Where did the '(r+e-1)' stuff come from? Start with your expression in the first post exp(-(r'+e)^2/(2s^2)+(D-1)ln(r'+e)). Expand the (r'+e)^2, expand ln(r'+e) using fabbagabbahey's suggestion. Keep the linear and quadratic terms in the ln series. Collect like powers of e. What do you get?

10. Aug 30, 2009

### CuppoJava

Hi Dick,
Thank you very much for your help and patience.

Here's my work so far, I have an extra term at the end that I cannot get rid of

numerator:
$$-(r+e)^2+2(D-1)s^2(ln(r)+ln(1+e/r))$$
$$=-(r+e)^2 + 2r^2(ln(r)+ln(1+e/r))$$
$$=-r^2-2re-e^2+2r^2(r-1-(r-1)^2/2+e/r-(e/r)^2/2)$$
$$=-r^2-2re-e^2+2r^2(r-1)(1-(r-1)/2)+2re-e^2$$
$$=-r^2-2e^2+r^2(r-1)(1-r)$$

Thanks
-Patrick

11. Aug 30, 2009

### Dick

You do not replace ln(r) with (r-1)-(r-1)^2/2. You can only series expand small quantities. r (or rather you should be writing r') is not necessarily small. Just leave that term as ln(r'). Remember you want f(r')*exp(...). The ln(r') is part of the f(r').

12. Aug 30, 2009

### CuppoJava

Of course I can't expand around a large number. I should know that. Thanks again Dick.

I left ln(r') as it is, and finally I'm left with:

$$-r'^2+2r'^2ln(r') - 2e^2$$ in the numerator.

The first two terms are part of f(r'), and the last term is the correction term. I get a different coefficient (-2) in front of e^2 though. It's different from both your answer and the textbook answer. I'll keep on looking for what I did wrong.

Thank you very much for your help.
-Patrick

13. Aug 30, 2009

### Dick

You multiplied the quantity you are expanding by 2*s^2. You should divide it back out and get f(r')exp(-e^2/s^2).

14. Aug 30, 2009

### CuppoJava

Right. I forgot I did that.
Thanks again Dick.