Guassian pdf is sum of two other RV Integral evaluation -

[EmmaSaunders1]

Could someone please advise me how to put the integral from + to - infinity notation used $ exp{-1/2(y-x)^2(A^-1) + x^2(B)^-1}dx Into the form k1 $ exp[-k2(x-k3)^2]dx using the fact that $exp(-1/2x^2)dx = 2pi^1/2

I have encountered this problem when trying to derive the conditional density function of p(x|y) where y = x + n, where y is a gaussian random variable and is the sum of two other random variables.

Help really appreciated with this - yhanks - - Ems x

 

[pmsrw3]

Just substitute x+n for y, expand, collect terms, and complete the square.

 

[EmmaSaunders1]

Hi Thanks for the help - when make the substitution I obtain $exp{-1/2(n)^2(A^-1) + x^2(B^-1)} - I am having trouble putting into the form k1 $ exp[-k2(x-k3)^2]. As it is now I would have an extra term of -2nx which doesn't fit the above - your help is appreciated.

Thanks

Emma x

 

[pmsrw3]

Hi Thanks for the help - when make the substitution I obtain $exp{-1/2(n)^2(A^-1) + x^2(B^-1)}

That's not right. There should be an -1/2 x^2(A^-1) term, and a -2nx term, too.

I am having trouble putting into the form k1 $ exp[-k2(x-k3)^2]. As it is now I would have an extra term of -2nx which doesn't fit the above - your help is appreciated.

You have inside the exp an expression of the form $a x^2 + b x + c$, where a, b, and c are combinations of n, A, and B. You now need to complete the square -- that means find a number d such that $a x^2 + b x + d = (x\sqrt{a}+\sqrt{d})^2$. You'll find that $d = \frac{b^2}{4a}$ works. Now you can write:

$$\begin{eqnarray*} a x^2 + b x + c & = & a x^2 + b x + \frac{b^2}{4a} - \frac{b^2}{4a} + c \\ & = & \left(x\sqrt{a} + \frac{b}{2\sqrt{a}} \right)^2 - \frac{b^2}{4a} + c \\ & = & a \left(x + \frac{b}{2a} \right)^2 + \left( c -\frac{b^2}{4a} \right)\\ \end{eqnarray*}$$

(I may have made a mistake or two in there, but that's the strategy.) You can read k2 and k3 off the first term. k1 is the exponential of the last thing. Actually, your 1/n^2 term will contribute to k1, too.

This should look familiar -- it's the way the quadratic equation $a x^2 + b x + c = 0$ is solved.

 

[EmmaSaunders1]

Thanks for your quick response. When making the substitution however, considering the first terms of A^-1, we get = 1/2{x^2+n^2+2xn+x^2-2x^2-2xn]A^-1 which all cancel to 1/2[n^2]A^-1. Are you leaving this expression in its full form or not?

Thanks again - Emmax

 

[pmsrw3]

You're right -- my error. That makes the problem considerably simpler -- you don't need to complete the square.

But what did you mean then, by saying, "I would have an extra term of -2nx"?

 

[EmmaSaunders1]

if after expansion and collection of terms - I get exp{-1/2(n)^2(A^-1) + x^2(B^-1). I can't see how to put this into the form exp[-k2(x-k3)^2]. Expanding the latter out results in terms of x^2 +k3^2 - 2k3x. This last term throws me off - if you can see this simpler I d appreciate your oppinion - thanks x

 

[pmsrw3]

Well, in your case k3 = 0.

 

[EmmaSaunders1]

But then you won't have an n^2 term if expanding

 

[pmsrw3]

Maybe this is what you're missing:

$$e^{\frac{x^2}{B}-\frac{n^2}{2 A}}=e^{\frac{x^2}{B}}e^{-\frac{n^2}{2 A}}=k_1e^{\frac{x^2}{B}}$$

Does that help?

 

[EmmaSaunders1]

possibly - could you please clarify the notation - - frac?

 

[pmsrw3]

possibly - could you please clarify the notation - - frac?

Ah! I see you're using a browser that doesn't display LaTex equations properly. Well, that will have made all my previous posts infinitely more difficult to read. \frac{numerator}{denominator} means numerator/denominator.

 

[EmmaSaunders1]

Thanks - I will have a look tonight and try to update my browser - could you possibly clarify - if I have an equation of the form Ax^2 + bx + c , how do I turn it into the form k2(x-k3)^2

 

[pmsrw3]

See http://en.wikipedia.org/wiki/Completing_the_square .

The point I was making in my previous post was e^(a-b) = e^-b e^a = k1 e^a. In other words, you don't get the n^2 term from expansion of (x+k3) -- you get it from k1.

 

[EmmaSaunders1]

Have you found the terms k1, k2, k3? I am thinking it may be easier without the substitution of y = x+n.

Thanks for your help x

 

[EmmaSaunders1]

Performing expansion and collection of terms I get - exp[-1/2{x^2(A^-1+B^-1)+n^2(A^-1)} - - using the n term substitution. Using no substitution I get exp[-1/2{x^2(A^-1+B^-1) - 2yxA^-1 + y^2A^-1] = which looks more like solving the square problem - but I still can't work out how to find the terms k1, k2 or k3 from this?

 

[pmsrw3]

Have you found the terms k1, k2, k3?/

k1 = e^(-n^2 / (2A))
k2 = -1/B
k3 = 0

(I'm assuming "x^2(B)^-1" in your original expression means x^2 / B.)

I am thinking it may be easier without the substitution of y = x+n.

Well, it is perfectly straightforward by substituting y = x + n. And I don't see how you could possibly do it without using that information.

 

[pmsrw3]

Performing expansion and collection of terms I get - exp[-1/2{x^2(A^-1+B^-1)+n^2(A^-1)} - - using the n term substitution.

Uhh... Now you're REALLY confusing me. You said earlier, and convinced me you were right, that you got exp{x^2(B^-1)+n^2((2A)^-1))}. Why did it change?

 

[EmmaSaunders1]

When I don't use the original y=x+n substitution I get [-1/2{x^2(A^-1+B^-1) - 2yxA^-1 + y^2A^-1] - then substituting y for x+n - I find exp[-1/2{x^2(A^-1+B^-1)+n^2(A^-1)} . I appreciate your help. The problems solution ultimately needs to be in terms of x and y. It is involved around evaluating the integral to find the posterior distribution for two gaussian rv's one of which is the sum (i.e y = x+ n) of another two rv's. When working through the problem it seems that k1 is of the form AB/(A+B)

 

[pmsrw3]

When I don't use the original y=x+n substitution I get [-1/2{x^2(A^-1+B^-1) - 2yxA^-1 + y^2A^-1]

I don't see how this can possible be true if you're starting from exp{-1/2(y-x)^2(A^-1) + x^2(B)^-1} as you said in the original post. Anyway, "[-1/2{x^2(A^-1+B^-1) - 2yxA^-1 + y^2A^-1]" is wrong: the "{" is unbalanced. What is being multiplied by -1/2 here?

then substituting y for x+n - I find exp[-1/2{x^2(A^-1+B^-1)+n^2(A^-1)} .

Well, if you say that's what you find, I can't say it's untrue. But if that's what you find, you've made a mistake. The -2yx/A term contributes a -2x^2/A that cancels out the x^2/A terms.

I appreciate your help. The problems solution ultimately needs to be in terms of x and y.

Emma, you said in your original post that you wanted a solution in the form "k1 $ exp[-k2(x-k3)^2]dx". There is no y in there. Why do you now say "The problems solution ultimately needs to be in terms of x and y?

It is involved around evaluating the integral to find the posterior distribution for two gaussian rv's one of which is the sum (i.e y = x+ n) of another two rv's.

What the Hell is an rv? I assume you don't mean "Recreational Vehicle".

When working through the problem it seems that k1 is of the form AB/(A+B)

Well, if so, then you must have given some incorrect information, because that is not the answer to the question you posed.

I suspect that you're supposed to normalize this distribution, and that that's where an AB/(A+B) factor might come in. However, there was no way to figure this out from what you've said so far. Honestly, I'm getting pretty frustrated. You keep giving inaccurate, incomplete, and inconsistent information, in a notation that is almost unreadable. Couldn't you be a little more careful?

 

[EmmaSaunders1]

Hi - thanks for your patience with me - I have added as an attatchment the problem I am trying to work through - its the evaluation of the integral, I apologise in advance if my notation wasn't accurate - I tried my best - Thanks x

 

[EmmaSaunders1]

As I have said previosuly y is a random variable (r.v) which is gaussian it is made up by the sum of two other independent random variables x and n so y = x+n

 

[pmsrw3]

OK. So the formula in your original post was wrong. And I guess that in your previous posts you were using a dollar sign to represent an integral. That clears things up a bit.

You said, "k1 is of the form AB/(A+B)". Why? I don't see anything in the problem you posted that says that.

Anyway, you deal with this with or without substituting for y. If you substitute y = x+n into the integral it'll be simpler to evaluate. You can then backsubstitute n = y-x in the solution to get it in the form they have. Or you can expand the (y-x) term, collect terms in x, and complete the square to get it in the k1/k2/k3 form, with the k's depending on y. This is apparently what the writer's expect you to do. It will also work.

Finally, you have to evaluate the integral. By substituting u = (x-k3)/sqrt(k2), you will be able to put it in the form whose value integrates to sqrt(2pi). Everything else in the solution comes from k1 (the exp{stuff} factor) or from the factor of k2 (the stuff^(-1/2) factor) you pull out of the integral after substitution.

 

[EmmaSaunders1]

Thanks for the tips - I was under the impression that k1 - is of the form AB/(A+B) - as when one moves from the fourth line of equations down (where the integral with the exp term is first present in the denominator there is also a term (AB)^-1/2. This then changes to (A+B)^-1/2 when the integral is evaluated hence I thought there must have been some multiplication of the form AB/(A+B) when the integral is evaluated.

I have done as you said, expanded without substituting for y to collect terms in the form Ax^2+bx+c

exp[-\frac{1}{2}((y^2+x^2-2xy)A^{-1}+x^2B^{-1})]

exp[-\frac{1}{2}((x^2(A^{-1}+B^{-1})-2xyA^{-1}+y^2A^{-1})]

I am having trouble completing the square of this above equation and also removing a common term which can hold as k1

I have figured out the Latex stuff - so hope this works out x

 

[EmmaSaunders1]

Here it is as a gif file - could you please advise how to add things straight to the browser in a nice format?

 

[pmsrw3]

So, you have an expression a x^2 + b x + c. a = (1/A + 1/B), b = -2y/A, and c = y^2/A, if I'm reading this right. Just follow the procedure for completing the square in the Wikipedia article I pointed you to, or in my post #4, if you can get LaTex to work. Once you complete the square, you'll end up with something like exp{a(x-h)^2 + k}, where k doesn't depend on x. You write that as exp{k} exp{a(x-h)^2} -- exp{k} is (or contributes to) k1.

To get LaTex to show up in the browser, you have to surround it with [ tex] [ /tex] or [ itex] [ /itex] tags. (But leave out the spaces. I put them into make them readable.) Or just hit the quote button on one of my posts to see how I did it.

 

[EmmaSaunders1]

This is super difficult!

 

[pmsrw3]

It will seem that way when you're not used to it.

 

[EmmaSaunders1]

So I have gotten this far (as attached) - the integral is now in the form as required - I am having a lot of trouble solving it - could you please offer any tips - do I need to factorize anymore - I appreciate your help - I need to get use to this maths (again) -

 

[pmsrw3]

OK, that's a definite integral from -inf to +inf. You need to do a substitution u = ax + b to make it looks like the integral you know, whose value is sqrt(2pi). When you do that the dx will turn into du/a, and you'll pull the 1/a out of the integral -- it'll become part of the final answer.

 

[EmmaSaunders1]

Hi I have done the integration - but still don't get the result from the book, the terms inside the exp differ in the book to what I have calculated is there some factorization that I am missing - (also in my previous post the k-term dropped a yvalue which has been put back in - apologies)

Your thoughts are appreciated x

 

[pmsrw3]

This is the denominator of the conditional probability expression, right? Well, it's not exactly right but there is a close resemblance. Now you go back and do it over again half a dozen times, looking for the sign errors.

 

[EmmaSaunders1]

Thats right - it's almost the denominator - I was just wondering - is there any methods to simply pull out the A^-1B^-1 from the exponential, some factorizing trick. - also I used the subsitutuion in my integral - u=(2k2)^1/2(x-k3). In a previous post you mentions u=(2k2)^-1/2(x-k3), was this merely due to a mistake I made in my notation, would you be-able to clarify that the substitution I have performed is okay? - Thanks

 

[pmsrw3]

Yeah, I probably made a mistake.

I think at this point you've basically got it. You just need to go back over your work, fidn the errors. Also, given that you know where you want to end up, check whether, where your answer and their answer seem to differ, they are just different ways of writing the same thing.

 

[EmmaSaunders1]

At last - all I needed was [A^-1B^-1]/[A^-1+B^-1] = [A+B]^-1

Thanks for the help - - this is a great forum