Finding Moment Generating Functions for Gamma and Poisson Distributions

[kingwinner]

Homework Statement

Suppose X1,X2,... are independent each having exponential distirbution with parameter lambda and N has a Poisson(lambda) distribution and is independent of the Xi's. You are given that the moment generating function of a Gamma(alpha,lambda) variable is m(t)=[lambda/(lambda-t)]^alpha
Find the moment generating function of S_N=X1+X2+...+X_N

Homework Equations

Just in case you are confused about the parameters above.
Poisson(lambda) =>E(W)=Var(W)=lambda
W~Gamma(alpha,lambda) => E(W)=alpha/lambda, Var(W)=alpha/lambda^2

The Attempt at a Solution

I know that the moment generating function of a sum of indepedent random variables is the product of the moment generating functions of each random variable. But here we have capital "N" and N follows some other distribution as well. This really scares me off...and I have no clue what to do in this case...

Does anyone have any idea? Please help!

 

[statdad]

Try the double of double expectation:
First, find the moment generating function of the sum given $$N = n$$ (this will go in the way you are accustomed)
Second: treat the result of the first step as a function of $$n$$ and take its expectation with respect to the distribution of $$N$$.

 

[kingwinner]

Hi statdad,

What is "double of double expectation"?

So S_N given some fixed N=n. Define a new random variable to be Y=S_N|N=n (holding n constant here)
Y~Gamma(n,lambda) with moment generating function m(t)=[lambda/(lambda-t)]ⁿ
What should we do next? I don't quite understand your second comment...I believe here we need the moment generating function, not the expectation...

Thanks for your help!

 

[statdad]

You have

$$S_N = X_1 + X_2 + \dots + X_n = \sum_i X_i$$

where the summands are i.i.d with one distribution, and $$N$$ has another (Poisson) distribution, independent of the summands. You want

$$m_S (t) = E[e^{tS}]$$

the moment generating function of the sum. The question -in this definition, with respect to what is the expectation taken? The answer is that it is taken with respect to two distributions - that of the $$X$$ values as well as that of $$N$$. You can do the work as follows.

$$m_S(t) = E[e^{tS}] = E [\underbrace{E[e^{tS} \mid N = n]}_{\substack{\text{Here}\\\text{N is fixed}}}]$$

That is, you can evaluate

$$E[e^{tS} \mid N = n]$$

just as you would the the mgf for any sum of independent r.v.s. The result will depend on $$n$$: in the second step calculate the expected value of this expression with respect to the Poisson distribution.

 

[kingwinner]

$$E[e^{tS} \mid N = n]$$=[lambda/(lambda-t)]ⁿ

$$E[e^{tS} \mid N]$$=[lambda/(lambda-t)]^N

$$E[E[e^{tS} \mid N]]$$=E[[lambda/(lambda-t)]^N] <---how can we calculate this?

Thank you!

 

[Pere Callahan]

Use the definition:
$$E[f(N)]=\sum_{n=0}^\infty{f(n)\operatorname{pmf}(n)}$$

where pmf(n) is the probabilty mass function of the Poisson distribution.

 

[statdad]

E[[lambda/(lambda-t)]N]

Since you know the distribution of $$N$$, use

$$E\left\[ \left(\frac{\lambda}{\lambda - t} \right)^n \right\] =\sum_n \left(\frac{\lambda} {t-\lambda}\right)^n \cdot p(n)$$

where by $$p(n)$$ I mean the probability function of $$N$$ - your Poisson function.

I'm not sure why the final code ended up on two lines.

 

[kingwinner]

E[[lambda/(lambda-t)]^N] <--I think here it should be capital N right? It should be the expectation of some 'random variable'. But in your post, it's small n...


And why does it become (t-lambda) in the denominator of your expression? Shouldn't it be (lambda-t)?


We are given: "...N has a Poisson(lambda) distribution and is independent of the Xi's" <---in which step did we actually used the fact that N is independent of the Xi's? Is this just extraneous information?

 

[statdad]

E[[lambda/(lambda-t)]^N] <--I think here it should be capital N right? It should be the expectation of some 'random variable'. But in your post, it's small n...


And why does it become (t-lambda) in the denominator of your expression? Shouldn't it be (lambda-t)?


We are given: "...N has a Poisson(lambda) distribution and is independent of the Xi's" <---in which step did we actually used the fact that N is independent of the Xi's? Is this just extraneous information?

When you begin you will have the capital $$N$$, when you begin writing the sum it is customary to use $$n$$ as the index of summation.

The denominator? Chalk that off to my typing too fast in order to finish before going to teach. You are correct.

 

[kingwinner]

So the moment generating function of S_N = E[[lambda/(lambda-t)]^N] = Sum from n=0 to n=infinity of [lambda/(lambda-t)]ⁿ * e^-lambdalambdaⁿ/n!] by definition of expectation. How should I proceed now?

 

[Pere Callahan]

So the moment generating function of S_N = E[[lambda/(lambda-t)]^N] = Sum from n=0 to n=infinity of [lambda/(lambda-t)]ⁿ * e^-lambdalambdaⁿ/n!] by definition of expectation. How should I proceed now?

Evaluate the sum using the exponential series
$$\sum_{n=0}^\infty{\frac{Z^n}{n!}}=e^Z$$
What is Z here?
There are already two powers with exponent n in your formula, namely $\lambda/(\lambda-t)$ and $\lambda$; combine these two to find what Z is.

 

[kingwinner]

OK, I get e^{-lambda+[lambda2/(lambda-t)]}

Am I right?

 

[Pere Callahan]

OK, I get e^{-lambda+[lambda2/(lambda-t)]}

Am I right?

This looks quite right.

 

[kingwinner]

So I am still wondering:
"...N has a Poisson(lambda) distribution and is independent of the Xi's" <---why is this information important? In which step did we actually used it??

 

[statdad]

The fact that $$N$$ is independent of the $$X_i$$ was used in the double expectation calculation, via the fact that we could base the calculation by considering the two marginal distributions separately. If they were not independent, the calculation would be more difficult due to the requirement of needing to work with a conditional distribution of $$N$$ given the $$X$$ values.

Did that make sense?

 

[kingwinner]

The fact that $$N$$ is independent of the $$X_i$$ was used in the double expectation calculation, via the fact that we could base the calculation by considering the two marginal distributions separately. If they were not independent, the calculation would be more difficult due to the requirement of needing to work with a conditional distribution of $$N$$ given the $$X$$ values.

Did that make sense?

Um...But I believe that the formula E[E(Y|X)]=E(Y) holds even if X and Y are dependent, so is the assumption "N independent of the Xi's" really necessary??

 

[statdad]

"Um...But I believe that the formula E[E(Y|X)]=E(Y) holds even if X and Y are dependent, so is the assumption "N independent of the Xi's" really necessary?"

You are correct - either I wasn't clear enough or you missed my point. The independence makes the calculation of $$E[E(Y \mid X)] = E[Y]$$ easier - easier than it would be if the two variables were dependent, since the marginal distributions would be more complicated - possible a lot, possibly a little, more still, more complicated.

 

[kingwinner]

The question has a second part to it:

Part b) Find the distribution of

   n
Y= ∑ [2lambda  X[sub]i[/sub]]
  i=1[/b]

How can we identify the dsitribution? I tried using moment generating functions(mgf), but the mgf of Y is not looking like anything close to the gamma mgf that we are given in the original question.

Please help...