Homework Help: Moment Generating Functions

1. Dec 4, 2008

kingwinner

1. The problem statement, all variables and given/known data
Suppose X1,X2,... are independent each having exponential distirbution with parameter lambda and N has a Poisson(lambda) distribution and is independent of the Xi's. You are given that the moment generating function of a Gamma(alpha,lambda) variable is m(t)=[lambda/(lambda-t)]alpha
Find the moment generating function of SN=X1+X2+...+XN

2. Relevant equations
Just in case you are confused about the parameters above.
Poisson(lambda) =>E(W)=Var(W)=lambda
W~Gamma(alpha,lambda) => E(W)=alpha/lambda, Var(W)=alpha/lambda^2

3. The attempt at a solution
I know that the moment generating function of a sum of indepedent random variables is the product of the moment generating functions of each random variable. But here we have capital "N" and N follows some other distribution as well. This really scares me off...and I have no clue what to do in this case...

2. Dec 4, 2008

Try the double of double expectation:
First, find the moment generating function of the sum given $$N = n$$ (this will go in the way you are accustomed)
Second: treat the result of the first step as a function of $$n$$ and take its expectation with respect to the distribution of $$N$$.

3. Dec 5, 2008

kingwinner

What is "double of double expectation"?

So SN given some fixed N=n. Define a new random variable to be Y=SN|N=n (holding n constant here)
Y~Gamma(n,lambda) with moment generating function m(t)=[lambda/(lambda-t)]n
What should we do next? I don't quite understand your second comment...I believe here we need the moment generating function, not the expectation...

4. Dec 5, 2008

You have

$$S_N = X_1 + X_2 + \dots + X_n = \sum_i X_i$$

where the summands are i.i.d with one distribution, and $$N$$ has another (Poisson) distribution, independent of the summands. You want

$$m_S (t) = E[e^{tS}]$$

the moment generating function of the sum. The question -in this definition, with respect to what is the expectation taken? The answer is that it is taken with respect to two distributions - that of the $$X$$ values as well as that of $$N$$. You can do the work as follows.

$$m_S(t) = E[e^{tS}] = E [\underbrace{E[e^{tS} \mid N = n]}_{\substack{\text{Here}\\\text{N is fixed}}}]$$

That is, you can evaluate

$$E[e^{tS} \mid N = n]$$

just as you would the the mgf for any sum of independent r.v.s. The result will depend on $$n$$: in the second step calculate the expected value of this expression with respect to the Poisson distribution.

5. Dec 5, 2008

kingwinner

$$E[e^{tS} \mid N = n]$$=[lambda/(lambda-t)]n

$$E[e^{tS} \mid N]$$=[lambda/(lambda-t)]N

$$E[E[e^{tS} \mid N]]$$=E[[lambda/(lambda-t)]N] <---how can we calculate this?

Thank you!

6. Dec 5, 2008

Pere Callahan

Use the definition:
$$E[f(N)]=\sum_{n=0}^\infty{f(n)\operatorname{pmf}(n)}$$

where pmf(n) is the probabilty mass function of the Poisson distribution.

7. Dec 5, 2008

E[[lambda/(lambda-t)]N]

Since you know the distribution of $$N$$, use

$$E\left$\left(\frac{\lambda}{\lambda - t} \right)^n \right$ =\sum_n \left(\frac{\lambda} {t-\lambda}\right)^n \cdot p(n)$$

where by $$p(n)$$ I mean the probability function of $$N$$ - your Poisson function.

I'm not sure why the final code ended up on two lines.

Last edited: Dec 5, 2008
8. Dec 5, 2008

kingwinner

E[[lambda/(lambda-t)]N] <--I think here it should be capital N right? It should be the expectation of some 'random variable'. But in your post, it's small n...

And why does it become (t-lambda) in the denominator of your expression? Shouldn't it be (lambda-t)?

We are given: "...N has a Poisson(lambda) distribution and is independent of the Xi's" <---in which step did we actually used the fact that N is independent of the Xi's? Is this just extraneous information?

Last edited: Dec 5, 2008
9. Dec 5, 2008

When you begin you will have the capital $$N$$, when you begin writing the sum it is customary to use $$n$$ as the index of summation.

The denominator? Chalk that off to my typing too fast in order to finish before going to teach. You are correct.

10. Dec 6, 2008

kingwinner

So the moment generating function of SN = E[[lambda/(lambda-t)]N] = Sum from n=0 to n=infinity of [lambda/(lambda-t)]n * e-lambdalambdan/n!] by definition of expectation. How should I proceed now?

11. Dec 6, 2008

Pere Callahan

Evaluate the sum using the exponential series
$$\sum_{n=0}^\infty{\frac{Z^n}{n!}}=e^Z$$
What is Z here?
There are already two powers with exponent n in your formular, namely $\lambda/(\lambda-t)$ and $\lambda$; combine these two to find what Z is.

12. Dec 6, 2008

kingwinner

OK, I get e-lambda+[lambda2/(lambda-t)]

Am I right?

13. Dec 7, 2008

Pere Callahan

This looks quite right.

14. Dec 8, 2008

kingwinner

So I am still wondering:
"...N has a Poisson(lambda) distribution and is independent of the Xi's" <---why is this information important? In which step did we actually used it??

15. Dec 8, 2008

The fact that $$N$$ is independent of the $$X_i$$ was used in the double expectation calculation, via the fact that we could base the calculation by considering the two marginal distributions separately. If they were not independent, the calculation would be more difficult due to the requirement of needing to work with a conditional distribution of $$N$$ given the $$X$$ values.

Did that make sense?

16. Dec 9, 2008

kingwinner

Um...But I believe that the formula E[E(Y|X)]=E(Y) holds even if X and Y are dependent, so is the assumption "N independent of the Xi's" really necessary??

17. Dec 9, 2008

"Um...But I believe that the formula E[E(Y|X)]=E(Y) holds even if X and Y are dependent, so is the assumption "N independent of the Xi's" really necessary?"

You are correct - either I wasn't clear enough or you missed my point. The independence makes the calculation of $$E[E(Y \mid X)] = E[Y]$$ easier - easier than it would be if the two variables were dependent, since the marginal distributions would be more complicated - possible a lot, possibly a little, more still, more complicated.

18. Dec 9, 2008

kingwinner

The question has a second part to it:

Part b) Find the distribution of
Code (Text):
n
Y= ∑ [2lambda  X[sub]i[/sub]]
i=1[/b]
How can we identify the dsitribution? I tried using moment generating functions(mgf), but the mgf of Y is not looking like anything close to the gamma mgf that we are given in the original question.