Statistics - Moment Generating Functions

[mliuzzolino]

Homework Statement

The moment generating function (m.g.f.) of a random variable X is defined as the Expected value of e^tX:

M(t) = E(e^tX).

The series expansion of e^tX is:

e^tX = 1 + tX + (t²X²)/(2!) + (t³X³)/(3!) + ...

Hence,

M(t) = E(e^tx) = 1 + tμ₁' + (t²μ₂')/2! + (t³μ₃')/3! + ...


If we differentiate M(t) r times with respect to t and then set t = 0 we shall therefore obtain the rth moment about the origin μ_r'.

For example, the m.g.f. of the uniform distribution is:
M(t) = E(e^tX) = \int_0^1 e^{tx} dx = \dfrac{1}{t}e^{tx} |_0^1 = \dfrac{1}{t}(e^t - 1)

= 1 + \dfrac{t}{2!} + \dfrac{t^2}{3!} + \dfrac{t^3}{4!} + ...

Differentiate this series r times and then set t = 0. Show that μ_r' = \dfrac{1}{r + 1}.

Homework Equations

The Attempt at a Solution

I tried to do this by differentiating a few times to find a pattern:


1 + \dfrac{t}{2!} + \dfrac{t^2}{3!} + \dfrac{t^3}{4!} + \dfrac{t^4}{5!} ...

Differentiate once:
\dfrac{1}{2!} + \dfrac{2t}{3!} + \dfrac{3t^2}{4!} + \dfrac{4t^3}{5!} ...

Differentiate twice:
\dfrac{2}{3!} + \dfrac{6t}{4!} + \dfrac{12t^2}{5!} ...

Differentiate thrice:
\dfrac{6}{4!} + \dfrac{24t}{5!} ...

Since t will be set to zero, only the first terms of each differentiation "survive:"

\dfrac{1}{2!}, \dfrac{2}{3!}, \dfrac{6}{4!}, \dfrac{24}{5!}, ...

The pattern from this I noticed was: \dfrac{r!}{(r + 1)!}, where (r+1)! = (r+1)r!

So, \dfrac{r!}{(r+1)r!} = \dfrac{1}{r+1}.

This seems correct as I've obviously arrived at the correct expression, but I am wondering if I have approached this correctly and if there might have been any errors in my thought process.

I am also a little bit lost conceptually as to why t is set equal to 0. What does setting t = 0 signify?

I'm self-studying this topic and I'd really like to have a full understanding of what's going on rather than a superficial understanding where I'm just accepting that it says set t = 0. I'd appreciate any amount of elucidation!

Thanks!

 

[BruceW]

your method is good. And as for why t is set to zero... I just think of it as a useful 'trick'. I guess you can think of it like you are just considering the zeroth order term of the series. If you want to look into more detail, then maybe look at the wiki page on "characteristic function (probability theory)" Which I think is the nice generalisation of the generating function.

 

[Ray Vickson]

Homework Statement

The moment generating function (m.g.f.) of a random variable X is defined as the Expected value of e^tX:

M(t) = E(e^tX).

The series expansion of e^tX is:

e^tX = 1 + tX + (t²X²)/(2!) + (t³X³)/(3!) + ...

Hence,

M(t) = E(e^tx) = 1 + tμ₁' + (t²μ₂')/2! + (t³μ₃')/3! + ...


If we differentiate M(t) r times with respect to t and then set t = 0 we shall therefore obtain the rth moment about the origin μ_r'.

For example, the m.g.f. of the uniform distribution is:
M(t) = E(e^tX) = \int_0^1 e^{tx} dx = \dfrac{1}{t}e^{tx} |_0^1 = \dfrac{1}{t}(e^t - 1)

= 1 + \dfrac{t}{2!} + \dfrac{t^2}{3!} + \dfrac{t^3}{4!} + ...

Differentiate this series r times and then set t = 0. Show that μ_r' = \dfrac{1}{r + 1}.

Homework Equations

The Attempt at a Solution

I tried to do this by differentiating a few times to find a pattern:


1 + \dfrac{t}{2!} + \dfrac{t^2}{3!} + \dfrac{t^3}{4!} + \dfrac{t^4}{5!} ...

Differentiate once:
\dfrac{1}{2!} + \dfrac{2t}{3!} + \dfrac{3t^2}{4!} + \dfrac{4t^3}{5!} ...

Differentiate twice:
\dfrac{2}{3!} + \dfrac{6t}{4!} + \dfrac{12t^2}{5!} ...

Differentiate thrice:
\dfrac{6}{4!} + \dfrac{24t}{5!} ...

Since t will be set to zero, only the first terms of each differentiation "survive:"

\dfrac{1}{2!}, \dfrac{2}{3!}, \dfrac{6}{4!}, \dfrac{24}{5!}, ...

The pattern from this I noticed was: \dfrac{r!}{(r + 1)!}, where (r+1)! = (r+1)r!

So, \dfrac{r!}{(r+1)r!} = \dfrac{1}{r+1}.

This seems correct as I've obviously arrived at the correct expression, but I am wondering if I have approached this correctly and if there might have been any errors in my thought process.

I am also a little bit lost conceptually as to why t is set equal to 0. What does setting t = 0 signify?

I'm self-studying this topic and I'd really like to have a full understanding of what's going on rather than a superficial understanding where I'm just accepting that it says set t = 0. I'd appreciate any amount of elucidation!

Thanks!

You are trying to get the Maclaurin expansion, which reads as
f(t) = f(0) + t f'(0) + \frac{t^2}{2!} f''(0) + \cdots + \frac{t^n}{n!} f^{(n)}(0) + \cdots
The coefficients are obtained by getting higher derivatives and evaluating them at $t=0$.

Another way to see this for your specific case is to look at the higher derivatives of the different terms. We have
\left(\frac{d}{dt}\right)^r t^k = \left\{<br /> \begin{array}{cl} 0 &amp;, \; r &gt; k\\<br /> r! &amp;, \; r = k \\<br /> r! t^{k-r}&amp;, \; r &lt; k<br /> \end{array}<br /> \right.
When we set t = 0 we are just picking out the rth term.

 

[mliuzzolino]

That makes a lot of sense. I appreciate it both of you!