How Can I find the PDF of This?

[EngWiPy]

Hi,

I have this random variable:

$$Z=\left(\sum_{m=1}^NX_m^{-1}\right)^{-1}$$

where Xm are i.i.d. exponential random variables with parameter 1, i.e. the PDF of these random variables is:

$$f_{X_m}(x)=e^{-x}$$

How can I find the CDF, PDF, and the MGF of Z?

Thanks

 

[chiro]

Hi,

I have this random variable:

$$Z=\left(\sum_{m=1}^NX_m^{-1}\right)^{-1}$$

where Xm are i.i.d. exponential random variables with parameter 1, i.e. the PDF of these random variables is:

$$f_{X_m}(x)=e^{-x}$$

How can I find the CDF, PDF, and the MGF of Z?

Thanks

Hey S_David.

The first thing I would recommend is to first find the PDF/CDF of $$X_m^{-1}$$ first before you do anything. To do this use a transformation method. Something with the ideas along the lines of this:

http://www.math.uiuc.edu/~r-ash/Stat/StatLec1-5.pdf

The above shows the idea used, as for the implementation that will depend on the transformation but I think it will be ok since the inverse of e^-x is simply -log(x) in terms of the normal inverse function relations (f o f^-1) = (f^-1 o f) = x.

Once you get the PDF/CDF then you get the MGF and hence the MGF of the sum of these. I'm going to guess that the MGF of the sum will match the form of an individual, but if it is not you will need to see what kind of PDF structure the MGF of the sum of variables represents.

Then once you have done this you use the transformation theorem again to get the PDF of the inverse of the sum of inverse exponentials and using this PDF/CDF you get the MGF of the entire thing by using the definition of the MGF.1

 

[EngWiPy]

Hey S_David.

The first thing I would recommend is to first find the PDF/CDF of $$X_m^{-1}$$ first before you do anything. To do this use a transformation method. Something with the ideas along the lines of this:

http://www.math.uiuc.edu/~r-ash/Stat/StatLec1-5.pdf

The above shows the idea used, as for the implementation that will depend on the transformation but I think it will be ok since the inverse of e^-x is simply -log(x) in terms of the normal inverse function relations (f o f^-1) = (f^-1 o f) = x.

Once you get the PDF/CDF then you get the MGF and hence the MGF of the sum of these. I'm going to guess that the MGF of the sum will match the form of an individual, but if it is not you will need to see what kind of PDF structure the MGF of the sum of variables represents.

Then once you have done this you use the transformation theorem again to get the PDF of the inverse of the sum of inverse exponentials and using this PDF/CDF you get the MGF of the entire thing by using the definition of the MGF.1

I tried that, however, it does not give a close form, since the MGF has no closed form inverse. I was wondering if we can find a closed form expression form this!

Thanks

 

[chiro]

I tried that, however, it does not give a close form, since the MGF has no closed form inverse. I was wondering if we can find a closed form expression form this!

Thanks

Can you show your working for getting (in order)

a) PDF for inverse of exponential distribution
b) MGF for inverse exponential
c) MGF for sum of i.i.d inverse exponentials

We'll take it from there since the rest will depend on what c) gives you.

So for a) have you used transformation method to get PDF for 1 inverse exponential distribution and if so what did you get? (Show working if you could please).

 

[EngWiPy]

Can you show your working for getting (in order)

a) PDF for inverse of exponential distribution
b) MGF for inverse exponential
c) MGF for sum of i.i.d inverse exponentials

We'll take it from there since the rest will depend on what c) gives you.

So for a) have you used transformation method to get PDF for 1 inverse exponential distribution and if so what did you get? (Show working if you could please).

I am sorry, I have not noticed your post. Anyway, let:

$$Y=\frac{1}{X}$$

Then the p.d.f of Y is given by:

$$f_Y(y)=\frac{1}{y^2}e^{-1/y}$$

Now let:

$$W=\sum_{m=1}^NY_m$$

Then the MGF of W is given by

$$\mathcal{M}_W(s)=\prod_{m=1}^{N}\mathcal{M}_m(s)$$

where

$$\mathcal{M}_m(s)=\int_0^{\infty}\frac{1}{y^2}e^{-sy-\frac{1}{y}}\,dy=2\sqrt{s}K_1\left(2\sqrt{s}\right)$$

where K1(.) is the modified Bessel function of the second kind and of order 1.

Ok, then what?

 

[chiro]

I think your MGF is wrong.

The MGF is calculated by finding E[e^(sx)] for a PDF in terms of x. In your expression on the last line you have a -sy instead of a +sy.

After this is corrected, the next thing is to figure out the form of the MGF of the multiplied result of the independent MGF's. The key thing here is to see whether the resultant MGF has the same form as the individual MGF (the parameters will obviously change) and if it does not, then you need to recognize what form it is.

So for above, the first thing to do is to check if the product of the MGF's retains the same form as the individual MGF in terms of any parameters the MGF has.

I'll wait for this before I proceed further.

 

[EngWiPy]

I think your MGF is wrong.

The MGF is calculated by finding E[e^(sx)] for a PDF in terms of x. In your expression on the last line you have a -sy instead of a +sy.

After this is corrected, the next thing is to figure out the form of the MGF of the multiplied result of the independent MGF's. The key thing here is to see whether the resultant MGF has the same form as the individual MGF (the parameters will obviously change) and if it does not, then you need to recognize what form it is.

So for above, the first thing to do is to check if the product of the MGF's retains the same form as the individual MGF in terms of any parameters the MGF has.

I'll wait for this before I proceed further.

It does not matter how you define the MGF. In my case I used the definition:

$$\mathcal{M}_X(s)=\mathbb{E}_X\{e^{-sx}\}$$

If I want to find the p.d.f from this definition of MGF, then I need just to find the inverse Laplace transform, however, if your definition is used, then the p.d.f will be the inverse Laplace transform of the M(-s).

It is obvious that:

$$\mathcal{M}_W(s)=\left(\mathcal{M}_m(s)\right)^N=2^N\,s^{N/2}\,K_1^N\left(2\sqrt{s}\right)$$

because all the variables are i.i.d.

 

[chiro]

It does not matter how you define the MGF. In my case I used the definition:

$$\mathcal{M}_X(s)=\mathbb{E}_X\{e^{-sx}\}$$

If I want to find the p.d.f from this definition of MGF, then I need just to find the inverse Laplace transform, however, if your definition is used, then the p.d.f will be the inverse Laplace transform of the M(-s).

It is obvious that:

$$\mathcal{M}_W(s)=\left(\mathcal{M}_m(s)\right)^N=2^N\,s^{N/2}\,K_1^N\left(2\sqrt{s}\right)$$

because all the variables are i.i.d.

OK, no worries I see what you mean now (and why you had to use Laplace transform to get the actual form of the expression).

Now when you have the MGF you have a few options.

One option is to see if someone else has figured it out or if there is an existing MGF for an existing distribution.

The other option is to use the MGF and all of it's derivatives to define the probability distribution or to use what is called a Probability Generating Function to get the probability information which can be derived from the MGF definition.

Now the PGF only works for discrete random variables but we can use the same kind of idea for continuous variables in what is called the characteristic function. Essentially we use a Fourier transform type analysis and replace the MGF s parameter with an is. Then we can use the definition of the MGF to get our PDF for that particular MGF. Since you have the MGF, you can get the PDF.

Then if you want to find the inverse of this distribution, you just use an ordinary transformation rule.

Here is a link for a PDF covering the characteristic function. Look at Theorem 10.5 on Page 34 of this document:

http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter10.pdf

 

[EngWiPy]

OK, no worries I see what you mean now (and why you had to use Laplace transform to get the actual form of the expression).

Now when you have the MGF you have a few options.

One option is to see if someone else has figured it out or if there is an existing MGF for an existing distribution.

The other option is to use the MGF and all of it's derivatives to define the probability distribution or to use what is called a Probability Generating Function to get the probability information which can be derived from the MGF definition.

Now the PGF only works for discrete random variables but we can use the same kind of idea for continuous variables in what is called the characteristic function. Essentially we use a Fourier transform type analysis and replace the MGF s parameter with an is. Then we can use the definition of the MGF to get our PDF for that particular MGF. Since you have the MGF, you can get the PDF.

Then if you want to find the inverse of this distribution, you just use an ordinary transformation rule.

Here is a link for a PDF covering the characteristic function. Look at Theorem 10.5 on Page 34 of this document:

http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter10.pdf

Any other thoughts? I need a closed form p.d.f, and the above method does not seem to provide one!

 

[chiro]

Any other thoughts? I need a closed form p.d.f, and the above method does not seem to provide one!

Since the method I have given is equivalent to finding the PDF given a MGF for a continuous function, unfortunately I can't think of a better way to do this.

However if you need this for doing calculations, then whether you need to generate your distribution as a RNG for that distribution, or whether you need to calculate the PDF densities, you could still probably get away with using for the first one, simulation algorithms like Metropolis-Hastings or something else and for the second one, just standard numerical integration techniques.

The key is figuring out the error you wish to tolerate and go from there.

Unfortunately I can't help you with getting a closed form PDF and since this will be the equivalent PDF, then if it is in a closed form, it must be able to generated from that integral.

If you need to use it for some applied purposes or even in a serious application like engineering, I would still urge you to consider using a numerical integration scheme and RNG algorithm like Metropolis-Hastings with some gauranteed error-bound for required accuracy since using the right implementation will give you something very accurate.

Perhaps you could talk this over with a senior engineer if you are doing this for some specific application where accuracy is vital, or some other senior scientist or your boss if this is vital in another endeavor.

 

[EngWiPy]

Since the method I have given is equivalent to finding the PDF given a MGF for a continuous function, unfortunately I can't think of a better way to do this.

However if you need this for doing calculations, then whether you need to generate your distribution as a RNG for that distribution, or whether you need to calculate the PDF densities, you could still probably get away with using for the first one, simulation algorithms like Metropolis-Hastings or something else and for the second one, just standard numerical integration techniques.

The key is figuring out the error you wish to tolerate and go from there.

Unfortunately I can't help you with getting a closed form PDF and since this will be the equivalent PDF, then if it is in a closed form, it must be able to generated from that integral.

If you need to use it for some applied purposes or even in a serious application like engineering, I would still urge you to consider using a numerical integration scheme and RNG algorithm like Metropolis-Hastings with some gauranteed error-bound for required accuracy since using the right implementation will give you something very accurate.

Perhaps you could talk this over with a senior engineer if you are doing this for some specific application where accuracy is vital, or some other senior scientist or your boss if this is vital in another endeavor.

Indeed it is for a wireless communication system, and I need closed form because I will use it in several steps further. For example, after I find the p.d.f, I need to compute the average probability of error using that p.d.f. Also, the existence of closed form expression helps giving more insight about the system performance.

 

[EngWiPy]

What about using upper bound like:

$$Z\geq\frac{1}{NX_{min}^{-1}}=\frac{X_{min}}{N}$$

Is this relation true?

 

[chiro]

What about using upper bound like:

$$Z\geq\frac{1}{NX_{min}^{-1}}=\frac{X_{min}}{N}$$

Is this relation true?

In terms of Z^-1 (not Z), our upper bound for this will be

$$NX_{max}^{-1}$$

which means our upper bound for Z should be

$$frac{1}{NX_{min}^-1}}$$

which is true.

As for the RHS argument, the problem is that $$frac{1}{X_{min}^-1}$$ is not the same as $$X_{min}$$ because the minimum of say X is not the same as the reciprocal of the minimum of 1/X. The For example with {1,4} as our set 1 = min(X), but 1/4 = Min(1/X).

Did you mean to say max of something instead of min?

 

[EngWiPy]

In terms of Z^-1 (not Z), our upper bound for this will be

$$NX_{max}^{-1}$$

which means our upper bound for Z should be

$$frac{1}{NX_{min}^-1}}$$

which is true.

As for the RHS argument, the problem is that $$frac{1}{X_{min}^-1}$$ is not the same as $$X_{min}$$ because the minimum of say X is not the same as the reciprocal of the minimum of 1/X. The For example with {1,4} as our set 1 = min(X), but 1/4 = Min(1/X).

Did you mean to say max of something instead of min?

Oh, you are the same guy who I was arguing with on another section about this! I did not notice that.

No, this is different, I am not saying:

$$\underset{m}{\text{min}}X_m^{-1}$$

but I am saying:

$$X_{\text{min}}^{-1}$$

and simply:

$$\frac{1}{X_{\text{min}}^{-1}}=X_{\text{min}}$$

 

[chiro]

Oh, you are the same guy who I was arguing with on another section about this! I did not notice that.

No, this is different, I am not saying:

$$\underset{m}{\text{min}}X_m^{-1}$$

but I am saying:

$$X_{\text{min}}^{-1}$$

and simply:

$$\frac{1}{X_{\text{min}}^{-1}}=X_{\text{min}}$$

Yeah I'm that guy :).

Well as long as the middle term is correct and your RHS is what you meant in your latest post then that should be fine.

The easiest way to prove it if you want to is you could start off with 1/Z and show the supremum of that and then show what happens when you invert 1/Z to get Z in terms of lower or upper bounds or start with 1/Z first.

Since this is for a wireless communications system I would double check to make absolutely sure, but if you need a bit of guidance to prove it then you could start off with the 1/Z case in terms of min and maximums and then look at what happens to 1/Z in terms of same min and maximums and keep in mind what happens when you take the reciprocal (max becomes min and min becomes max). That's the idea behind how to show it.