Register to reply

Expected value on subset of Pareto

by xag
Tags: expected, pareto, subset
Share this thread:
xag
#1
Mar12-08, 12:37 PM
P: 6
Hi -
I have a Pareto distribution X (xm=1, k known)*. High-value samples are filtered out and I want the expected value of the remaining. Namely: E[X|X<a].

*Wikipedia notations

Many thanks in advance.
Xavier
Phys.Org News Partner Science news on Phys.org
Experts defend operational earthquake forecasting, counter critiques
EU urged to convert TV frequencies to mobile broadband
Sierra Nevada freshwater runoff could drop 26 percent by 2100
Pere Callahan
#2
Mar13-08, 09:30 AM
P: 587
What is the definition of the conditional expectation you are referring to? How did you try to calculate it? Where did you get stuck?
xag
#3
Mar13-08, 10:47 AM
P: 6
Hi Pere -

What I'm looking for is a mean value.
I found out that the samples of interest belong to a larger set that roughly follow a Pareto distribution, and that among this larger set, the samples of interest are the ones whose values are below a certain number, 'a'.

I can estimate the parameters of the Pareto once for all, and calculate 'a' for each run. But I can't access the samples for each run, so if there was a way to predict it with 'k' (the Pareto shape) and 'a', that would be great.

My first guess is that the samples of interest also follow a Pareto, but I've not shown that. And in this case, could we express its shape K and scale Xm? Then the mean I'm looking for would be K Xm/(K-1).

Best regards,
Xavier

xag
#4
Mar14-08, 02:07 PM
P: 6
Expected value on subset of Pareto

Ok. It seems I got all the way to the finish line and stopped just before :

I guess I just had to integrate the probability density function between 1 and a (it's defined between 1 and +oo).
It gives : (a ^(1 - k) - 1) * k / (1 - k)

Can anyone confirm?
Pere Callahan
#5
Mar15-08, 01:19 PM
P: 587
Quote Quote by xag View Post
I guess I just had to integrate the probability density function between 1 and a.
Can anyone confirm?
What you did gives you the probability of a sample with value less than a.

Try the following instead: First calculate the conditional distribution function[itex]\mathbb{P}(X\leqx| X \leq a)[/itex].
By definition this is equal to

[tex]
\frac{\mathbb{P}(X\leq x \wedge X \leq a)}{\mathbb{P}(X\leq a)}
[/tex]

(You already calculated the denominator.)

You can also easily calculate the numerator. For x>a the numerator is [itex]\mathbb{P}(X \leq a)[/itex] which cancels with denominator giving one. For x>a you calculate it in the same way as you did for [itex]\mathbb{P}(X \leq a)[/itex] with a replaced by x. Then you take the derivative with respect to x to find the conditional density function [itex]\rho_a(x)[/itex] (suported on [1,a]), which you then use to calculate the conditional expectation as
[tex]\mathbb{E}\left[X|X\leq a\right]=\int_1^a{x\rho_a(x)dx}[/tex]
A more direct though maybe less obvious formula would be
[tex]\mathbb{E}\left[X|X\leq a\right]=\frac{\int_1^a{x\rho(x)dx}}{\int_1^a{\rho(x)dx}}[/tex],
where [itex]\rho(x)[/itex] is the density function of your original Pareto distribution.
I suggest you try them both and check if they really are the same
-Pere
xag
#6
Mar15-08, 06:10 PM
P: 6
Quote Quote by xag View Post
Ok. It seems I got all the way to the finish line and stopped just before :

I guess I just had to integrate the probability density function between 1 and a (it's defined between 1 and +oo).
It gives : (a ^(1 - k) - 1) * k / (1 - k)

Can anyone confirm?
Forget this post: the thing I integrated was x * dF(x) and moreover the result I gave here is wrong.
xag
#7
Mar15-08, 06:17 PM
P: 6
Quote Quote by Pere Callahan View Post
Try the following instead: First calculate the conditional distribution function[itex]\mathbb{P}(X\leqx| X \leq a)[/itex].
[...]
I suggest you try them both and check if they really are the same
-Pere
Thank you so much.

In both cases I find [tex] \mathbb{E}\left[X|X\leq a\right]= \frac{k}{k - 1} \frac{a ^ k - a}{a ^ k - 1}[/tex]


Register to reply

Related Discussions
Subset vs proper subset? Set Theory, Logic, Probability, Statistics 22
Subset and proper subset Set Theory, Logic, Probability, Statistics 6
The Pareto and Exponential Distr. Where is the mistake? Calculus & Beyond Homework 2
Expected Value, Expected Variance,covariance General Math 0
Pareto Heavy Tailed Distribution Set Theory, Logic, Probability, Statistics 0