# Expected value on subset of Pareto

#### xag

Hi -
I have a Pareto distribution X (xm=1, k known)*. High-value samples are filtered out and I want the expected value of the remaining. Namely: E[X|X<a].

*Wikipedia notations

Xavier

Related Set Theory, Logic, Probability, Statistics News on Phys.org

#### Pere Callahan

What is the definition of the conditional expectation you are referring to? How did you try to calculate it? Where did you get stuck?

#### xag

Hi Pere -

What I'm looking for is a mean value.
I found out that the samples of interest belong to a larger set that roughly follow a Pareto distribution, and that among this larger set, the samples of interest are the ones whose values are below a certain number, 'a'.

I can estimate the parameters of the Pareto once for all, and calculate 'a' for each run. But I can't access the samples for each run, so if there was a way to predict it with 'k' (the Pareto shape) and 'a', that would be great.

My first guess is that the samples of interest also follow a Pareto, but I've not shown that. And in this case, could we express its shape K and scale Xm? Then the mean I'm looking for would be K Xm/(K-1).

Best regards,
Xavier

#### xag

Ok. It seems I got all the way to the finish line and stopped just before :

I guess I just had to integrate the probability density function between 1 and a (it's defined between 1 and +oo).
It gives : (a ^(1 - k) - 1) * k / (1 - k)

Can anyone confirm?

#### Pere Callahan

I guess I just had to integrate the probability density function between 1 and a.
Can anyone confirm?
What you did gives you the probability of a sample with value less than a.

Try the following instead: First calculate the conditional distribution function$\mathbb{P}(X\leqx| X \leq a)$.
By definition this is equal to

$$\frac{\mathbb{P}(X\leq x \wedge X \leq a)}{\mathbb{P}(X\leq a)}$$

You can also easily calculate the numerator. For x>a the numerator is $\mathbb{P}(X \leq a)$ which cancels with denominator giving one. For x>a you calculate it in the same way as you did for $\mathbb{P}(X \leq a)$ with a replaced by x. Then you take the derivative with respect to x to find the conditional density function $\rho_a(x)$ (suported on [1,a]), which you then use to calculate the conditional expectation as
$$\mathbb{E}\left[X|X\leq a\right]=\int_1^a{x\rho_a(x)dx}$$
A more direct though maybe less obvious formula would be
$$\mathbb{E}\left[X|X\leq a\right]=\frac{\int_1^a{x\rho(x)dx}}{\int_1^a{\rho(x)dx}}$$,
where $\rho(x)$ is the density function of your original Pareto distribution.
I suggest you try them both and check if they really are the same
-Pere

Last edited:

#### xag

Ok. It seems I got all the way to the finish line and stopped just before :

I guess I just had to integrate the probability density function between 1 and a (it's defined between 1 and +oo).
It gives : (a ^(1 - k) - 1) * k / (1 - k)

Can anyone confirm?
Forget this post: the thing I integrated was x * dF(x) and moreover the result I gave here is wrong.

#### xag

Try the following instead: First calculate the conditional distribution function$\mathbb{P}(X\leqx| X \leq a)$.
[...]
I suggest you try them both and check if they really are the same
-Pere
Thank you so much.

In both cases I find $$\mathbb{E}\left[X|X\leq a\right]= \frac{k}{k - 1} \frac{a ^ k - a}{a ^ k - 1}$$

"Expected value on subset of Pareto"

### Physics Forums Values

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving