Simple Expectation Value Question

[Harrisonized]

I was told that given a probability distribution p(x) dx, the expected value for x is given by:

<x> = Ʃ x_i P(x_i) = ∫ x P(x) dx

This part makes sense to me. It was justified to me through the use of weighted averages. However, my teacher then made a hand-wavy move to generalize the above formula. I quote:

This way of calculating the average can be easily generalized, since it depends neither on the numbers of different events nor on the total number of events, it only depends on the probabilities of all different possibilities. So we can consider an experiment where we are measuring some quantity x, and all the possible outcomes are x₁, x₂, ... , x_n. If we denote the probability of the outcome x_i to be P(x_i) then we can write the average of x as

<x>= Ʃ x_i P(x_i) = ∫ x P(x) dx (17)

We may also be interested in calculating the average of some given function of x, call it f(x). The different possible values of f(x) are f(x₁), f(x₂), ... , f(x_n), and the probability P(f(x_i)) of the value f(x_i) is, of course, the same as for x to have the value x_i, i.e. P(x_i)

P(f(x_i)) = P(x_i)

We can now use the rule (17) to find the average of f(x)

<f>= Ʃ f(x_i) p(f(x_i)) = ∫ f(x) P(f(x)) dx
<f>= Ʃ f(x_i) p(x_i) ∫ f(x) P(x) dx

-end of quote-

It's this last part I don't understand:

P(f(x_i)) = P(x_i)

I don't see how this can be true for anything other than f(x_i)=x_i or P(x_i)=(a constant). Can someone please justify this to me?

-----

Edit: after extensive searching, I finally came across this:

http://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician

I've been searching for a long time now, and I still haven't found justification for why this works.

 

[clamtrox]

Yeah that's really stupid way of putting it.

The expectation value of a function is just
<f> = Ʃ f(x_i) P(x_i) though,
right? That is, it's the average of the function weighted by the probability density of the individual events.

Now your teacher is using horrible notation P(f(x_i)) for the weight of each f(x_i) in the sum; in a rigorous sense it's complete nonsense, but his underlying point is correct.

 

[Ray Vickson]

I was told that given a probability distribution p(x) dx, the expected value for x is given by:

<x> = Ʃ x_i P(x_i) = ∫ x P(x) dx

This part makes sense to me. It was justified to me through the use of weighted averages. However, my teacher then made a hand-wavy move to generalize the above formula. I quote:

This way of calculating the average can be easily generalized, since it depends neither on the numbers of different events nor on the total number of events, it only depends on the probabilities of all different possibilities. So we can consider an experiment where we are measuring some quantity x, and all the possible outcomes are x₁, x₂, ... , x_n. If we denote the probability of the outcome x_i to be P(x_i) then we can write the average of x as

<x>= Ʃ x_i P(x_i) = ∫ x P(x) dx (17)

We may also be interested in calculating the average of some given function of x, call it f(x). The different possible values of f(x) are f(x₁), f(x₂), ... , f(x_n), and the probability P(f(x_i)) of the value f(x_i) is, of course, the same as for x to have the value x_i, i.e. P(x_i)

P(f(x_i)) = P(x_i)

We can now use the rule (17) to find the average of f(x)

<f>= Ʃ f(x_i) p(f(x_i)) = ∫ f(x) P(f(x)) dx
<f>= Ʃ f(x_i) p(x_i) ∫ f(x) P(x) dx

-end of quote-

It's this last part I don't understand:

P(f(x_i)) = P(x_i)

I don't see how this can be true for anything other than f(x_i)=x_i or P(x_i)=(a constant). Can someone please justify this to me?

-----

Edit: after extensive searching, I finally came across this:

http://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician

I've been searching for a long time now, and I still haven't found justification for why this works.

As you have already realized, the result E f(X) = \sum P(X=x) f(x) or E f(X) = \int f(x) dP(X=x) is true, and is know as the "law of the unconscious statistician", because it was reputed to be used indiscriminately by statisticians without knowing that it is a *theorem* rather than a definition. It can be found in Section 7 of
http://www.math.harvard.edu/~lauren/154/Outline8-DRV-Expect2009.pdf .
There the discussion is pretty elementary but also quite detailed. After reading it you will understand why your teacher's explanation is only partly correct. This link is much more informative than the one you gave above.

RGV