Probability function for discrete functions

  • #1
Math_QED
Science Advisor
Homework Helper
2019 Award
1,655
675

Main Question or Discussion Point

My textbook says that if ##X: \Omega \to \mathbb{R}## is discrete stochast (I.e., there are only countably many values that get reached), then it suffices to know the probability function ##p(x) = \mathbb{P}\{X =x\}## in order to know the distribution function ##\mathbb{P}_X: \mathcal{R} \to \mathbb{R}: A \mapsto \mathbb{P}\{X \in A\} = \mathbb{P}(X^{-1}A)##


Indeed, if ##S:= \{x : p(x) > 0\}##, then for ##A \in \mathcal{R}##, it follows that $$\mathbb{P}\{X \in A\} = \sum_{x \in S \cap A}p(x)$$

But how do they get this formula?

I tried the following:

$$\mathbb{P}\{X \in A\} = \mathbb{P}\left(X^{-1}\left(\bigcup_{a \in S\cap A}\{a\} \cup\bigcup_{a \in A\setminus S}\{a\}\right)\right) $$

$$= \mathbb{P}\left(X^{-1}\left(\bigcup_{a \in S \cap A}\{a\}\right)\right) + \mathbb{P}\left(X^{-1}\left(\bigcup_{a \in A\setminus S}\{a\}\right)\right) $$
$$=\sum_{a \in S \cap A}p(a) + \mathbb{P}\left(X^{-1}\left(\bigcup_{a \in A\setminus S}\{a\}\right)\right)$$

But how do I show that the probability on the right is zero? I can't use ## \sigma##-additivity on uncountable disjoint unions.

EDIT: ##\mathcal{R}## is the smallest sigma algebra that contains the usual topology on the real numbers, i.e. it are the Borel parts of the reals.
 
Last edited:

Answers and Replies

  • #2
StoneTemplePython
Science Advisor
Gold Member
2019 Award
1,163
566
$$=\sum_{a \in S \cap A}p(a) + \mathbb{P}\left(X^{-1}\left(\bigcup_{a \in A\setminus S}\{a\}\right)\right)$$

But how do I show that the probability on the right is zero? I can't use ## \sigma##-additivity on uncountable disjoint unions.
I'll take a shot in the dark here-- I don't totally follow your notation or even what a "distribution function" is (generally means CDF but...). The problem should be easy if you can find a way to focus your attention on countable sets.

In any case, your problem reminded me of a quote from Kolmogorov that I like: "Behind every theorem lies an inequality."

- - - -

You have some special structure in that probabilities are always real non-negative and sum to one. The axioms immediately give that

##\Pr\{B\} + \Pr\{B^c\} = 1##

supposing you are allowed to use that, perhaps you can try re-running your argument over something complementary?

The goal would be to combine the two via linearity, show that your probabilities sum to one, and you end up with

##0\leq \mathbb{P}\left(X^{-1}\left(\bigcup_{a \in A\setminus S}\{a\}\right)\right) \leq 0##

(or equivalently: take advantage of positive definiteness after subtracting 1 from each side of your equation -- but the idea is to show that the thing in question is bounded above and below by zero, and hence is zero.)

- - - -
Hopefully this helps or at least provides some inspiration toward a way to get the result you want via an inequality.
 
Last edited:
  • Like
Likes Math_QED
  • #3
Math_QED
Science Advisor
Homework Helper
2019 Award
1,655
675
I'll take a shot in the dark here-- I don't totally follow your notation or even what a "distribution function" is (generally means CDF but...). The problem should be easy if you can find a way to focus your attention on countable sets.

In any case, your problem reminded me of a quote from Kolmogorov that I like: "Behind every theorem lies an inequality."

- - - -

You have some special structure in that probabilities are always real non-negative and sum to one. The axioms immediately give that

##\Pr\{B\} + \Pr\{B^c\} = 1##

supposing you are allowed to use that, perhaps you can try re-running your argument over something complementary?

The goal would be to combine the two via linearity, show that your probabilities sum to one, and you end up with

##0\leq \mathbb{P}\left(X^{-1}\left(\bigcup_{a \in A\setminus S}\{a\}\right)\right) \leq 0##

(or equivalently: take advantage of positive definiteness after subtracting 1 from each side of your equation -- but the idea is to show that the thing in question is bounded above and below by zero, and hence is zero.)

- - - -
Hopefully this helps or at least provides some inspiration toward a way to get the result you want via an inequality.
Thanks. I defined probability distribution at the beginning of the post. I will take some time to digest your answer.
 
  • #4
mathman
Science Advisor
7,839
439
The statement looks obvious. Since the positive probability elements are countable (elements of S), to get P(A), add up all the probabilities of those elements in A which have positive probabilities.
 
  • #5
Math_QED
Science Advisor
Homework Helper
2019 Award
1,655
675
The statement looks obvious. Since the positive probability elements are countable (elements of S), to get P(A), add up all the probabilities of those elements in A which have positive probabilities.
I don't quite understand what you mean. Can you elaborate?
 
  • #6
mathman
Science Advisor
7,839
439
I don't quite understand what you mean. Can you elaborate?
I am not sure what I need to say. S is a subset (countable) of R containing all the points with positive probability. A is a subset of R. The probability of A is the sum of the probabilities of all points of A which have positive probability. Since S consists of all points of probability, the intersection of S and A contains all the points of A needed to define the probability of A.
 
  • Like
Likes Math_QED
  • #7
Math_QED
Science Advisor
Homework Helper
2019 Award
1,655
675
I am not sure what I need to say. S is a subset (countable) of R containing all the points with positive probability. A is a subset of R. The probability of A is the sum of the probabilities of all points of A which have positive probability. Since S consists of all points of probability, the intersection of S and A contains all the points of A needed to define the probability of A.
I think I understand what you mean. Let me write it out:

$$P(X \in A) = P(X\in A\cap X(\Omega))$$
$$= \sum_{x \in A \cap X(\Omega)} P(X=x)$$
$$=\sum_{x\in A \cap S} P(X=x) + \sum_{x \in A \cap X(\Omega) \setminus S} P(X=x)$$

And the last sum is 0. Does this make sense? In the second equality I used that the image of X is at most countable and for the last equality that ##S \subseteq X(\Omega)##
 
  • #8
mathman
Science Advisor
7,839
439
Your notation throws me. That's why I prefer words. [tex]What\ is\ X(\Omega)?[/tex] X does not have to be countable - only the subset of X consisting of points of positive probability.
 
  • #9
Math_QED
Science Advisor
Homework Helper
2019 Award
1,655
675
Your notation throws me. That's why I prefer words. [tex]What\ is\ X(\Omega)?[/tex] X does not have to be countable - only the subset of X consisting of points of positive probability.
##X(\Omega) = Im(X)##, the image of the function ##X##. I.e. all the values that X attain. And clearly this is countable, because by assumption this is countable (the variable is discrete)
 
  • #10
mathman
Science Advisor
7,839
439
I think I understand what you mean. Let me write it out:

$$P(X \in A) = P(X\in A\cap X(\Omega))$$
$$= \sum_{x \in A \cap X(\Omega)} P(X=x)$$
$$=\sum_{x\in A \cap S} P(X=x) + \sum_{x \in A \cap X(\Omega) \setminus S} P(X=x)$$

And the last sum is 0. Does this make sense? In the second equality I used that the image of X is at most countable and for the last equality that ##S \subseteq X(\Omega)##
In your original definition of X, it appears that X is countable, so there are only a countable number (if any) points in the second sum where each point has P(X=x)=0.
 
  • #11
Math_QED
Science Advisor
Homework Helper
2019 Award
1,655
675
In your original definition of X, it appears that X is countable, so there are only a countable number (if any) points in the second sum where each point has P(X=x)=0.
Every term in the last sum is zero, because the sum runs over elements not in S (so by definitions points with probability 0)
 
  • #12
mathman
Science Advisor
7,839
439
Every term in the last sum is zero, because the sum runs over elements not in S (so by definitions points with probability 0)
The point I was making is that if the number of points in X were not countable, the last term would be problematical. Summations have meaning only if the number of terms is countable.
 

Related Threads on Probability function for discrete functions

Replies
25
Views
8K
Replies
1
Views
4K
Replies
3
Views
1K
  • Last Post
Replies
1
Views
1K
Replies
2
Views
1K
Replies
6
Views
3K
Replies
7
Views
542
Replies
5
Views
2K
Replies
17
Views
20K
Top