MHB Defining a Probability Distribution with Measure Spaces and Delta Functions

[mathmari]

Hey!

Let $M$ be a measure space and $(a_i)_{i\in \mathbb{N}}\subset M$. I want to show that for positive $p_1, \ldots , p_n$ with $\displaystyle{\sum_{i=1}^np_i=1}$ by $\displaystyle{Q=\sum_{i=1}^np_i\delta_{a_i}}$ a probability distribution is defined. Do we have to show that $$Q(x)=P(X=x)=\left\{\begin{matrix}
p_i & \text{ for } x=x_i, \ (i=1, 2, \ldots , n) \\
0 & \text{ otherwise }
\end{matrix}\right.$$ where $\displaystyle{\sum_{i=1}^np_i=1}$ ? (Wondering) We have the following:

Since $\delta_{a_i}(x)=\left\{\begin{matrix}
1 & \text{ if } x\in a_i \\
0 & \text{ otherwise }
\end{matrix}\right.$

we get
$$Q(x)=\sum_{i=1}^np_i\delta_{a_i}(x)=\sum_{i=1}^n\left (p_i\cdot \left\{\begin{matrix}
1 & \text{ if } x\in a_i \\
0 & \text{ otherwise }
\end{matrix}\right.\right )$$

Is this correct so far? How could we continue? (Wondering)

 

[Euge]

The question is stated strangely. What is the domain of $Q$, and what is $X$?

 

[mathmari]

The question is stated strangely. What is the domain of $Q$, and what is $X$?

By $X$ I mean a random variable. Shouldn't the domain of $Q$ be the random variable $X$ and the range $[0,1]$ ? (Wondering)

 

[I like Serena]

It looks as if we need to verify that the conditions/axioms of a probability distribution are satisfied.

Can you provide us with the definition of a probability distribution?
And also the definition of a measure space? (Wondering)

 

[mathmari]

A probability distribution is a measure $P$ on the assumed probability space $\Omega$ with the total mass $1$, i.e. $P(\Omega ) = 1$. So, do we have to show in this case that $Q(M)=1$ ? (Wondering)

 

[Euge]

It's as I thought -- your definition of a probability distribution is the same as the definition of a probability measure, not a probability distribution function. So in particular, the domain of $Q$ is the sigma-algebra on M, not M itself.

The Dirac masses $\delta_{a_i}$ concentrated at $a_i$ are positive measures on $M$, so $Q$, being a linear combination of those measures, is a positive measure on $M$. Now since the $a_i\in M$, $\delta_{a_i}(M) = 1$ for each $i$, and hence $Q(M) = \sum p_i \delta_{a_i}(M) = \sum p_i = 1$. This shows that $Q$ is a probability measure on $M$.

 

[I like Serena]

A probability distribution is a measure $P$ on the assumed probability space $\Omega$ with the total mass $1$, i.e. $P(\Omega ) = 1$. So, do we have to show in this case that $Q(M)=1$ ? (Wondering)

It's as I thought -- your definition of a probability distribution is the same as the definition of a probability measure, not a probability distribution function. So in particular, the domain of $Q$ is the sigma-algebra on M, not M itself.

The Dirac masses $\delta_{a_i}$ concentrated at $a_i$ are positive measures on $M$, so $Q$, being a linear combination of those measures, is a positive measure on $M$. Now since the $a_i\in M$, $\delta_{a_i}(M) = 1$ for each $i$, and hence $Q(M) = \sum p_i \delta_{a_i}(M) = \sum p_i = 1$. This shows that $Q$ is a probability measure on $M$.

I take it that our measure space is actually a measurable space given by the tuple $(\Omega, M)$?
And we have the Dirac measures $\delta_{a_i}$ on it?
And we have the presumed probability measure $P$ for the presumed probability space $(\Omega, M, P)$?

According to the definition of Probability measure, we would need to verify that:

• $P$ is countably additive (also called σ-additive): if ${\displaystyle \{A_{i}\}_{i=1}^{\infty }\subseteq {M}}$ is a countable collection of pairwise disjoint sets, then ${\displaystyle \textstyle P(\bigcup _{i=1}^{\infty }A_{i})=\sum _{i=1}^{\infty }P(A_{i}),}$
• the measure of entire sample space is equal to one: ${\displaystyle P(\Omega )=1}$.

Don't we? (Wondering)

It also seems to mean that the domain of $Q$ is $\Omega$ isn't it?
That is, not the σ-algebra $M$, but the set of outcomes $\Omega$. (Wondering)

 

[Euge]

It's common abuse of notation to represent a measure space by the underlying set. So It's fair to assume that $M$ is the underlying set.

 

[Euge]

I take it that our measure space is actually a measurable space given by the tuple $(\Omega, M)$?

Sometimes the terms measure space and measurable space are used interchangeably. I'm referring to $M$ as a measurable space, and my argument shows (assuming that one already knows Dirac masses are measures) that $Q$ is a probability measure on $M$, that is, $Q$ is a probability measure on the sigma-algebra of $M$ (the underlying set).

 

[mathmari]

According to the definition of Probability measure, we would need to verify that:

• $P$ is countably additive (also called σ-additive): if ${\displaystyle \{A_{i}\}_{i=1}^{\infty }\subseteq {M}}$ is a countable collection of pairwise disjoint sets, then ${\displaystyle \textstyle P(\bigcup _{i=1}^{\infty }A_{i})=\sum _{i=1}^{\infty }P(A_{i}),}$
• the measure of entire sample space is equal to one: ${\displaystyle P(\Omega )=1}$.

I got stuck right now. Besides $Q(\Omega)=1$ do we have to show that ${\displaystyle \textstyle Q(\bigcup _{i=1}^{\infty }A_{i})=\sum _{i=1}^{\infty }Q(A_{i})}$ ? (Wondering)

 

[I like Serena]

I got stuck right now. Besides $Q(\Omega)=1$ do we have to show that ${\displaystyle \textstyle Q(\bigcup _{i=1}^{\infty }A_{i})=\sum _{i=1}^{\infty }Q(A_{i})}$ ? (Wondering)

I believe so, since it's part of the definition on wiki, although you did not mention it in your definition. (Nerd)

Either way, I'm still trying to sort out which sets, $\sigma$-algebras, and functions we have exactly.
Rereading the problem statement and Euge's comments I'm deducing that we do not have a set of outcomes $\Omega$. Instead we have a set of outcomes $M$.
We have an as yet unspecified $\sigma$-algebra - let's call it $\mathcal F$ - and also an unspecified measure.
I'm still not clear on the domain of $\delta_{a_i}$.
It doesn't seem to match the wiki article on Dirac measure's.
Instead it seems to match the indicator function $1_{a_i}$ that is also mentioned there.
Consequently I'm not clear on the domain of $Q$ either.

Can you perhaps clarify? (Wondering)

 

[Euge]

To clarify matters, let's first be more explicit, not abusing notation. We have a measurable space $(\Omega, \mathcal{F})$ where $\Omega$ is a set and $\mathcal{F}$ is a sigma-field (or sigma-algebra) on $\Omega$. Also, we fix points $a_1,\ldots, a_n\in \Omega$, and let $p_1,\ldots p_n$ be positive numbers such that $\sum p_i = 1$. Then we consider the mapping $Q : \mathcal{F} \to [0, \infty)$ given by $Q = \sum p_i \delta_{a_i}$. The goal is to prove $Q$ is a probability measure on $\Omega$.

If we are to prove $Q$ is a probability measure directly from the axioms, we must show that $Q(\emptyset) = 0$, $Q(\Omega) = 1$, $0 \le Q(A) \le 1$ for all $A\in \mathcal{F}$, and $Q$ is countably additive, i.e., if $\{E_k\}_{k=1}^\infty$ is a sequence of events (that is, elements of $\mathcal{F}$) and $E = \cup E_k$, then $Q(E) = \sum_k Q(E_k)$.

 

[mathmari]

If we are to prove $Q$ is a probability measure directly from the axioms, we must show that $Q(\emptyset) = 0$, $Q(\Omega) = 1$, $0 \le Q(A) \le 1$ for all $A\in \mathcal{F}$, and $Q$ is countably additive, i.e., if $\{E_k\}_{k=1}^\infty$ is a sequence of events (that is, elements of $\mathcal{F}$) and $E = \cup E_k$, then $Q(E) = \sum_k Q(E_k)$.

I have done the following:

• $Q(\emptyset) = 0$ :
  
  It holds that $\delta_{a_i}(M)=1$ if $a_i\in M$ and $\delta_{a_i}(M)=0$ if $a_i\notin M$.
  
  Since the empty set cannot contain any element we have that $\delta_{a_i}(\emptyset)=0$, and so we get $\displaystyle{Q(\emptyset)=\sum_{i=1}^np_i\delta_{a_i}(\emptyset)=\sum_{i=1}^n\left (p_i\cdot 0\right )=0}$.
• $Q(\Omega) = 1$ :
  
  $\Omega$ is in this $M$, or not?
  
  Since $(a_i)_{i\in \mathbb{N}}\subset M$ it follows that $a_i\in M$ for each $i\in \mathbb{N}$.
  
  Therefore, we get $\displaystyle{Q(M)=\sum_{i=1}^np_i\delta_{a_i}(M)=\sum_{i=1}^n\left (p_i\cdot 1\right )=\sum_{i=1}^np_i=1}$.
• $0 \le Q(A) \le 1$ for all $A\in \mathcal{F}$ :
  
  Are the $A$'s in this case $(a_i)$ ?
• $Q\left (\cup E_k\right ) = \sum_k Q(E_k)$ :
  
  How can we show this property?

 

[Euge]

Let me add that the events $\{E_k\}$ are to be mutually exclusive so that they form a partition of $E$.

By definition, $Q(E) = \sum p_i \delta_{a_i}(E)$, and each $a_i$ belongs to most one of the $E_k$. Therefore $\delta_{a_i}(E) = \sum_k \delta_{a_i}(E_k)$. Hence $$Q(E) = \sum_i p_i\delta_{a_i}(E) = \sum_i p_i \sum_k \delta_{a_i}(E_k) = \sum_k \sum_i p_i\delta_{a_i}(E_k) =\sum_k Q(E_k)$$
To get $0 \le Q(A) \le 1$ for all $A\in \mathcal{F}$ (note the $A$'s are not $a_i$'s), use the fact that for all $A\in \mathcal{F}$, the $\delta_{a_i}(A)$ can only take values $0$ or $1$, and $\sum p_i = 1$.

 

[mathmari]

Let me add that the events $\{E_k\}$ are to be mutually exclusive so that they form a partition of $E$.

By definition, $Q(E) = \sum p_i \delta_{a_i}(E)$, and each $a_i$ belongs to most one of the $E_k$. Therefore $\delta_{a_i}(E) = \sum_k \delta_{a_i}(E_k)$.

We have that if $\delta_{a_i}(E)=0$ then $a_i\notin E$ and so $a_i\notin E_k$ for all $k$. This means that $\delta_{a_i}(E_k)=0$ for all $k$.

If $\delta_{a_i}(E)=1$, it follows that $a_i\in E$. From that we have that $a_i\in E_{k_0}$ for a $k_0$ and $a_i\notin E_k$ for all $k\neq k_0$. In other words we have that $\delta_{a_i}(E_{k_0})=1$ and $\delta_{a_i}(E_k)=0$ for $k\neq k_0$.

In both cases we get that $\delta_{a_i}(E) = \sum_k \delta_{a_i}(E_k)$.

Is the justification correct? (Wondering)

Hence $$Q(E) = \sum_i p_i\delta_{a_i}(E) = \sum_i p_i \sum_k \delta_{a_i}(E_k) = \sum_k \sum_i p_i\delta_{a_i}(E_k) =\sum_k Q(E_k)$$
To get $0 \le Q(A) \le 1$ for all $A\in \mathcal{F}$ (note the $A$'s are not $a_i$'s), use the fact that for all $A\in \mathcal{F}$, the $\delta_{a_i}(A)$ can only take values $0$ or $1$, and $\sum p_i = 1$.

I understand! (Smile)

We have a measurable space $(\Omega, \mathcal{F})$ where $\Omega$ is a set and $\mathcal{F}$ is a sigma-field (or sigma-algebra) on $\Omega$. Also, we fix points $a_1,\ldots, a_n\in \Omega$, and let $p_1,\ldots p_n$ be positive numbers such that $\sum p_i = 1$. Then we consider the mapping $Q : \mathcal{F} \to [0, \infty)$ given by $Q = \sum p_i \delta_{a_i}$.

In this case we have that $M$ is a measure space. Does this mean that the definition of $M$ is $(\Omega, \mathcal{F})$ ? (Wondering)

 

[Euge]

We have that if $\delta_{a_i}(E)=0$ then $a_i\notin E$ and so $a_i\notin E_k$ for all $k$. This means that $\delta_{a_i}(E_k)=0$ for all $k$.

If $\delta_{a_i}(E)=1$, it follows that $a_i\in E$. From that we have that $a_i\in E_{k_0}$ for a $k_0$ and $a_i\notin E_k$ for all $k\neq k_0$. In other words we have that $\delta_{a_i}(E_{k_0})=1$ and $\delta_{a_i}(E_k)=0$ for $k\neq k_0$.

In both cases we get that $\delta_{a_i}(E) = \sum_k \delta_{a_i}(E_k)$.

Is the justification correct? (Wondering)

Yes, that's correct!

In this case, we have that $M$ is a measure space. Does this mean that the definition of $M$ is $(\Omega, \mathcal{F})$? (Wondering)

Usually, we say $M$ is a measurable space, which is a pair $(\Omega, \mathcal{F})$ where $\Omega$ is a set and $\mathcal{F}$ is a sigma-algebra on $\Omega$.

 

[mathmari]

I understand! Thank you so much! (Yes)