How to prove Bayes' rule for probability measures?

[winterfors]

Consider a probability space $(\Theta, \Sigma_\Theta, P_\Theta)$, where $P_\Theta$ is a probability measure on the sigma-algebra $\Sigma_\Theta$.

Each element $x \in \Theta$ maps onto another probability measure $P_{\Omega | x}$, on a sigma-algebra $\Sigma_\Omega$ on another space $\Omega$.

In this situation, one should (as far as I can see) be able to write write a measure-theoretic generalization of Bayes' rule

$${P_{\Theta |y}}(A) = \int\limits_{x \in A} {\frac{{d{P_{\Omega |x}}}}{{d{P_\Omega }}}(y)d{P_\Theta }}$$

for any $A \subseteq \Theta$, given an observation $y \in \Omega$ where

$${P_\Omega } = \int\limits_{x \in \Theta } {{P_{\Omega |x}}d{P_\Theta }}$$

and ${d{P_{\Omega |x}}/d{P_\Omega }}$ is the Radon–Nikodym derivative of $P_{\Omega |x}$ with respect to $P_{\Omega}$.


The problem is that I cannot see how to prove it (I;m sure the proof is fairly simple). Anyone wants to help?

 

[SW VandeCarr]

This proof may or may not be what you are looking for. It's based on a filtration: $F_1, F_2, ...,F_{N-1}, F_N$ as an increasing sequence of sigma algebras.

http://01law.wordpress.com/2011/04/09/bayes-rule-and-forward-measure/

 

[winterfors]

This proof may or may not be what you are looking for. It's based on a filtration: $F_1, F_2, ...,F_{N-1}, F_N$ as an increasing sequence of sigma algebras.

http://01law.wordpress.com/2011/04/09/bayes-rule-and-forward-measure/

As far as I can see, it's not quite what I'm looking for. What I'm trying to do above is to reformulate Bayes' rule for probability densities, usually expressed

$$p(x|y) = \frac{{p(y|x)}}{{p(y)}}p(x)$$

which follows trivially from the definition of a joint probability density $p(x,y) = p(y|x) p(x)$. But for probabilty measures, it gets slightly more tricky...

 

[SW VandeCarr]

As far as I can see, it's not quite what I'm looking for. What I'm trying to do above is to reformulate Bayes' rule for probability densities, usually expressed

$$p(x|y) = \frac{{p(y|x)}}{{p(y)}}p(x)$$

which follows trivially from the definition of a joint probability density $p(x,y) = p(y|x) p(x)$. But for probabilty measures, it gets slightly more tricky...

The rule for probability densities follows from Bayes' Rule and the Law of Total Probabilities:

$f_X(x|Y=y) = \frac{f_{X,Y}(x,y)}{f_Y(y)} = \frac{f_Y(y|X=x)\,f_X(x)}{f_Y(y)} = \frac{f_Y(y|X=x)\,f_X(x)}{\int_{-\infty}^{\infty} f_Y(y|X=\xi )\,f_X(\xi )\,d\xi }\!$.

Do you want to reformulate this?

EDIT: Any reformulation will need to take into account that the application of Bayes' Rule must be expressed as a posterior probability which is defined in terms of a probability space and the Law of Total Probabilities.

 

[vladb]

Each element $x \in \Theta$ maps onto another probability measure $P_{\Omega | x}$, on a sigma-algebra $\Sigma_\Omega$ on another space $\Omega$.

What is this mapping? How exactly is $P_{\Omega | x}$ defined?

 

[winterfors]

What is this mapping? How exactly is $P_{\Omega | x}$ defined?

I only meant that for every $x \in \Theta$ there is one probability measure $P_{\Omega | x}$ on $\Sigma_\Omega$ over the space $\Omega$. The probaility measures $P_{\Omega | x}$ are thus consitional on $x$.

This allows us to define a joint probability measure on the (Cartesian) product space $(\Theta \times \Omega ,{\Sigma _\Theta } \times {\Sigma _\Omega })$
$${P_{\Theta \times \Omega }}(C) \equiv \int\limits_{x\in A} {{P_{\Omega |x}}({B_x})d{P_\Theta }}$$ for any $C \in {\Sigma _\Theta } \times {\Sigma _\Omega }$, where $A \in {\Sigma _\Theta }$ and ${B_x} \in {\Sigma _\Omega }$ are defined as $A = \{ x:(x,y) \in C\}$ and ${B_x} = \{ y:(x,y) \in C\}$.If one could prove that $${P_{\Omega |x}} = \frac{{d{P_{\Theta \times \Omega }}}}{{d{P_\Theta }}}(x)$$ where $d{P_{\Theta \times \Omega }}/d{P_\Theta }$ is the Radon–Nikodym derivative of ${P_{\Theta \times \Omega }}$ with respect to ${P_\Theta }$, then the theorem above (that I want to prove), i.e.
$${P_{\Theta |y}}(A) = \int\limits_{x\in A} {\frac{{d{P_{\Omega |x}}}}{{d{P_\Omega }}}(y)d{P_\Theta }}$$ would follow by symmetry, but I'm not sure of how to do that...

 

[vladb]

This allows us to define a joint probability measure on the (Cartesian) product space $(\Theta \times \Omega ,{\Sigma _\Theta } \times {\Sigma _\Omega })$
$${P_{\Theta \times \Omega }}(C) \equiv \int\limits_{x\in A} {{P_{\Omega |x}}({B_x})d{P_\Theta }}$$

Perhaps I've misunderstood you, but... $f$ would be a Radon-Nikodym derivative if
$$P_{\Theta\times\Omega}(A) = \int_A f \, dP_\Theta$$ for all $A \in \Sigma_\Theta\times\Sigma_\Omega$ (product sigma-field). However this doesn't make any sense, because you have different measurable spaces on the left and right hand sides. Actually, the right hand side does not mean anything.

 

[winterfors]

Perhaps I've misunderstood you, but... $f$ would be a Radon-Nikodym derivative if
$$P_{\Theta\times\Omega}(A) = \int_A f \, dP_\Theta$$ for all $A \in \Sigma_\Theta\times\Sigma_\Omega$ (product sigma-field). However this doesn't make any sense, because you have different measurable spaces on the left and right hand sides. Actually, the right hand side does not mean anything.

In the equation you refer to, the integration is not over a subset $A \in \Sigma_\Theta\times\Sigma_\Omega$ but over a subset of $\Theta$.

 

[vladb]

In the equation you refer to, the integration is not over a subset $A \in \Sigma_\Theta\times\Sigma_\Omega$ but over a subset of $\Theta$.

I was just quoting your definition of $P_{\Theta\times\Omega}$. Later in you question you refer to $\frac{dP_{\Theta\times\Omega}}{dP_\Omega}$ as a Radon-Nikodym derivative, but this, as I tried to point out in my previous reply, doesn't make sense.

 

[winterfors]

I was just quoting your definition of $P_{\Theta\times\Omega}$. Later in you question you refer to $\frac{dP_{\Theta\times\Omega}}{dP_\Omega}$ as a Radon-Nikodym derivative, but this, as I tried to point out in my previous reply, doesn't make sense.

Yeah, I meant that ${dP_{\Theta\times\Omega}}/{dP_\Omega}$ would not be a function $\Theta\rightarrow[0,\infty)$, but a function $\Theta\rightarrow\Gamma$, where $\Gamma$ is a set of probaility measures on $\Sigma_\Omega$. It might be a bit of a strech to call it a Radon-Nikodym derivative...

 

[winterfors]

The rule for probability densities follows from Bayes' Rule and the Law of Total Probabilities:

$f_X(x|Y=y) = \frac{f_{X,Y}(x,y)}{f_Y(y)} = \frac{f_Y(y|X=x)\,f_X(x)}{f_Y(y)} = \frac{f_Y(y|X=x)\,f_X(x)}{\int_{-\infty}^{\infty} f_Y(y|X=\xi )\,f_X(\xi )\,d\xi }\!$.

Do you want to reformulate this?

EDIT: Any reformulation will need to take into account that the application of Bayes' Rule must be expressed as a posterior probability which is defined in terms of a probability space and the Law of Total Probabilities.

Yes, reformulating this using probaility measures instead of probability densities would allow me to prove what I want.

 

[vladb]

Hmm.. you denote by $P_{\Omega | x}$ just some arbitarily chosen probability measure on $(\Omega, \Sigma_\Omega)$, i.e. you have a family of measures parametrized by $x \in \Theta$. Then you define

a joint probability measure on the (Cartesian) product space $(\Theta \times \Omega ,{\Sigma _\Theta } \times {\Sigma _\Omega })$
$${P_{\Theta \times \Omega }}(C) \equiv \int\limits_{x\in A} {{P_{\Omega |x}}({B_x})d{P_\Theta }}$$

To answer the question, whether
$${P_{\Omega |x}} = \frac{{d{P_{\Theta \times \Omega }}}}{{d{P_\Theta }}}(x)$$, where the RHS is not a Radon-Nikodym derivative, you need to define the RHS, i.e. it cannot be just some function $\Theta \to \Gamma$. Also you can't define it using the same equation you used for definition of $P_{\Theta\times\Omega}$. A theorem would show that two separately defined things are equal, but I see only one defined object.

I sort of understand what you are trying to do, but not quite. Can you give a simple example of calculations with concrete values/sets of what the theorem would look like? E.g. in discrete case?