An upper bound for a conditional probability

[sabbagh80]

Hi everyone,

The problem:
Is this relation true? If so, how (or maybe where) it could be proved?P(A│B∪C)≤P(A│B)+P(A│C)-P(A|BC)

and what about its possible generalization?
thanks a lot in advance.

 

[Stephen Tashi]

What about this:

$P(x)= 1/8$ for $x = 1,2,..,8$

$$B = \{ 1,2,3 \}$$
$$C = \{ 3,4,5 \}$$
$$A = \{2,3,4,6,7,8 \}$$

 

[sabbagh80]

so you mean that this inequality is not always true. But of course we have: P(A│B∪C)≤P(A│B)+P(A│C)
Can we find a better upper bound for P(A│B∪C)?

 

[SW VandeCarr]

so you mean that this inequality is not always true. But of course we have: P(A│B∪C)≤P(A│B)+P(A│C)
Can we find a better upper bound for P(A│B∪C)?

The upper bound of $$P(A|B\cup C)$$ is when $$P(A|B) + P(A|C)\geq P(A)$$. Draw Venn diagrams to see why.

 

[Stephen Tashi]

Can we find a better upper bound for P(A│B∪C)?

You'll have to explain what you are trying to do before anyone will know what "better"means in this context.

Using the idea:

$$P(X|Y) = P(Y|X) \frac{P(X)}{P(Y)}$$

and not worrying about the possibility of division by zero, you could try:

$$P(A| B \cup C) = P(B \cup C | A) \frac{P(A)}{P(B \cup C)}$$
$$= \Bigg( P(B|A) + P(C|A) - P(BC| A) \Bigg) \frac{P(A)}{P(B \cup C)}$$
$$= \Bigg( P(A|B) \frac{P(B)}{P(A)} + P(A|C) \frac{P(C)}{P(A)} - P(A|BC) \frac{P(BC}{P(A)} \Bigg) \frac{P(A)}{P(B \cup C)}$$
$$= P(A|B) \frac{P(B)}{P(B \cup C)} + P(A|C)\frac{P(C)}{P(B \cup C)} - P(A|BC)\frac{P(BC)}{P(B \cup C)}$$
$$\leq P(A|B) + P(A|C) - P(A|BC) \frac{P(BC)}{P(B \cup C)}$$

Another approach is the algebraic realization of a Venn diagram. The possible sets in this problem are: $$ABC, ABC^c, AB^cC,A^cBC,A^cBC^c,A^cB^cC,A^cB^cC^c$$
Each probability, such as $$P(A|B)$$ can be expressed as a fraction involving sums of the probabilities of those sets.

 

[sabbagh80]

by "better", I mean "tighter" than the derived upper bound. can we further improve this bound?

 

[Stephen Tashi]

by "better", I mean "tighter" than the derived upper bound. can we further improve this bound?

That doesn't clarify what you want because it doesn't say what quantities are desired on the right hand side.

The equality:
$$P(A|B \cup C) = P(A|B) \frac{P(B)}{P(B \cup C)} + P(A|C)\frac{P(C)}{P(B \cup C)} - P(A|BC)\frac{P(BC)}{P(B \cup C)}$$
is "tighter" than any inequality.

 

[sabbagh80]

That doesn't clarify what you want because it doesn't say what quantities are desired on the right hand side.

The equality:
$$P(A|B \cup C) = P(A|B) \frac{P(B)}{P(B \cup C)} + P(A|C)\frac{P(C)}{P(B \cup C)} - P(A|BC)\frac{P(BC)}{P(B \cup C)}$$
is "tighter" than any inequality.

you are right, I have not explain it precisely.
What I mean is this:
If I want to have both P(A|B) and P(A|C) on the right hand side of inequality and I want to find a coefficient for P(A|BC) such that the obtained expression be the tightest upper bound for the left side expression, what is it?
in other words,
$$P(A|B \cup C) \leq P(A|B)+P(A|C)-?P(A|B \cap C)$$

 

[Stephen Tashi]

See if I have rephrased you problem correctly:

To find
Max $K$ such that $P(A|B \cup C) \leq P(A|B)+P(A|C)- K P(A|B \cap C)$


Define:

$$P_1 = Pr( A B C)$$
$$P_2 = Pr( A B C^c)$$
$$P_3 = Pr( A B^c C)$$
$$P_4 = Pr( A B^c C^c)$$
$$P_5 = Pr( A^c B C)$$
$$P_6 = Pr( A^c B C^c)$$
$$P_7 = Pr( A^c B^c C)$$
$$P_8 = Pr( A^c B^c C^c)$$


The problem becomes: Minimize $K$ subject as a function of the $P_i$ subject to the constraints:

$$0 \leq P_i \leq 1$$
$$\sum_{i=1}^8 P_i = 1$$
$$\frac{P_1 + P_2 + P_3}{P_1 + P_2 + P_3 + P_5 + P_6 + P_7} = <br /> \frac{P_1 + P_2}{P_1 + P_2 + P_5 + P_6} + \frac{P_1 + P_3}{P_1 + P_3 + P_5 + P_7} - K \frac{P_1}{P1 + P_5}$$

 

[SW VandeCarr]

you are right, I have not explain it precisely.
What I mean is this:
If I want to have both P(A|B) and P(A|C) on the right hand side of inequality and I want to find a coefficient for P(A|BC) such that the obtained expression be the tightest upper bound for the left side expression, what is it?
in other words,
$$P(A|B \cup C) \leq P(A|B)+P(A|C)-?P(A|B \cap C)$$

If B and C are disjoint sets then $$P(A|B\cup C)=P(A|B)+P(A|C)- (K = P(A|B\cap C))$$ where K=0.

The maximum value of K will be the maximal intersection of B and C. K is not a coefficient. It is simply the value of the last term on the right side of the equation. Note the maximum value of the left side is when K=0.

 

[Stephen Tashi]

$$P(A|B\cup C)=P(A|B)+P(A|C)- (K = P(A|B\cap C))$$ where K=0.

Are we using the same notation? For $P(X|Y)$ meaning "the probability of X given Y" we have:
$$P(B \cup C | A) = P(B|A) + P(C|A) - P(B \cap C | A)$$
but not
$$P(A| B\cup C) = P(A|B) + P(A|C) - P(A| B \cap C)$$

 

[SW VandeCarr]

Are we using the same notation? For $P(X|Y)$ meaning "the probability of X given Y" we have:
$$P(B \cup C | A) = P(B|A) + P(C|A) - P(B \cap C | A)$$
but not
$$P(A| B\cup C) = P(A|B) + P(A|C) - P(A| B \cap C)$$

I'm taking the OPs question at face value. As posed, this is question is about the summation of (conditional) probabilities, not a calculation of posterior probabilities.

It seems clear that if B and C are disjoint sets then:
$$P(A|B\cup C)= P(A|B)+P(A|C).$$

This would be the maximum value of the left side of the equation which I understand to be the OP's question. It's not clear to me why you replaced the OP's notation with the likelihoods.

 

[Stephen Tashi]

this is question is about the summation of (conditional) probabilities

It's not clear to me why you replaced the OP's notation with the likelihoods.

In terms of conditional probabilities, an example of what I'm talking about is:

$P(x)= 1/8$ for $x = 1,2,..,8$

$$B = \{ 1,2\}$$
$$C = \{ 4,5\}$$
$$A = \{2,3,4,6,7,8 \}$$

$$P(A|B \cup C) = \frac{ P(\{2,4\} }{P(\{1,2,4,5\}} = (2/8)/(4/8) = 1/2$$
$$P(A|B) = \frac{ P(\{2\})}{P(\{1,2\}} = (1/8)/(2/8) = 1/2$$
$$P(A|C) = \frac{ P(\{4\}}{P(\{4,5\})}= (1/8)/(2/8) = 1/2$$

sabbagh80 can say if he means something different.

I don't know what distinction you are making between probabilities and liklihoods.

 

[SW VandeCarr]

Are we using the same notation? For $P(X|Y)$ meaning "the probability of X given Y" we have:
$$P(B \cup C | A) = P(B|A) + P(C|A) - P(B \cap C | A)$$
but not
$$P(A| B\cup C) = P(A|B) + P(A|C) - P(A| B \cap C)$$

I don't know what distinction you are making between probabilities and liklihoods.

The top line of terms above are the likelihoods of the corresponding terms of the bottom line.

 

[sabbagh80]

I'm satisfied with both comments #5 and #9 (by "Stephen Tashi").
I also thank "SW VandeCarr" for his/her useful discussions.