I How Does the Inclusion-Exclusion Principle Relate to Probability Theory?

[Aslet]

Hello everyone!
I'm studying the physics of complex systems and I'm approaching probability theory.
I understand that we need a $\sigma-algebra$ and the Kolmogorov axioms in order to define a probability space but then I bumped into the following relation:
$$ p(A_1 \cup A_2 ) = p( A_1 ) + p( A_2 ) - p( A_1 \cap A_2 ) $$
where $A_1, A_2$ are two sets of events that satisfy Kolmogorov axioms. I used all the properties i know to understand it but I wasn't able to demonstrate it. The book says that this equation comes from Kolmogorov axioms...
Can you help me?

 

[S.G. Janssens]

Can you show that if $A, B$ are events with $B \subseteq A$, then $P(A \setminus B) = P(A) - P(B)$?
Using this, now write $A_1 \cup A_2 = A_1 \cup A_2 \setminus (A_1 \cap A_2)$. The right-hand side is a disjoint union and $A_1 \cap A_2 \subseteq A_2$. Now you are almost there.

(In these cases, it helps to draw Venn diagrams.)

 

[Aslet]

I understand that if $A_2 \subseteq A_1$, then the equation it's easy to understand, more or less. Unfortunately the book doesn't give me this hypothesis.
Is it possible that the authors forgot about it or is this hypothesis not necessary?

 

[S.G. Janssens]

No, $A_1$ and $A_2$ are arbitrary events. It is not necessary to assume that $A_2 \subseteq A_1$, this is not what I meant.

Do you agree with
$$
A_1 \cup A_2 = A_1 \cup A_2 \setminus (A_1 \cap A_2),
$$
and that this is a disjoint union? If so, then by Kolmogorov's axioms,
$$
P(A_1 \cup A_2) = P(A_1) + P(A_2 \setminus (A_1 \cap A_2)),
$$
right? Now look at the set inside the second probability on the right. We have $(A_1 \cap A_2) \subseteq A_2$ so, by what I wrote on the first line of my initial reply,
$$
P(A_2 \setminus (A_1 \cap A_2)) = P(A_2) - P(A_1 \cap A_2),
$$
which is what you need.

 

[Aslet]

Thank you very much!
I only need the hypothesis in writing the difference, and obviously $A_1 \cap A_2 \subseteq A_2$. I also get that $A_1 \cap [ A_2 \setminus ( A_1 \cap A_2 ) ] = \emptyset$.
I still don't get the first relation that you wrote. :(

 

[S.G. Janssens]

I still don't get the first relation that you wrote. :(

As you have seen, in these cases it is often useful to write a set as a disjoint union of other sets. Here the same happens: For $A,B$ events with $B \subseteq A$,
$$
A = (A \setminus B) \cup B,
$$
and this is a disjoint union, so by Kolmogorov,
$$
P(A) = P(A \setminus B) + P(B)
$$
and therefore,
$$
P(A \setminus B) = P(A) - P(B).
$$
Another exercise in this spirit is to show that any countable union $A_1 \cup A_2 \cup \ldots$ of events can be written as a countable disjoint union.

Another often used technique is splitting out an event: You want to calculate $P(A)$ for some event $A$ and you have a partition $B_1,B_2,\ldots$ of the underlying sample space. Then write $A = (A \cap B_1) \cup (A \cap B_2) \cup \ldots$ as a disjoint union. This comes in handy when it is somehow easier to calculate $P(A \cap B_n)$ for all $n \in \mathbb{N}$.

 

[Aslet]

Sorry, I was ambiguous! I meant that I don't understand
$$ A_1 \cup A_2 = A_1 \cup [ A_2 \setminus ( A_1 \cap A_2 ) ] $$
It seems like $A_2 = A_2 \setminus ( A_1 \cap A_2 )$. Thank you again. :)

 

[S.G. Janssens]

Sorry, I was ambiguous! I meant that I don't understand
$$ A_1 \cup A_2 = A_1 \cup [ A_2 \setminus ( A_1 \cap A_2 ) ] $$
It seems like $A_2 = A_2 \setminus ( A_1 \cap A_2 )$. Thank you again. :)

By itself, the second equality is not true. However, the first equality is true.

To see that the first one is true, try to prove two inclusions: Prove that if $x \in A_1 \cup A_2$, then $x \in A_1 \cup [ A_2 \setminus ( A_1 \cap A_2 )]$ and, conversely, if $x \in A_1 \cup [ A_2 \setminus ( A_1 \cap A_2 )]$ then $x \in A_1 \cup A_2$.

If you make a drawing (draw the sets as two intersecting circles, this is called a Venn diagram), then you can see it more clearly.

 

[Aslet]

Ok, I made a drawing and I realized my mistake.
I will also try to demonstrate the two inclusions, thank you for the big help!