Why it doesn't sum to one in this simple naive Bayes classification?

Click For Summary
SUMMARY

This discussion addresses the issue of why the sum of probabilities P(Other|Long, Sweet, Yellow) + P(Banana|Long, Sweet, Yellow) does not equal 1 in a naive Bayes classification scenario with three classes: Orange, Banana, and Other. The specific probabilities are calculated as P(Banana|Long, Sweet, Yellow) = 0.25 / 0.27 and P(Other|Long, Sweet, Yellow) = 0.01 / 0.27, resulting in a total of 0.26 / 0.27, which is less than 1. The confusion arises from the interpretation of the features and the assumption that P(Orange|Long, Sweet, Yellow) = 0 is a rounding error rather than a definitive conclusion.

PREREQUISITES
  • Understanding of naive Bayes classification
  • Familiarity with probability theory
  • Knowledge of feature representation in machine learning
  • Experience with Python libraries for data analysis (e.g., NumPy, Pandas)
NEXT STEPS
  • Study the mathematical foundations of naive Bayes classifiers
  • Learn about feature independence assumptions in naive Bayes
  • Explore rounding errors and their impact on probability calculations
  • Investigate alternative classification algorithms and their advantages
USEFUL FOR

Data scientists, machine learning practitioners, and students studying classification algorithms who seek to deepen their understanding of naive Bayes and probability theory.

Karagoz
Messages
51
Reaction score
5
Summary:: When we have only three classes (Orange, Banana and Other) and three features (Long, Sweet and
Yellow), why P(Other|Long, Sweet, Yellow) + P(Banana|Long, Sweet, Yellow) is not equal to 1 when P(Orange|Long, Sweet, Yellow) = 0 ?

In this example:
https://towardsdatascience.com/all-about-naive-bayes-8e13cef044cf

There's an example of data on fruits with different features, and how do predict probability of what class fruit is it given some features. There are similar guides online using similar examples.

There are only 3 classes of fruits. Banana, Orange and Other.
And we have only 3 features; long, sweet and yellow.

P(Orange|Long,Sweet,Yellow) = 0
The probability given fruit is Orange are zero because the Probability of Orange when given fruit is long are zero.

P(Banana|Long, Sweet, Yellow) = 0.25 / 0.27

P(Other|Long, Sweet, Yellow) = 0.01 / 0.27

But: P(Other|Long, Sweet, Yellow) + P(Banana|Long, Sweet, Yellow) = 0.25/0.27 + 0.01/0.27 = 0.26/0.27 < 1

If the features are given as "long, sweet and yellow" it's impossible to be an orange. It must be either banana or other when features are "long, sweet and yellow".
If the features are given as "long, swet and yellow", then it must be either a banana or "other".

But why the P(Other|Long, Sweet, Yellow) + P(Banana|Long, Sweet, Yellow) is not equal to 1?
Shouldn't it be equal to 1? Also P(Other|Long, Sweet, Yellow) + P(Banana|Long, Sweet, Yellow) = 1 ?

[Moderator's note: moved from a technical forum.]
 
Physics news on Phys.org
Karagoz said:
Summary:: When we have only three classes (Orange, Banana and Other) and three features (Long, Sweet and
Yellow), why P(Other|Long, Sweet, Yellow) + P(Banana|Long, Sweet, Yellow) is not equal to 1 when P(Orange|Long, Sweet, Yellow) = 0 ?

In this example:
https://towardsdatascience.com/all-about-naive-bayes-8e13cef044cf
First, do not use undefined commas in a probability like Long, Sweet, Yellow. Do you mean "and" or "or"?
Second, P(Orange|Long or Sweet or Yellow) does not = 0
 
  • Like
Likes   Reactions: Karagoz
That's just a rounding error.
 
  • Like
Likes   Reactions: Karagoz

Similar threads

  • · Replies 9 ·
Replies
9
Views
2K
  • · Replies 19 ·
Replies
19
Views
2K
  • · Replies 47 ·
2
Replies
47
Views
5K
Replies
4
Views
14K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 13 ·
Replies
13
Views
2K
  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 4 ·
Replies
4
Views
4K
  • · Replies 80 ·
3
Replies
80
Views
8K