Type 1 and type 2 errors?

  • Thread starter moonman239
  • Start date
  • #1
282
0
Let's say I'm testing anti-spam software. The number of false positives (aka, friendly messages misidentified as spam, for those who don't know the term) is 40. The number of false negatives (spam messages misidentified as friendly) is also 40. I'm testing 100 messages. How many more messages would I need to test in order to be 99.99% that the null hypothesis can/cannot be rejected?
 
Last edited:

Answers and Replies

  • #3
282
0
Thanks for the link, but I'm not seeing any equation whatsoever that will help. As far as I can see, all the listed equations have to do with means.
 
  • #4
EnumaElish
Science Advisor
Homework Helper
2,304
124
Oh, so what's your null hypothesis? I thought you were to test that the average message is not spam.
 
Last edited:
  • #5
282
0
I just don't see how a sample mean would be relevant. Your understanding of the question is correct.
 
  • #6
D H
Staff Emeritus
Science Advisor
Insights Author
15,393
684
Be very specific, please: What is your null hypothesis?

Also, you have told us about 80 messages, what about the other 20?
 
  • #7
282
0
Be very specific, please: What is your null hypothesis?

Also, you have told us about 80 messages, what about the other 20?
Let's say my null hypothesis is that a spam message will be correctly marked as spam. As for the 20 messages, lets say that those are false negatives.
 
  • #8
D H
Staff Emeritus
Science Advisor
Insights Author
15,393
684
You already said you had 40 false positives and 40 false negatives out of 100 tests. That makes for a total of 80 out of 100. Those remaining 20 are either true positives or true negatives.

What does the confusion matrix for your test results look like?
 

Related Threads on Type 1 and type 2 errors?

Replies
9
Views
175
  • Last Post
Replies
4
Views
3K
  • Last Post
Replies
3
Views
265
  • Last Post
Replies
4
Views
1K
  • Last Post
Replies
1
Views
1K
  • Last Post
Replies
3
Views
341
  • Last Post
Replies
0
Views
1K
  • Last Post
Replies
0
Views
929
  • Last Post
Replies
0
Views
1K
  • Last Post
Replies
11
Views
4K
Top