Probability of Bias?

[bsharvy]

TL;DR Summary

Can you find the probability that a coin or die is loaded?

I was thinking that the probability of a set of events not happening is the same as the probability of that the die/coin is biased.

So, if I flip a coin 10 times and get heads every time, the probability the coin is biased is 1- (.5)^7.

Roll a die 5 times, get "4" all times, probability of bias = 1 - (1/6)^5

But that suggests the probability of bias after one coin toss is 50%, which can't be right. I'm also not sure how to calculate when the results are mixed, such as flipping a coin 10 times and getting heads 7 times.

Help!

 

[FactChecker]

Your question has a subtle, but important, mistake. Whether a coin is biased is not a random variable with a probability. So you should not talk about its probability. The only probability that you can talk about is that of getting a certain result from a fair coin. If that probability is too low, you are justified in rejecting the assumption that the coin is fair. This is called "rejecting the null hypothesis". There are certain standard levels of small probability, called "significance levels", that allow you to reject the null hypothesis. They include 0.05, 0.025, and 0.01. These are the probabilities of rejecting the null hypothesis even though it is true, so smaller is better. For something like the discovery of a new elementary particle in science, the significance level required is extremely demanding. It is "5 sigma", which is a probability of $3\times 10^{-7}$ or about one in 3.5 million.

 

[PeroK]

TL;DR Summary: Can you find the probability that a coin or die is loaded?

I was thinking that the probability of a set of events not happening is the same as the probability of that the die/coin is biased.

So, if I flip a coin 10 times and get heads every time, the probability the coin is biased is 1- (.5)^7.

Roll a die 5 times, get "4" all times, probability of bias = 1 - (1/6)^5

But that suggests the probability of bias after one coin toss is 50%, which can't be right. I'm also not sure how to calculate when the results are mixed, such as flipping a coin 10 times and getting heads 7 times.

Help!

You cannot compute the probability that a coin is biased without a prior probability of the coin being biased. Consider the two extreme cases:

1) There are no biased coins.

2) All coins are biased.

To take a more realistic example: assume that one coin in a hundred is biased (and to keep things simple comes up heades every times). Out of approximately every 1100 coins we have:

1000 fair coins that give ten heads in a row approximately once;
11 biased coins that come up heads every time.

If we get 10 heads in a row, then the probability is about 11/12 that the coin is biased.

If we assume that one coin in a thousand or one coin in a million is biased, we will get very different answers.

 

[Dale]

Whether a coin is biased is not a random variable with a probability. So you should not talk about its probability.

It certainly can be considered a random variable with a probability distribution in Bayesian statistics. But in frequentist statistics it would not be a random variable.

You cannot compute the probability that a coin is biased without a prior probability of the coin being biased.

Yes, I agree, the prior probability is needed.

@bsharvy basically this is an exercise in Bayesian statistics. I have a set of Insights articles on this topic. In particular, the third one may be useful for you since I specifically examine the fair coin question in some depth:

https://www.physicsforums.com/insights/how-bayesian-inference-works-in-the-context-of-science/