How to verify dice by statistic means ?

[NdotA]

Hello all,

I got stuck with some problem of probability

What I understand is how to verify by statistics, that a die (is that really the singular of dice ??) is crooked:
- cast it a number of times, say twenty
- count the number that shows most, say we find eight times the six
- compute probability of this result for an unforged die with 1/6 propability for each number, if I entered my figures properly, then p (k = 8) = 0.0084
so we have a signifcance of about 1 %, that means probability for the die to be forged is 99 %.

What I could not figure out is how to proceed to verify that the die is straight. Something like how often must the number show at least to prove it. But in all my musings I come back to the binomial probability function and come out with the same figures I used to verify it is forged.

Any ideas ?
Thanks in advance.
N.A

 

[Bacle2]

You can never actually verify the dice is crooked, but you can have a level of confidence.
For one, it is not impossible for any given n-ple to occur, even if your die is fair.
One test you can do is the Chi-squared.

 

[NdotA]

You can never actually verify the dice is crooked

Yes I know that. Maybe 'verify' is not the proper word (not being a native speaker of English). By my evaluation of a level of probability for the forged one I established a level of confidence, (or suspicion in that case :-)) I think.

I just looked up chi-square test in wikipedia. I am not so sure that I would be happy with this for absence of a mean value and variance.

But thanks anyway for your fast response.

N.A

 

[Ibix]

You can approach this via hypothesis testing. Start with a null hypothesis: the die is fair. You now have a probability distribution on the number of sixes - ~B(N,1/6), for N throws. Make N throws and record the number of sixes. How probable is it, given the binomial distribution, that you would get that many or more sixes? That's your confidence that the die is fair (in the language of hypothesis testing, you "accept/reject the null hypothesis with confidence X").

I've been a bit sloppy above. If you know nothing about the die, you should use a two-tailed confidence interval. That is, to establish a 95% confidence that the die is fair, the results should lie between the 2.5th percentile and the 97.5th percentile of the binomial distribution. If you have reason to suspect that the die is loaded in favour of sixes (they're a winning throw in some game, so biasing a die against winning is unlikely) then you can use a one-tailed confidence interval: the result should be below the 95th percentile of the distribution (or above the 5th percentile if biasing against sixes is a winning strategy).

By the way, you are throwing away quite a lot of information by using a binomial distribution. You can record the number of times each face comes up and use a multinomial distribution instead. The maths is significantly more complex, so for study purposes it's probably best to stick with the binomial. But if you actually want to do something like this in reality, the binomial method will take a lot longer than the multinomial method to achieve the same level of confidence.

 

[Stephen Tashi]

so we have a signifcance of about 1 %, that means probability for the die to be forged is 99 %.

No, you can't say the probability that the die is unfair ("forged") is about 99%. All the probabilities you have computed are based on the assumption that the die is fair. (You didn't allow for any probability of it being unfair. You assumed that it is fair with probability 100%.)

If the probability of the data is small given that assumption, you may subjectively doubt that the die is fair, but you cannot deduce any probability that the die is unfair. It's the difference between "The probability of A given B" and "The probability of B given A". The "Probability of the observed data given the die is fair" is not necessarily the same quantity as "The probability the die is fair given the observed data.".

If you want to compute the probability that the die is unfair given the data, you must take a Bayesian approach and quantify the probability the die is unfair before the data is considered.

Your approach is called "Hypothesis testing" and this approach has a procedure for "rejecting" the hypothesis that the die is fair. However this procedure is just a procedure, it is not a proof of anything. It doesn't quantify the probability that rejecting or not-rejecting the hypothesis is the correct thing to do.

 

[NdotA]

Thanks Ibix and Stephen.
So I take it the thing is more demanding than I thought. My problem is, I never came into contact with statistics during my studies and professional life and I am just trying to understand things. On research reports I often read that the result is 'significant' or it is not and I want to get an understanding of the background of it, to get a grab on numbers involved. That is why I came up with the (hopefully) easy to comprehend and easy to compute example with the dice, which is of course no real task.

If you know nothing about the die, you should use a two-tailed confidence interval.

This is what I want to do, and succeeded in doing half the job, at least to my understanding of the matter. I had my hypothesis 'the die is fair' tested by evaluating the probability of the result of my test throws. I found that probability for 8 sixes out of 20 throws to be roughly 0.8 % (I know I should add the probabilities of 9, 10, 11, ... sixes, but for the sake of simplicity I skipped this for the rapidly decreasing magnitude of the figures).

Its the other end I have problems with. For no sixes at all in 20 throw I find the probability to equal 2.6 % for a fair dice. To reach a 99 % confidence level, that the die is fair, I would have to increase the numbers of test throws, say 30. For thirty throws the probability is about 0.4 % that a fair dice does not show any six. So if I do not find any six in thirty throws, I have a confidence level of 99.6 % that the die is fair.

Is this proper thinking ?

If yes, then I would have reached my goal, which would be to have an understanding of the numbers involved to define 'surely unfair', 'surely fair' and how big the grey zone in between is and I can fool around a bit with the figures to see e.g. the influence 0f 99 % or 95 % confidence level.

And it beats me how to distinguish between 'a little unfair' and 'very unfair'.

N.A

 

[Bacle2]

Basically, you accept/reject depending on the confidence level you set for your experiment. If your confidence level is, say, 99% (say, 2-tailed) , this means you
will accept all outcomes except those whose probability of occurring (given your assumed model/distribution) is less than 100%-99% =1%. Under a choice of 99%, a result/outcome with probability less than 1% is described as being significant at the 99% level, and not significant at the same level otherwise.

I was actually thinking of using the normal approximation to the binomial, which I think is pretty good for N=30, I think. Then your confidence interval would be centered at
N*(1/6) ( so, of course, choose N to be a multiple of 6 ), and the standard deviation is
1/6*5/6

 

[NdotA]

Yes, clearly, that is the case in statistics lingo.

But I understand the two tailed confidence level in a way, that it indicates that the die is not crooked in favor of the sixes - it would be if the number of sixes exceeds the 95 or 99 % probability, and it is not crooked against the sixes if the number of sixes is not less than the minimum number corresponding to 5 or 1 % probability.

But not having a confidence level that the die is unfair does not prove it to be fair.

So somehow I am missing something. I would need it in layman's terms how to establish a confidence level in favor of the die being fair using the results of my testthrows. And for simplicity, keep it on the sixes. Imagine somebody claimed your die being unfair for its unusual number of sixes and you want to convince your opponent, that this is not true.

Could I clarify my problem ?

N.A

 

[Stephen Tashi]

So somehow I am missing something. I would need it in layman's terms how to establish a confidence level in favor of the die being fair using the results of my testthrows

The term "confidence" has a technical meaning in statistics and that meaning is not what a layman means by "confidence". The technical meaning of "confidence" is not synonmyous with "probability". As I said before, the type of statistics you are using cannot quantify the the probability that the die is fair. It also cannot quantify the probability that the die is unfair. All you can do is quantify the probability of various aspects of the data given the assumption that the die is fair.

If you need a layman's explanation of how to find a "confidence level", you must say what the layman means by "confidence level".

Perhaps you are confusing hypothesis testing http://en.wikipedia.org/wiki/Statistical_hypothesis_testing ( which uses "acceptance intervals" or "acceptance regions") with estimation, which uses "confidence intervals" http://en.wikipedia.org/wiki/Confidence_interval.

 

[NdotA]

Thanks for the links, I will come back when I think I grabbed what is in there.

But how to procede to convince my opponent that my die is not unfairly preferring sixes ?

(or is this a stupid question ?)

N.A

 

[Bacle2]

Maybe you can turn this into a binomial probability problem : six or non-six ,calculate the expected number of sixes in N throws (which is N*(1/6) ) and test against the actual number, by constructing the confidence interval.

For N=30, I think there is a pretty good normal approximation to the binomial. Then you can test your hypothesis that the mean is N*1/6 , using a standard deviation of 1/6*5/6.

 

[Dotini]

...out of 20 throws...

Is this proper thinking ?

In my youth, I played in over 400 backgammon tournaments, a few at the international level, read copiously, and thought I learned something about dice. In my humble opinion, 20 is way too few throws for a realistic sample size. One hundred, even one thousand is called for. Oh, and I'm retired from Boeing where I did statistical process control, and my fond memories of that also tell me 20 is too few a sample size.

Apropos of nothing, I recall one evening in the '70's at Las Vegas, where I was playing in the US Open Backgammon Championship. I watched a few games of the fabled Oswald Jacoby, who doing money games on the side at $100/pt, and he rolled double 5's four times in a row to win a pile of hundred dollar bills so thick it was holding up one edge of the board!

Respectfully submitted,
Steve

 

[Bacle2]

Maybe this will be helpful to NdotT

http://en.wikipedia.org/wiki/Binomial_test

 

[NdotA]

Hmmm,
thank you all for your contribution - but here I am, just as clever as before.
I completely understand - at least I think I do - the approach to check if the die is unfair. It is something like this:
- assume the die is fair (null hypothesis)
- establish the probability that such a die produces the result of my testthrows
- if this is out of a predfined range, say 5 to 95 % the result is significant that the die is not fair.

Accepted and understood - in the wording of a mechanical angineer, that never came nearer to statistics than doing regressions and standard deviations and six sigma with help of his computer.

So, but how to set about for the opposite problem ?
How to set about to have a test on the die to convince my oponent (I do not say to prove) that the die is fair? Or is the answer somewhere in the responses here and I just fail to recognise it ?
What would be the null hypothesis ?
What kind of test to apply ?
[... scratching head and feeling sort of stupid ...]
N.A

 

[Stephen Tashi]

So, but how to set about for the opposite problem ?
How to set about to have a test on the die to convince my oponent (I do not say to prove) that the die is fair?

If you don't reject the null hypothesis then you accept that the die is fair.

However, how to convince someone using statistics is sociological problem, not a mathematical problem. If the person you are trying to convince is impressed or intimidated by a "standard" statistical procedure then some hypothesis testing procedure will be adequate. If they aren't convinced, there is no mathematicl proof that a particular type of hypothesis test reaches the correct answer. You can see this same phenomenon at work if you look at academic journals. In different fields of research, different editors have different opinions about what kind of statistical procedures constitute evidence.

 

[chiro]

The posters above have mentioned some of the tests including a Chi-Square Goodness Of Fit test as well as using the Binomial and Multinomial distributions and their estimators to form a hypothesis test.

For the first one, look up the Chi-Square Goodness Of Fit test in a statistics resource or uses a computer software package to generate a probability value that you can compare to with some statistical significance.

 

[NdotA]

If you don't reject the null hypothesis then you accept that the die is fair.

Oh, I never thought of it this way. I thought I understood not having enough significance to reject the null hypothesis, does not mean it is correct. How would I set about a null hypothesis 'the die is unfair' and test it ?

Something like in court: If there is not enough proof that the defendant is guilty is no proof that he did not commit the crime.

how to convince someone using statistics is sociological problem

I am aware of that - but I am having trouble to convey what I want to express. What I was wondering about is how to set the test or work on the data to get a notion if the die is working properly. I have no idea, if 'evidence', 'proof', 'hint', 'significance' or whatever is an unambiguous word for the fact that I can do some maths on my data and the result shows or points in the way or implies or indicates or signifies or whatever is the proper term that the die is fair.

Now I will be looking into that Chi-Square Goodness Of Fit test...

N.A

 

[chiro]

You can't prove either way that the result is certain.

In hypothesis tesing you get two kinds of errors that relate to false positives (the result is false, but we have come to the conclusion that it's true) and false negatives (we come to the conclusion something is false but the reality is that it's positive) where positive means true and negative means false.

If we don't get a false positive or false negative then we have made the correct hypothesis.

There will always be the chance of a false positive and false negative and it's important to realize how significant these are in the context of statistical tests and hypothesis testing.

 

[Stephen Tashi]

How would I set about a null hypothesis 'the die is unfair' and test it ?

To test a hypothesis, the hypothesis must be specific enough to compute the probability of the data. The statement "The die is unfair" isn't specific enough to compute the probability of the data. A die can be unfair in many ways.

If you assume the die is unfair in a specific way (e.g. Pr(1) = 0.10, Pr(2) = 0.15, Pr(3) = 0.10, Pr(4) = 0.15, Pr(5) = .26, Pr(6)= .24 ) then you can test that hypothesis.

If you assume a specific "prior probability distribution" on 6 random variables that represent the probabilities of the outcomes of the die then you can compute the 'posterior probability distribution" for those random variables given the observed data. That would be the Bayesian approach. I think E.T. Jaynes actually worked this problem in one of his writings, perhaps it was in "Probability The Logic Of Science".

The use of the null hypothesis that "the die is fair" can be subjectively justified in many ways. "Innocent until proven guilty" is one way. "No reason to favor one face over others without considering data" is another. Sometimes the null hypothesis is touted as the "skeptical" hypothesis. Considering the imperfections in real dice, I would say the the hypothesis that "the die is fair" is a gullible hypothesis. However, most people are thinking in terms of "the die is nearly fair" without specifyiing quantitatively what they mean by that.

 

[NdotA]

Thanks Stephen,

now I see a little more clearly.

But I came to the conclusion, that I will continue on this path only when forced to do so.
I just wanted to get a basic understanding of the concept (I guess I succeeded to a certain extent thanks to the posters here) but to go more into detail will be too time consuming for my situation.

Thanks all

N.A