Rolling a Fair Die: Analyzing Data & Results

[DaveC426913]

I don't know how to analyze data proper-like.

What variance would be considered statistically significant, such that one might conclude a die is not fair?

I just rolled a six-sided die 192 times.

There are my results: 32, 28, 35, 37, 29, 31.
So, they vary from average by 0, -4, +3, +5, -3, -1.

Should I just keep rolling?

 

[MarneMath]

Are the numbers the number of occurrences? For example is 32 the number of times 1 came up and 31 the number of times 6 came up? One common way to do this is to do a hypothesis test and use the pearson chi-square test. I can say with reasonably high confidence that the dice is likely fair. I got a p-value of .96ish.

 

[DaveC426913]

Are the numbers the number of occurrences? For example is 32 the number of times 1 came up and 31 the number of times 6 came up?

Yes. No.

I mean yes the number is the number occurrences. But no, it is not 1 through 6. (though that is not relevant to the fairness)

One common way to do this is to do a hypothesis test and use the pearson chi-square test. I can say with reasonably high confidence that the dice is likely fair. I got a p-value of .96ish.

I ... guess I'll be Googling 'pearson chi-square test'.

Or, I suppose, I can just Google 'how to determine of a die is fair'...

 

[jedishrfu]

I think the numberphile channel on YouTube had a video on dice fairness or on creating a special dice with unique properties.

 

[FactChecker]

The Chi-squared goodness of fit test that @MarneMath recommends is standard and routinely used for your type of question. There are several calculators on the internet. Here is one https://graphpad.com/quickcalcs/chisquared2/

For your data, it gave a two-tailed P value of 0.8662, which means that the odds of getting sample results from a fair die that differ from expected (32 per side) by that amount or more are 0.8662. So your results are very reasonable for a fair die.

Whether you should keep rolling the die is up to you. You can assume that your die is not perfectly fair since no die is perfect. But you may have to roll a huge number of times to detect it with any statistical certainty.

 

[Dale]

You could also use a Bayesian approach. You would probably model it with a Dirichlet distribution prior and posterior. Then you could either decide how far from fair you are willing to ignore or you could compare directly to the fair dice hypothesis.

 

[DaveC426913]

Too bad it'll be something I can only revel in alone.

My gamer friends are all
"Hey that die has a nine."
"Hey that die has two threes."
"Hey that die has a zero."

stupid mathphobes...

 

[sysprog]

If it's a pitted die, as depicted, it is imperfect with respect to fairness by virtue of non-uniform density.

These flush-faced casino dice are closer to fair:

​

 

[DaveC426913]

If it's a pitted die, as depicted, it is imperfect with respect to fairness by virtue of non-uniform density.​

Indeed. And worse than normal dice because of the large discrepancy in number of holes.
Also, I did not distribute the numbers around the faces. You'll notice that 7 and 9 are adjacent.
My plan is to fill the pits with epoxy, then retest to see if that fairs them up.(Also: Dude wth! My post is all centre-jusitifed cause-uh you! )​

 

[FactChecker]

I would be surprised if the difference shows up before millions of tosses. I think that the tiny difference would be lost in other random influences.

 

[DaveC426913]

I would be surprised if the difference shows up before millions of tosses. I think that the tiny difference would be lost in other random influences.

Well, that's why we do empirical observations. To surprise us (or more accurately, to disabuse us of our preconceptions).

 

[FactChecker]

Well, that's why we do empirical observations. To surprise us (or more accurately, to disabuse us of our preconceptions).

I agree. It would be interesting to me if a reasonable number of trials could detect that they are not fair.

 

[FactChecker]

Just for fun, I tested what the Chi-squared result would be if all your results were multiplied by 10 as though you had done 10 times more experiments and got those proportions. Those results would be statistically very significant. If the die were fair, results that "unfair" or worse would only happen 2 in every 1000 times.

With 5 times as many trials giving the same proportions, the odds are 1 in 10 of getting those results or worse from a fair die.

 

[DaveC426913]

Just for fun, I tested what the Chi-squared result would be if all your results were multiplied by 10 as though you had done 10 times more experiments and got those proportions.

Isn't that kind of counter-productive?

I mean, if I rolled a perfectly fair only 6 times in total, then multiplied the result by 333, I'd obviously get vastly biased results.

 

[FactChecker]

Isn't that kind of counter-productive?

I mean, if I rolled a perfectly fair only 6 times in total, then multiplied the result by 333, I'd obviously get vastly biased results.

I wasn't interested in a perfectly fair die. Just to get some scope on the problem, I was trying to see of how many rolls it would take if the biased results were due to an unfair die and were continued in a larger experiment. Five times as many rolls would not be enough and 10 times would be more than enough.

 

[MarneMath]

I wasn't sure if just multiplying the number by 10 would be a good way to calculate how my reps the OP would need to check if the results would be significant. So I ran a quick power calculation. With an effect size of w = .1, sig level of .05 and power at .95. The OP would need 1979 rolls. The practical problem here is obviously what you consider to be practically significantly different. I can always find something to be statistical significant if I just increase my sample size. By the nature of most statistical test, they become rather sensitive as our sample size increases.

 

[FactChecker]

I wasn't sure if just multiplying the number by 10 would be a good way to calculate how my reps the OP would need to check if the results would be significant. So I ran a quick power calculation. With an effect size of w = .1, sig level of .05 and power at .95. The OP would need 1979 rolls. The practical problem here is obviously what you consider to be practically significantly different. I can always find something to be statistical significant if I just increase my sample size. By the nature of most statistical test, they become rather sensitive as our sample size increases.

The original data is not uniform, but the difference between it and a uniform, fair die is very insignificant. If the lack of uniformity is entirely due to an unfair die, then it would continue the same proportions in a larger experiment. So multiplying the results and the expected uniform results would be give an idea of how an unfair die like that would perform in a Chi-squared test. Using expected bin sizes of 320 and data 320,280, 350, 370, 290, 310 gave these results:

P value and statistical significance: 
  Chi squared equals 18.750 with 5 degrees of freedom. 
  The two-tailed P value equals 0.0021 
  By conventional criteria, this difference is considered to be very statistically significant.

The P value answers this question: If the theory that generated the expected values were correct, what is the probability of observing such a large discrepancy (or larger) between observed and expected values? A small P value is evidence that the data are not sampled from the distribution you expected.

Multiplying by 5 instead of 10 gives the expected bin size of 160 and data 160, 140, 175, 185, 145, 155. The Chi-squared results are:

P value and statistical significance: 
  Chi squared equals 9.375 with 5 degrees of freedom. 
  The two-tailed P value equals 0.0950 
  By conventional criteria, this difference is considered to be not quite statistically significant.

The P value answers this question: If the theory that generated the expected values were correct, what is the probability of observing such a large discrepancy (or larger) between observed and expected values? A small P value is evidence that the data are not sampled from the distribution you expected.

 

[MarneMath]

Well, I understand what you're saying. I'm just not sure that's a method I would use to calculate the required number of reps needed. We essentially came up with same number of required reps though. At your particular sample size the effects your measuring is basically that a difference of .1. Perhaps in dice land that's a practically significant difference. I'm not sure. Either way, that's my only caution with regards to increasing sample size until you have a decent p-value.

*Disclosure I'm assuming a power of .9

 

[FactChecker]

Well, I understand what you're saying. I'm just not sure that's a method I would use to calculate the required number of reps needed.

You're right. It would only work for an unfair die that gives exactly the biased results of that data sample. It could not be used for an unknown die. It's just an example that I thought was interesting.

At your particular sample size the effects your measuring is basically that a difference of .1. Perhaps in dice land that's a practically significant difference.

That is the result for 5 times as many trials. It is not significant enough. The result for 10 times as many is shown above. It is very significant (0.002).

 

[MarneMath]

Maybe there's a misunderstanding when I say effects size. I'm referring to this https://en.wikipedia.org/wiki/Effect_size not the p-value.

 

[FactChecker]

Maybe there's a misunderstanding when I say effects size. I'm referring to this https://en.wikipedia.org/wiki/Effect_size not the p-value.

Oh. I see what you mean. Good point. I stand corrected. Yes, I think that level of "unfairness" would be considered significant in a die.

 

[Mark44]

Indeed. And worse than normal dice because of the large discrepancy in number of holes.
Also, I did not distribute the numbers around the faces. You'll notice that 7 and 9 are adjacent.
My plan is to fill the pits with epoxy, then retest to see if that fairs them up.(Also: Dude wth! My post is all centre-jusitifed cause-uh you! )​

It's because of pairs of

and​

tags. You can see them if you click Edit on the post, and then click the little paper-looking icon at the upper right side of the window. However, since the post I replied to is several days old, the time window for being able to edit your post has elapsed. I have tweaked the two relevant tags to keep them from rendering.

At any rate, this is what the text above looks like in the BBCode editor (which you get by clicking the paper thingy I mentioned):

Indeed. And worse than normal dice because of the large discrepancy in number of holes.
Also, I did not distribute the numbers around the faces. You'll notice that 7 and 9 are adjacent.
My plan is to fill the pits with epoxy, then retest to see if that fairs them up.(Also: Dude wth! My post is all centre-jusitifed cause-uh you! )​

 

[stevendaryl]

I looked up what should be expected for die rolls.

Roll a 6-sided die N times, and let f be the fraction of times that you roll a 1 (for a fair dice, it should be the same for 1-6).

Then:

\langle f \rangle = 1/6 (naturally)
\sigma_N(f) = \sigma_1(f)/\sqrt{N}

where \sigma_N is the standard deviation of f for N rolls.

As for \sigma_1, we can compute it this way:

\sigma_1 = \sqrt{var_1} where var_1 is the variance. By definition, the variance in f is:

var_1 = \langle f^2 \rangle - \langle f \rangle^2

For a single roll, f = 1 (with probability 1/6) and f=0 with probability 5/6. So \langle f^2 \rangle = 1/6 and \langle f \rangle = 1/6. So var_1 = 1/6 - (1/6)^2 = 5/36. So \sigma_1 = \frac{\sqrt{5}}{6}

So the standard deviation in f for N rolls should be \frac{\sqrt{5/N}}{6}

In your case, N = 192, so this gives \sigma_N = 0.027.

So you can expect that typically 1/6 - \sigma_N < f < 1/6 + \sigma_N. So 0.140 < f < 0.194

Your particular frequencies were:

• 32/192 = 0.167
• 28/192 = 0.146
• 35/192 = 0.182
• 37/192 = 0.193
  
• 29/192 = 0.151
• 31/192 = 0.161

So all your frequencies are within 1 standard deviation of 1/6 (although that 37/192 is getting close to the limit)