Understand Beta-Binomial Model for Win/Loss Rating Systems

  • Thread starter ParoXsitiC
  • Start date
  • Tags
    Model
In summary, the beta-binomial model is a method for estimating players' success rates in a rating system. It combines prior information from a beta distribution with data on wins and losses to calculate a posterior estimate of the success rate. This method is often referred to as empirical Bayes and the parameters used to specify the beta distribution are estimated using frequentist methods. However, there may be some ambiguity in the calculation of the 2nd sample moment and further clarification is needed.
  • #1
ParoXsitiC
58
0
I asked a question here: http://math.stackexchange.com/questions/183483/rating-system-incorporating-experience and I'd like to understand how I do the beta-binomial model, but he is taking a bit long to answer so I am hoping to get an answer.


His answer is as follows:


One option is to use something like the [beta-binomial model][1].

The general idea is that players' true *success rates* (or winning frequency) come from an underlying distribution (e.g. a beta distribution). As a player plays more games and you get actual information on wins and losses, the prior information from the beta distribution is combined with the wins/losses information which is expected to follow a binomial distribution (based on the true success rate $p$), and a posterior estimate of the success rate made as
$$\hat{p}=\frac{n_{\text{wins}}+\nu\rho}{n_{\text{games}}+\nu}$$
where the beta distribution essentially has the effect of a prior information equivalent to $\nu$ games with $\rho$ success rate.

The advantage of this method is that for playes with few games, the estimated success rate is *shrinked* towards the population mean; extreme success rates due to highly uncertain success rate estimates for playes with few games are avoided.

Due to the similarity with Bayesian methods, this type of approach is often referred to as empirical Bayes. However, the parameters $\nu$ and $\rho$ used to specify the beta distribution are estimated using traditional frequentist methods (moment or maximum likelihood estimates).

------

I looked the wikipedia on it here: http://en.wikipedia.org/wiki/Beta_binomial

and found an example calculating the alpha and beta using sample methods - Given this data set:

Males 0 1 2 3 4 5 6 7 8 9 10 11 12

They came up with the 1st sample moment as : 6.23
2nd sample moment as 42.31

I searched for a while how this calculation was actually done, and finally came to this wikipedia page: http://en.wikipedia.org/wiki/Beta_distribution#Parameter_estimation

Which says the 1st sample moment and 2nd sample moment are just the sample mean and sample variance,

which I found to be 6 and 15.166667 - different than their 6.23 and 42.31.

-------------------------

I just want find a good rating system that involves win and losses and need a laymen explanation on how it's done. I also asked the question here: http://www.rugatu.com/questions/2845/understanding-statistics-beta-binomial-model but he also failed to follow up and make sure I did it right :\
 
Physics news on Phys.org
  • #2
ParoXsitiC said:
Which says the 1st sample moment and 2nd sample moment are just the sample mean and sample variance,

That article says the variance is the second moment centered about the the mean, so perhaps the [itex] m_2 [/itex] in the other article is the second moment centered about 0 , which is just the mean of the squares of the data. I leave the arithmetic of testing that theory to you!

The Wikipedia article on the beta-binomial is ambiguous about the meaning of [itex] m_2 [/itex] given that the Wikipedia article on moments refers only to "moments about" some value. I'll put a comment on the talk page of the article and see if there is a response.
 

1. What is the Beta-Binomial Model?

The Beta-Binomial Model is a statistical model used to analyze win/loss data in rating systems. It takes into account both the number of wins and losses, as well as the overall success rate of the system.

2. How does the Beta-Binomial Model work?

The Beta-Binomial Model uses a combination of the beta distribution and the binomial distribution to calculate the probability of a win or loss in a given rating system. It takes into account the number of trials (games or matches played) and the success rate (win/loss ratio) to estimate the true underlying skill level of each player or team.

3. What are the assumptions of the Beta-Binomial Model?

The main assumptions of the Beta-Binomial Model are that the outcomes of each trial (game or match) are independent of each other, and that the success rate of the system is constant over time. Additionally, the model assumes that the true underlying skill levels of the players or teams are normally distributed.

4. How is the Beta-Binomial Model used in rating systems?

The Beta-Binomial Model is often used in conjunction with other rating systems, such as Elo or Glicko, to calculate the ratings of players or teams. It can also be used to predict future outcomes and make adjustments to ratings based on the results of new games or matches.

5. What are the limitations of the Beta-Binomial Model?

While the Beta-Binomial Model is a useful tool for analyzing win/loss data in rating systems, it is not without its limitations. It assumes that the success rate of the system is constant over time, which may not always be the case. It also does not take into account external factors that may affect the outcomes of games or matches, such as injuries or changes in strategy. Additionally, the model may not be suitable for very small or very large sample sizes.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • General Discussion
Replies
2
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • Calculus and Beyond Homework Help
Replies
2
Views
3K
Replies
28
Views
11K
  • STEM Academic Advising
Replies
13
Views
2K
Replies
10
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
6
Views
5K
  • Math Proof Training and Practice
3
Replies
101
Views
11K
  • Mechanical Engineering
Replies
1
Views
3K

Back
Top