Estimating a mean from games of ruin

[lalekl]

Imagine a gambler playing a casino game with fixed bet, fixed odds, no skill, and a starting bankroll, $M_0$. She plays until she can no longer afford to bet and records only how many bets she was able to make, $N_0$, until she could not afford to bet. Each day she goes back to the casino with a larger bankroll, $M_i$, plays the same game until she could no longer afford to bet, and records how many bets she made, $N_i$. Can she determine the true EV of the game using only this information ( her sequences of bankrolls, $\{M_i\}$, and $\{N_i\}$, how many bets she was able to make on day $i$ before she could no longer afford to bet that day)?

 

[andrewkirk]

She can't determine it, but she can make an unbiased estimate of it, for which the expected error decreases with every new day's observation.

Write $p$ for the unknown probability of her winning a bet. On each day $k$ write the probability $R_k$ of losing the initial sum $M_k$ in exactly $N_k$ bets. That will be a binomial probability with that is a function of $p$, $N_k$ and $W_k$. The product of all the $R_k$ for days $k$ she has experienced is the likelihood of the experience she has had. You can use this in Maximum Likelihood Estimation to estimate $p$.

 

[lalekl]

that is an interesting answer, but the casino game could for instance be a slot machine with non-negative, unbounded, discrete support and i don't think it would have any commonly known distribution that would allow us to specify the kind of product you are suggesting.

 

[Dale]

The product of all the RkRkR_k for days kkk she has experienced is the likelihood of the experience she has had. You can use this in Maximum Likelihood Estimation to estimate ppp.

You can also use a Bayesian approach to calculate a new posterior distribution for p after data from each day’s game.

 

[andrewkirk]

that is an interesting answer, but the casino game could for instance be a slot machine with non-negative, unbounded, discrete support and i don't think it would have any commonly known distribution that would allow us to specify the kind of product you are suggesting.

It sounds like you're envisaging a game in which one pays 1 to enter, and can win an amount that is any non-negative integer, with probabilities $p_0,p_1,...$. Then it will be the case that
$$p_k=f(k, a_1,a_2,...,a_m)$$
for some function $f$ and parameters $a_1,...a_m$.

There are two possibilities
Case 1: Gambler knows $f$ but not the parameters $a_1,...,a_m$. Then she can use MLE to estimate those parameters based on her experience.
Case 2: Gambler does not know $f$. Then she can't use MLE directly. But if the experience is long enough, she can make a pragmatic approximation. Let $n$ be the largest win she's ever made in a single play. Then assume temporarily that the probability of winning more than $n$ in a single play is zero. So now we have a game in which she just needs to estimate $p_1,...p_n$, which she can do using MLE. Having done that, she can draw a graph of the $p_k$ estimates and fit a suitable curve to it, that has support on $\mathbb R_+$. She then multiplies that curve by a scalar so that its integral is 1. That accounts for the unobserved tail of big wins.

 

[lalekl]

so we are definitely in case 2. but you could also win less than an integer or any number between the integers, i'll have to think about how that affects your argument. however, she isn't allowed to use any information about the game other than the two recorded sequences, so she can't write down her largest win. Is there another way to do that?

 

[andrewkirk]

you could also win less than an integer

No she can't. In my formulas the winnings are gross not net of the price to play. If we wanted to do it based on net winnings we'd just include an extra parameter $p_{-1}$. In the above, gross, formulation a win of zero on a single play means that she has lost the unit of money she paid to play.

she isn't allowed to use any information about the slots other than the two recorded sequences

That's a peculiar condition, as it implies she has amnesia about everything that happened while inside the casino.
But it can be surmounted. It just makes the approximation a little more rough. She just guesses $n$ as a number greater than she expects any machine would pay out. The curve-fitting and scaling will adjust for this to some extent if she guesses too low.

that throws much of a wrench in your argument

It's not an argument, but a way of estimating unknown parameters given limited information. There's nearly always a way to make an estimate in a given set-up. It's just that they become progressively more inaccurate and slower to converge as the information set is constrained.

 

[lalekl]

so she bets a dollar, wins 0.33333 repeating or maybe 22+pi, we are meant to be completely ignorant of the support...for a given n the amount of parameters to be estimated could be much larger than n, and certainly larger than the number of observations we have.

yes it is peculiar. you can reframe it if you'd like: she doesn't even try to estimate it, she just records that information on a blog each day and that's all you have access to in order to try to estimate the mean

 

[Dale]

If the odds and the bankroll are known then you can calculate the likelihood of any given number of bets being played. So you can use Bayesian methods to go from the observed number of bets played to a posterior distribution on the odds. Once you know the odds then you can directly calculate the EV. So you wind up with a posterior distribution on the EV.

 

[lalekl]

If the odds and the bankroll are known then you can calculate the likelihood of any given number of bets being played. So you can use Bayesian methods to go from the observed number of bets played to a posterior distribution on the odds. Once you know the odds then you can directly calculate the EV. So you wind up with a posterior distribution on the EV.

we don't know the odds of the game, only that they don't change.

 

[Dale]

Right. But if the odds were known then you could calculate the likelihood of the different possible outcomes. That allows you to learn about the unknown odds by observing the outcomes.

 

[lalekl]

Right. But if the odds were known then you could calculate the likelihood of the different possible outcomes. That allows you to learn about the unknown odds by observing the outcomes.

sorry i read your response too quickly. I'm still having an issue where if we don't know the distribution, how can we construct this likelihood function? It seems to me that the number of parameters grows faster than the observations...that's a problem isn't it?

 

[Dale]

It seems to me that the number of parameters grows faster than the observations

No, there is only one unknown parameter, the odds. Unless I am misunderstanding the problem

 

[lalekl]

No, there is only one unknown parameter, the odds. Unless I am misunderstanding the problem

the support of the game is unbounded, should i treat an infinite vector of the odds as one parameter? that doesn't seem to work for me...

 

[Dale]

the support of the game is unbounded,

What do you mean by this?

 

[andrewkirk]

It seems to me that the number of parameters grows faster than the observations

The parameters fix the distribution of the win from a single play, not the distribution of the number of plays to exhaust the initial sum on a given day. So the number of parameters does not grow at all. It is fixed.

the support of the odds is unbounded

This was already covered in posts 5 and 7. And it's not the support of the odds that is unbounded but the support of the winnings from a single play. It's very important to keep one's terminology straight in probability theory or we just end up completely confused.

 

[lalekl]

What do you mean by this?

the unknown distribution could pay out any non-negative number as far as the player is concerned

 

[andrewkirk]

so she bets a dollar, wins 0.33333 repeating or maybe 22+pi

Then the probability of losing all her money on a given day is negligible, because she will end up with an amount less than the price of a single play, unless she never wins anything. In addition, it makes no sense to have a play pay off a real number. Currency is integers, in cents if not in dollars. Nobody ever paid anybody pi dollars.

 

[Dale]

the unknown distribution could pay out any non-negative number as far as the player is concerned

So what do you mean by “fixed odds” in your first post? If the payout is variable in what way are the odds fixed?

Can you describe the game a little more carefully please

 

[lalekl]

So what do you mean by “fixed odds” in your first post? If the payout is variable in what way are the odds fixed?

Can you describe the game a little more carefully please

it means the odds of any given pay do not change day to day, but the player is ignorant of the paytable.

 

[lalekl]

Then the probability of losing all her money on a given day is negligible, because she will end up with an amount less than the price of a single play, unless she never wins anything.

i updated the wording to be less confusing in that respect

 

[lalekl]

In addition, it makes no sense to have a play pay off a real number. Currency is integers, in cents if not in dollars. Nobody ever paid anybody pi dollars.

Either way, the point is she is ignorant of how many possible parameters there are to estimate, can you guarantee that your estimation method doesn't demand more parameters than there is data?

 

[Dale]

it means the odds of any given pay do not change day to day, but the player is ignorant of the paytable.

Then you just treat the paytable as your unknown. The more succinctly that you can parameterize your unknown paytable then the faster you can gather information about it.

 

[lalekl]

Then you just treat the paytable as your unknown. The more succinctly that you can parameterize your unknown paytable then the faster you can gather information about it.

Yes I agree, I think I am having the same issue with your solution as I am with andrewkirk's: a paytable consisting of every pay and a fixed probability for that pay requires you to estimate the number of parameters in the set up. But since she doesn't know those she has to find all possible pays that cannot be paid and assign zero to those. This list of parameters seems to me to be too big to be unique and to require more observations than she will ever have...

 

[Dale]

This list of parameters seems to me to be too big to be unique and to require more observations than she will ever have...

Sure. As @andrewkirk mentioned, you can set up the player’s ignorance in a way that makes it so that very little information is obtained from each play and therefore it will take a long time (like longer than the age of the universe) to converge.

However, it is often possible to parameterize a good approximation to something like the unknown paytable with a relatively few number of parameters. Even though the paytable may not be exact, it can still give you good information in a relatively short time.

The other approach could be to forget the paytable altogether. If all you want to estimate is the EV then simply use that as your parameter to calculate the likelihood of a given number of games

 

[lalekl]

Sure.

The other approach could be to forget the paytable altogether. If all you want to estimate is the EV then simply use that as your parameter to calculate the likelihood of a given number of games

Now we’re cooking with whale oil! But how?

 

[Stephen Tashi]

Imagine a gambler playing a casino game with fixed bet, fixed odds, no skill, and a starting bankroll, $M_0$.

Can she determine the true EV of the game using only this information ( her sequences of bankrolls, $\{M_i\}$, and $\{N_i\}$, how many bets she was able to make on day $i$ before she could no longer afford to bet that day)?

Imagining a gambler at a casino game makes us imagine more information than you specify as given. For example, the gambler would know her winnings on each play of the game. A gambler with a rudimentary understanding of the game would know the amounts that are possible to win, even if she does not know the probability of winning each amount.

Perhaps we should imagine a person Alice who gives her son Bob an amount of money m[j] on each day j and goes to watch a movie while Bob plays the game at a casino. Later in the day Alice askes Bob "How many turns did you play today?" Bob, who always plays till he runs out of money, tells Alice he played n[j] turns. So Alice's only information on day j is (m[j],n[j]). What's a good way for Alice to estimate the expected return for 1 play of the game?

 

[lalekl]

Imagining a gambler at a casino game makes us imagine more information than you specify as given. For example, the gambler would know her winnings on each play of the game. A gambler with a rudimentary understanding of the game would know the amounts that are possible to win, even if she does not know the probability of winning each amount.

Perhaps we should imagine a person Alice who gives her son Bob an amount of money m[j] on each day j and goes to watch a movie while Bob plays the game at a casino. Later in the day Alice askes Bob "How many turns did you play today?" Bob, who always plays till he runs out of money, tells Alice he played n[j] turns. So Alice's only information on day j is (m[j],n[j]). What's a good way for Alice to estimate the expected return for 1 play of the game?

Yeah i like that...I also got complaints i think for using “runs out of money” though, so rewrote as “can no longer afford to bet” or something like that...although i will have to say you can say a player knows that there are inifinite possibilities on s slot machine but it is stretching it to say that they know all the values...

 

[andrewkirk]

It is always possible to do MLE. The general approach is to assume a distribution type (aka family) for the underlying random process. Say the distribution PDF is $f(x;\vec a)$ where $\vec a$ is a vector of unknown parameters. The functional form of $F$ has to be assumed and the parameters $a_j$ are to be estimated.

We have a set of observations which in this case are the pairs $(M_k,N_k)$. To do MLE, we just need to find a formula for the likelihood of each observation, which will be a probability or a probability density for $N_k$ given $M_k$. Then we construct the overall likelihood which is the product of the likelihoods of the individual daily observations:
$$L(\vec N;\vec M, \vec a) = \prod_{k=1}^n L(N_k; M_k,\vec a)$$
We take partial derivatives of this wrt each $a_j$, set them to zero and so get $m$ equations in $m$ unknowns, where $m$ is the dimensionality of $\vec a$ (number of parameters). We solve the system to find $\vec a$.

In case of this casino problem the underlying distribution is that of the payout on a single play. The only thing we know about $f$ is that its integral on $[0,\infty)$ must be 1. We make a pragmatic choice based on what shape we might expect and how many observations we have. The fewer observations we have, the smaller should be the number of parameters, to avoid overfitting.

Say we decide the observation set is large enough to justify $m$ parameters. Then a suitable functional form would be
$$f(x;\vec a)=a_m \delta(x) + C\exp\left(-x^m + \sum_{j=1}^{m-1} a_j x^j\right)$$
The $\delta(x)$ is the Dirac delta function. We use it to allow a nonzero probability $a_m$ of no payout.
The parameter $C$ is chosen to make the integral 1, so it is calculated from the other parameters, rather than estimated. This distribution is always positive and will tend asymptotically to zero as $x\to\infty$, which is what we would expect for a payoff.

Given that, the likelihood of the observation $(M_k,N_k)$ is

$$L(N_k;M_k,\vec a) = \int_0^{D_k-B_1} \int_0^{D_k - B_2}
...
\int_0^{D_k - B_{N_k}} \prod_{r=1}^{N_k} f(x_r;\vec a)
dx_{N_k}...dx_1
$$

where $D_k\triangleq N_k-M_k+1$ and $B_j\triangleq \sum_{r=1}^{j-1} x_r$. Note that $B_1=0$ and $B_2=x_1$.

In solving the MLE, we apply the constraint that $a_m\in [0,1]$ as that is the probability of a zero payout. All the other $a_j$ can be any real number.

Once we have solved the MLE, we have a full estimated specification of the PDF, so we can integrate that to find the expected payoff from a single play.

Given the problem has been specified in recent posts to have continuous rather than discrete support, talk of a 'paytable' is misplaced. Tables only make sense for discrete sample spaces. We are estimating a function, not a table, and we estimate it by assuming a suitable form, then estimating its parameters.

The minimum number of parameters that can be used in this method is one, in which case the parameter is the probability of no payout on a play and the distribution of nonzero payouts is exponential with parameter 1. As the number of observations increases (the number of days for which $(M_k,N_k)$ was observed), the number of parameters can be increased, but I prefer to err on the side of under-fitting, so would only add another parameter after a load more observations.

 

[lalekl]

Listen for any practical real application, your method “works”; but it misses part of the spirit of the problem which is to not have a different method depending on what kind of distributions you think you might be getting from the very limited information set.

The game still has disrcrete support, but a priori any value may be chosen. your solution as phrased is very akin to Freiling’s axiom of symmetry (https://en.wikipedia.org/wiki/Freiling's_axiom_of_symmetry). So maybe there is just a philisophical difference. the problem does not rule out any nonnegative real number precisely to discourage you from creating too many parameters in a philisophically suspect way. also the spirit of the problem is to provide that your solution in general will give us what we want...if you can show that regardless of support and how you mis-specify it or the functional form of the distribution, you will get the correct mean, then you win. You could for instance explain how to update the parameter space or alter the distributional form based on new information...

Complaining about definintions of “paytables” or “going broke” or the impracticality of paying someone in infinite decimals is just poor sportsmanship. That is only natural for me to expect given that i apparently haven’t found a good way to phrase the problem. I am grateful for the efforts you have made, and for the record, and for any existing slot machine, your continuous method would work very well...although I wouldn't be able to formally prove it.