Is there a way to calculate expected value from probabilistic data?

[Office_Shredder]

Once you understand that adding these stops doesn't change your expected value, you can do a very cool computation.

Suppose I sell if it goes down to 980, or up to 1060. What's the probability it hits 980 first?

Hint: if the probability is p, compute your expected value. What does it have to equal?

 

[Feynstein100]

The average profit should not be higher @FactChecker the graph they put in the original post is a terrible way of visualizing what is going on.

That's a bit strong lol. Graphs are useful as long as you know what they represent. My original graph was meant to represent the possible paths followed by a game of consecutive coin flips. It's literally what's going on. It was never meant to represent the number of heads and tails you got. Although, the other graph sounds interesting too. I'll see if I can change my code to draw that graph too.

 

[Feynstein100]

Once you understand that adding these stops doesn't change your expected value, you can do a very cool computation.

Suppose I sell if it goes down to 980, or up to 1060. What's the probability it hits 980 first?

Hint: if the probability is p, compute your expected value. What does it have to equal?

Hmm that's an interesting question. My intuition says that both outcomes are equally likely i.e. p = 1-p = 0.5
So, the expectation value would be
1020 * 0.5 - 980 * 0.5 = 20
Which shouldn't be possible because I feel like the expectation value should still be zero. There must be an error here somewhere
Edit: changed 1060 to 1020 because 1060 isn't equally likely as 980, 1020 is.
Edit2: My formula for expectation value was wrong. The actual formula is just the sum of each outcome multiplied by its probability. Which gives us
1020 * 0.5 + 980 * 0.5 = 1000
As expected

 

[jbriggs444]

Hmm that's an interesting question. My intuition says that both outcomes are equally likely i.e. p = 1-p = 0.5
So, the expectation value would be
1020 * 0.5 - 980 * 0.5 = 20
Which shouldn't be possible because I feel like the expectation value should still be zero. There must be an error here somewhere
Edit: changed 1060 to 1020 because 1060 isn't equally likely as 980, 1020 is.

You are subtracting when you should be adding.

 

[Feynstein100]

You are subtracting when you should be adding.

Damn, dude. You beat me to the punch by a millisecond

 

[pbuk]

1020 * 0.5 + 980 * 0.5 = 1000

Yes but that is trivial, what @Office_Shredder was asking is more interesting: 1060 * ? + 980 * ? = ?

 

[Feynstein100]

Yes but that is trivial, what @Office_Shredder was asking is more interesting: 1060 * ? + 980 * ? = ?

Eh idk. In the end it should still come out to 1000. 1060 may be a higher value but it's less likely to happen than 980. These two factors balance each other out and the result is still 1000.

 

[Feynstein100]

The expected value is not a function of standard deviation.

@BWV
Turns out, there's more to this. I took this from Wikipedia:

Here, mu is the mean/expected value. So while mu isn't a function of sigma, the two do interact with each other. It's just that when you change sigma, you also change the probability and the resulting data points, in just such a way that mu remains unchanged. My hunch was correct, in a sense

 

[Office_Shredder]

This is a general statement that if you want to compute the standard deviation, you have to subtract the mean out. This is actually what prevents them from usually being related - if you adjust the mean by adding some quantity to every outcome, that doesn't affect the standard deviation at all.

 

[jbriggs444]

Eh idk. In the end it should still come out to 1000. 1060 may be a higher value but it's less likely to happen than 980. These two factors balance each other out and the result is still 1000.

We are trying to solve for the probability ratio. I do not know if @Office_Shredder has a particular approach in mind, but here is how I would attack it.

Let $p$ be the probability of first hitting 980. Then $1-p$ is the probability of first hitting 1060. We are in an infinite random walk (as I recall), so the probability of never hitting either is obviously zero.

We know that the mean of the infinite random walk needs to be 1000 because that is the starting point and every step is unbiased. So we write down an equation: $980 p + 1060(1-p) = 1000$

Now we just have to solve for p. Easy algebra. Then write down the ratio in the form $p:1-p$.

If intuition serves, we should find that the sides of the ratio are in inverse proportion to each side's deviation from the mean. The deviations are 20:60 so the probability ratio will be 3:1.

[It feels like there should be some more directly compelling algebra if I'd made a better choice of variables. But I will not change horses mid-stream]

Let us see whether the algebra bears this out...

$980p + 1060 (1-p) = 1000$
$(980 - 1060)p + 1060 = 1000$
$-80p = -60$
$p = \frac{3}{4}$
$1-p = \frac{1}{4}$

Ratio = 3:1

Bingo.

 

[Feynstein100]

We are trying to solve for the probability ratio. I do not know if @Office_Shredder has a particular approach in mind, but here is how I would attack it.

Let $p$ be the probability of first hitting 980. Then $1-p$ is the probability of first hitting 1060. We are in an infinite random walk (as I recall), so the probability of never hitting either is obviously zero.

We know that the mean of the infinite random walk needs to be 1000 because that is the starting point and every step is unbiased. So we write down an equation: $980 p + 1060(1-p) = 1000$

Now we just have to solve for p. Easy algebra. Then write down the ratio in the form $p:1-p$.

If intuition serves, we should find that the sides of the ratio are in inverse proportion to each side's deviation from the mean. The deviations are 20:60 so the probability ratio will be 3:1.

[It feels like there should be some more directly compelling algebra if I'd made a better choice of variables. But I will not change horses mid-stream]

Let us see whether the algebra bears this out...

$980p + 1060 (1-p) = 1000$
$(980 - 1060)p + 1060 = 1000$
$-80p = -60$
$p = \frac{3}{4}$
$1-p = \frac{1}{4}$

Ratio = 3:1

Bingo.

Oh.....kay? Don't get me wrong. That was quite interesting but I was kinda expecting a cheat or a workaround of some kind As it stands, we're right where we started. Conservation of expectation value shall not be violated.

 

[jbriggs444]

Oh.....kay? Don't get me wrong. That was quite interesting but I was kinda expecting a cheat or a workaround of some kind As it stands, we're right where we started. Conservation of expectation value shall not be violated.

Right. Re-read #31. @Office_Shredder is not trying to prove conservation of expected value. He is trying to use the principle of conservation of expected value to reach a conclusion about the probability of hitting the stop points.

Conservation of expected value is on pretty solid ground (for a reasonable set of assumptions). I took the assumption to be a starting bankroll of 1000 quatloos and a series of unbiased coin flips that result in a gain or loss of 1 quatloo with probability 0.5 each (an unbiased infinite random walk) with stop points at 980 quatloos or 1060 quatloos.

 

[Feynstein100]

This is a general statement that if you want to compute the standard deviation, you have to subtract the mean out. This is actually what prevents them from usually being related - if you adjust the mean by adding some quantity to every outcome, that doesn't affect the standard deviation at all.

Ah okay. It's more nuanced than I'd anticipated. Btw the equations don't tell us how they change when we use a stop-loss to change sigma. I mean, I kind of discovered it empirically from my python code but is there a way to see it happen mathematically? A proof or something? Idk. I tried to read the proof of the optional stopping theorem but it was beyond me. Way too abstract lol.

 

[Feynstein100]

Right. Re-read #31. @Office_Shredder is not trying to prove conservation of expected value. He is trying to use the principle of conservation of expected value to reach a conclusion about the probability of hitting the stop points.

Oh. My bad. I thought we were trying to find a workaround/loophole

 

[jbriggs444]

Oh. My bad. I thought we were trying to find a workaround/loophole

There are none unless you decide to allow shenanigans such as unbounded bets. e.g. a Martingale scheme. There was a reference upthread to a relevant theorem.

 

[Hornbein]

The sum of the expectations is the expectation of the sum. Each bet has zero expectation. So the expectation of the sum of those bets is zero. It doesn't matter how you decide whether or not to make those bets.

 

[BWV]

Is conservation of expected value is just another way of stating the process is a Martingale - defined as E(T1)=E(Tn) for all n. Not every process would have this attribute

 

[BWV]

@BWV
Turns out, there's more to this. I took this from Wikipedia:
View attachment 314407
Here, mu is the mean/expected value. So while mu isn't a function of sigma, the two do interact with each other. It's just that when you change sigma, you also change the probability and the resulting data points, in just such a way that mu remains unchanged. My hunch was correct, in a sense

If you want to see where the variance influence the mean, change the payoff to + or - 10% of the previous value then compare to another where it is + or - 50% (does not change the mean but the expectation relates to the difference between the mean and median: at + or - 50% you almost surely will go broke if you play long enough even though the expected gain/loss is zero

 

[Feynstein100]

Is conservation of expected value is just another way of stating the process is a Martingale - defined as E(T1)=E(Tn) for all n. Not every process would have this attribute

Sorry I didn't get that. Could you explain it again?

 

[jbriggs444]

Sorry I didn't get that. Could you explain it again?

Does this wiki page make it any more clear?

 

[Feynstein100]

The average profit should not be higher @FactChecker the graph they put in the original post is a terrible way of visualizing what is going on.

@jbriggs444
Here's the frequency distribution of outputs at the end of each game:

This is without the stop-loss.

This is with the stop-loss at 980. The right side actually goes on for quite a while but I wanted the spike at the stop-loss to be visible.

 

[pbuk]

Rather than play around with this, do you think it would be a good idea to learn about probability and statistics in a more structured way? This would prevent you wasting time going down blind alleys, help you focus on what is important, and perhaps also introduce you to even more interesting problems.

You could look at this open course which covers the material touched on in this post:
https://ocw.mit.edu/courses/18-440-probability-and-random-variables-spring-2014/

Or for a more general introduction to probability and statistics try here:
https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2014/

 

[Feynstein100]

Rather than play around with this, do you think it would be a good idea to learn about probability and statistics in a more structured way? This would prevent you wasting time going down blind alleys, help you focus on what is important, and perhaps also introduce you to even more interesting problems.

You could look at this open course which covers the material touched on in this post:
https://ocw.mit.edu/courses/18-440-probability-and-random-variables-spring-2014/

Or for a more general introduction to probability and statistics try here:
https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2014/

Thanks for the recommendations. I'll check them out. However, I don't see any problem with playing around. There's really no substitute for firsthand experience. For instance, if I hadn't messed around with this, I wouldn't have learned that stop-losses and take-profits don't change the expected value. I can't help but wonder what profound implications that might have for daytrading. But then again, is the stock market inherently probabilistic? I'm inclined to think not. And even if it were, it has a net positive expectation value over time. How does that affect our simulation? Is the simulation even applicable? Those are the kinds of questions that interest me.

 

[Office_Shredder]

Thanks for the recommendations. I'll check them out. However, I don't see any problem with playing around. There's really no substitute for firsthand experience. For instance, if I hadn't messed around with this, I wouldn't have learned that stop-losses and take-profits don't change the expected value. I can't help but wonder what profound implications that might have for daytrading. But then again, is the stock market inherently probabilistic? I'm inclined to think not. And even if it were, it has a net positive expectation value over time. How does that affect our simulation? Is the simulation even applicable? Those are the kinds of questions that interest me.

Anyone with a bit of probability knowledge can immediately tell you that giving the stock positive drift means your stop loss order can only lower the expected value. You don't need to run simulations and ponder whether you got the right result when you understand the underlying concepts.

 

[Feynstein100]

giving the stock positive drift

I don't know what that means But based on personal experience, stop-losses do seem to make a difference in daytrading. How can we explain that?

 

[pbuk]

I don't know what that means But based on personal experience, stop-losses do seem to make a difference in daytrading. How can we explain that?

You could try following the advice I gave a few days ago.

@Feynstein100 you don't need to do any maths to see that the stop loss does not change the expected value if returns are symmetrical, you just need to look at the symmetry. Think about the expected value of the investments you sell at 980.

Once you have thought this through, see if you can predict what a stop loss will do in a rising market, then run a simulation.

 

[BWV]

I don't know what that means But based on personal experience, stop-losses do seem to make a difference in daytrading. How can we explain that?

Maybe your personal experience is not reliable?

But you could try this:
https://eml.berkeley.edu//~anderson/Sources-042212.pdf
https://en.wikipedia.org/wiki/Autoregressive_model

 

[Feynstein100]

You could try following the advice I gave a few days ago.

Yeah I didn't find that helpful at all. I mean, the whole reason I posted here is because based on the symmetry, I anticipated an increasing expected value then couldn't figure out why that wasn't true.
I have no idea what a stop-loss would do in a bull market. I expect it to not even be relevant because if the market is indeed ascending then the stop-loss shouldn't even be triggered. That's kind of the point, isn't it? In a bull market, a stop-loss is basically non-existent. In a bear market, the stop-loss minimizes your losses. Looks good on paper but how does it fare in reality?

 

[BWV]

Yeah I didn't find that helpful at all. I mean, the whole reason I posted here is because based on the symmetry, I anticipated an increasing expected value then couldn't figure out why that wasn't true.
I have no idea what a stop-loss would do in a bull market. I expect it to not even be relevant because if the market is indeed ascending then the stop-loss shouldn't even be triggered. That's kind of the point, isn't it? In a bull market, a stop-loss is basically non-existent. In a bear market, the stop-loss minimizes your losses. Looks good on paper but how does it fare in reality?

Define bull vs bear market - how do you know which state you are in until after the fact?

 

[jbriggs444]

The way I see it, there are only a few ways to make money in the stock market.

You can be smarter than most everyone else. That is hard to arrange. It is even harder if you give up and try to pretend that it's all just random.

You can be faster than most everyone else. Also hard to arrange. Fast switches, routers and algorithms.

You can know more than everyone else. Sometimes, that's doing your homework. Other times, it's called insider trading.

You can manipulate news articles and such and thereby manipulate market prices.

You can buy and hold and trust that the companies whose stock you bought will make make a good return on investment.

You can convince someone else that you are skilled enough so that they will pay you to invest their money for them.

My impression is that day trading is like poker. You make your money off of people that think that they are good enough to make money off of you. But it is a negative sum game. Somebody has to lose.