A problem with the standard solution of the two envelopes paradox

[matt grime]

Why would they do that? "Common sense" is irrelevant here. They are trying to "solve" the two envelopes problem. To "solve" it means to show the readers exactly what's wrong with the calculation that says that we should switch. That's why they "chose" 1/2, not because it's common sense, but because it's the probability that appears in the calculation that says that we should switch.

You're confusing things I have said. The reason why it appears paradoxical is becuase 1/2 appears to be the correct probability to use from a naive vewi point. The explanation is that this is not using any correct posterior distribution.

Devlin's "solution"

try using 'explanation of the paradox'

Devlin also explains that the conditional probabilites in the correct expression for the expected gain given A=a depend on the prior distribution. He then shows that they can't be 1/2 for all a.

just remember you said this..

I will try one more time to explain why this is less than satisfying to me. Devlin claims that he has solved the problem. To solve it means to identify the mistake in the argument that told us to switch. Devlin says that that mistake is that we used the prior probabilites instead of the posterior probabilites. That implies (in the strictest logical sense of the word) that if we had used the posterior probabilites, we would have found that it makes no difference if we switch or not.

No it doesn't. As you yourself point out, different posteriors give different expected gains on the switch depending on the observed value.

Edit: A calculation of the expected gain given A=a can tell us what the correct decision is, but only if we know both the value of a and the prior distribution (i.e. the method was used to prepare the envelopes). If we know these things, the problem is a completely different problem than the one we started with.

According to you the expected gain is zero, always. Which is incorrect.There are key points in the paradox.

1) the expected gain given the observed amount must be by a factor of 5/4
2) this is independent of making the observation, thus every swap multiplies your winnings by 5/4.

These just are not true statements.

If we did not open the envelope, the the expected value in it is 5/4*(smaller value), which is coincidentally the same as the expectation of switching, hence there is no value in changing. But given we do observe the amount in the envelope this is no longer valid. We don't know what is the correct answer because we don't know the posterior.

 

[Fredrik]

The reason why it appears paradoxical is becuase 1/2 appears to be the correct probability to use from a naive vewi point.

No, it isn't. The reason is that a correct calculation of E(B|A=a)-E(A|A=a) will tell us what the correct decision is in a different situation. See my answer to Hurkyl below for more details on this.

According to you the expected gain is zero, always. Which is incorrect.

That's not what I'm saying. I'm just saying if it isn't, then the details about how the probabilities depend on the prior distribution is irrelevant, and so is the fact that there's no prior distribution that makes the probabilities 1/2 for all a. See my answer to Hurkyl below for more details on this.

1) the expected gain given the observed amount must be by a factor of 5/4

The result of the naive calculation of the expected gain given A=a is 1/4*a, because the the result of the naive calculation of the expected value of B given A=a is 5/4*a.

If we did not open the envelope, the the expected value in it is 5/4*(smaller value), which is coincidentally the same as the expectation of switching,...

It's 3/2*(smaller value).

...hence there is no value in changing. But given we do observe the amount in the envelope this is no longer valid. We don't know what is the correct answer because we don't know the posterior.

Knowledge of the amount in the first envelope changes nothing. We still know for sure that there's no value in changing (assuming that we have no information about the prior distribution). It is only if we learn both the amount and the prior distribution that the problem changes into another one. In this other (very different) problem we can expect to gain E(B|A=a)-a by switching.

 

[Fredrik]

What bothers me about this argument is that if this whole line of reasoning is valid, then we should be able to show that E(B|A=a[/color])=a.

(my edit in blue)

I know there are inaccuracies like this one in the posts I made before post #26. (You're quoting #22). I understand what I'm supposed to call things now. You made that clear in #24.

Devlin says that that mistake is that we used the prior probabilites instead of the posterior probabilites. That implies (in the strictest logical sense of the word) that if we had used the posterior probabilites, we would have found that it makes no difference if we switch or not.

I don't follow either of these assertions.

Neither does Matt. I don't see why. You even say that you don't follow either of my assertions. That surprises me since there are only two sentences in the text you quoted and the first one is obviously true. Just look at what Devlin wrote:

The correct expected gain calculation is:

P(B=2A|A=a) * 2a + P(A=2B|A=a) * a/2 - a​

The paradox above arose because you assumed that

P(B=2A|A=a) = P(A=2B|A=a) = 1/2​

...

To summarize: the paradox arises because you use the prior probabilities to calculate the expected gain rather than the posterior probabilities.

Those are his words. I have only changed the notation.

What I said in the second sentence you quoted is not obvious, but still not very difficult to see. The paradox arose because a naive (and incorrect) calculation of E(B|A=a)-E(A|A=a) yielded the result 1/4*a, which is positive. Now, why would a positive result cause a paradox? It doesn't. Not by itself. Only a person who believes that a calculation of E(B|A=a)-E(A|A=a) can tell us what the correct decision is in the situation that was described in the specification of the problem will conclude that a positive result is a paradox!

If you just tell this person that he got the probabilites wrong in the calculation of E(B|A=a)-E(A|A=a), have you really explained the paradox to him? Absolutely not. You also have to tell him that even if he gets the probabilities right, the result will only tell him what the correct decision is in a completely different situation (a situation in which both the prior distribution and the value a is known).

Now, if you tell him that, then that's all he needs to understand what caused the paradox. That whole thing about the probabilities not really being 1/2 is completely irrelevant. It adds nothing to his understanding of the resolution of the paradox. (It adds to his general knowledge, and is something he will probably find interesting, but that doesn't make it relevant).

 

[matt grime]

I completely fail to see what it is that you state is missing from the explanation in, say, Devlin, that is actually missing. All of the things you claim are required for a full explanation of the paradox are in that linked article.

If you get the probabilities correct, then it tells you the correct answer in *every* situation, well, the *only* situation that there is. It really isn't hard: if you use the correct probabilities then you get the correct answer. If you don't you get junk. The explanation of the paradox is that the probabilities that are shoved in are nonsense. Devlin, et al, tell you what the correct things to put in are. What on Earth is the issue here? Since the probabilities to be put in are completely unknown, how can any solution tell you the expected earnings?

 

[matt grime]

Neither does Matt. I don't see why. You even say that you don't follow either of my assertions. That surprises me since there are only two sentences in the text you quoted and the first one is obviously true.

No, it isn't. The 'first one' was that you asserted that the expected gain from swapping is zero. It isn't. We don't know what the expected gain is except in terms of a posterior distribution that we do not know.

 

[Fredrik]

I completely fail to see what it is that you state is missing from the explanation in, say, Devlin, that is actually missing. All of the things you claim are required for a full explanation of the paradox are in that linked article.

I have explained it lots of times. I don't know what else I can say. Devlin never mentions that if it's possible to calculate the expected gain as a function of the amount in the first envelope, the result of the calculation is the solution of a different problem.

If you get the probabilities correct, then it tells you the correct answer in *every* situation, well, the *only* situation that there is.

I guess you can say that, but this "only situation there is" is not the situation that was specified in the problem we're working with! Read post #1 again. We don't know the prior distribution!

It really isn't hard: if you use the correct probabilities then you get the correct answer.

Yes, the correct answer to the wrong problem.

The explanation of the paradox is that the probabilities that are shoved in are nonsense.

No it isn't. The first step of the explanation is to realize that the probabilities in the calculation of E(B|A=a)-E(A|A=a) depend on the prior distribution. The second and final step is to realize that that means that E(B|A=a)-E(A|A=a) will tell us what the correct decision is in a situation where both a and the prior distribution is known, but not in the situation that was specified in the problem!

Devlin doesn't do the second step. Everything he does after the first step is irrelevant.

No, it isn't. The 'first one' was that you asserted that the expected gain from swapping is zero.

Unless Hurkyl has told you privately that he meant something other than what he said, you are wrong. He quoted two sencences of mine and said that he can't follow "either of these assertions". Each sentence is one assertion. The 'first one' is that Devlin claims that the cause of the paradox is the confusion of prior and posterior probabilities.

And stop saying that I have asserted that E(B|A=a)-E(A|A=a) is zero! I have never said that.

 

[matt grime]

Given your misuse of terminology, it is hard to decide what you have asserted is the problem. See post 25 where you contradict yourself.

 

[Fredrik]

No, I don't. And what misuse of terminology? Yes, I wrote E(B) when I should have written E(B|A=a), but that's a minor detail. It can't have caused any of your confusion.

However, there are a few things in #25 that reveals that at the time I had not fully understood what the cause of the paradox is. It didn't become completely clear to me until I wrote #28.

For example, I said in #25 that I don't know if we should be able to show that E(B|A=a)=a. At the time, I didn't know. (Now I know we can't). I just knew that if we can't, then there's something wrong with Devlin's argument. (That's what made me consider the possibility that maybe we should be able to show that E(B|A=a)=a after all). It took me a while to figure out what was missing from his argument.

 

[EnumaElish]

That whole thing about the probabilities not really being 1/2 is completely irrelevant. It adds nothing to his understanding of the resolution of the paradox.

Don't your observations 1 and 2 in #25 two roundabout (or overly general) ways of saying that p=1/2 doesn't work?

 

[Hurkyl]

In #29, I quoted two separate passages, each of them comtaining an assertion (this implies that). I meant I didn't follow either passage.

(I mentioned my edit in because wouldn't be right to have corrected the typo unannounced)



As for what "resolves" the paradox, I consider pointing out the flaw in the argument a resolution. The flaw in the envelope paradox is (in my notation) that it uses the value P(A=2B) when it's supposed to use P(A=2B|A=a).

Anything beyond merely pointing out the flawed step in the argument is simply a bonus.