Maximizing College Counselor Accuracy: A, B, or C?

  • Thread starter Thread starter hupdy
  • Start date Start date
  • Tags Tags
    Choices
hupdy
Messages
8
Reaction score
0
Hello all, I have a question about a problem given in class. Here is the quetion:

There are 3 college counselors( A, B, C). The counselors will pick a college for the student.

1. A is correct 80% of the time.
2. B is correct 10% of the time.
3. C choice depends on A and B. If A and B choose to agree then C agrees, if A and B disagree then C choice will always be wrong.

The student can choose to see A alone or B alone.
The student can choose to see A,B,C.
The student can choose to see A and B.

Can the % of being correct excesses 80%?

Any help will be great.
 
Physics news on Phys.org
hupdy said:
Hello all, I have a question about a problem given in class. Here is the quetion:

There are 3 college counselors( A, B, C). The counselors will pick a college for the student.

1. A is correct 80% of the time.
2. B is correct 10% of the time.
3. C choice depends on A and B. If A and B choose to agree then C agrees, if A and B disagree then C choice will always be wrong.

The student can choose to see A alone or B alone.
The student can choose to see A,B,C.
The student can choose to see A and B.

Can the % of being correct excesses 80%?

Any help will be great.
I think I know this guy B!

You already know the probabilities if you consult only A or only B.
What if you consult both A and B?
Assuming independence, then A and B will both be correct .8(.1)= 0.08 or 8% of the time. They will both be wrong .2(.9)= .18= 18% of the time. The remaining 74% of the time one will be wrong and the other right but you have no way of knowing which. What do you do then?

If you consult A, B, and C, then you know that in the 26% of the cases that they agree, C will also agree with them, but they will all three be wrong 18% of the time. You also know that if A and B disagree, then C is always wrong. How about this strategy: consult A, B, and C. Ignore A and B but whatever advice C gives, do the opposite! Since C will be wrong 18%+ 74%= 92% of the time, you now have a 92% chance of being right! (And if A and B have disagreed, you know your choice is right.)
 
Thanks for the help, can you elaborate more on the 2nd part?
 
Your strategy ("C is always wrong so do the opposite") makes sense if there are two colleges to choose between, but the problem doesn't specify only two! If there are N colleges to choose from, then all you know is C is wrong about one of them and the right one lies in the N-1 remaining choices. I feel you can still exceed 80% chance of success, but I have to think longer about proving it.
 
A good point. I thought about that as I was responding but then I don't see anyway to answer the question. I suspect that the problem is assuming the simple "yes" "no" situation.
 
Ok, here’s my attempt to answer the question for N colleges and seeing counselors A and B. Counselor A recommends college H and we'd do pretty well listening since A is right most of the time. If B's recommendation differs, we should have a little greater confidence since B is mostly wrong and extra information is helpful. (Think of it like this: B’s answer effectively eliminates one college since B is usually wrong, so A is now selecting mostly correctly from a smaller pool of choices, boosting our confidence.) So my answer is "yes".

There might be an easy way to show it, but I only know Bayes' rule. The probability of H being correct given that A recommends H and B recommends something else is
P(H|A_H, B_~H) = P(A_H, B_~H|H) * P(H) / P(A_H, B_~H),
where
P(A_H, B_~H|H) is prob. that A recommends H and B recommends something else given that H is correct,
P(H) is probability that H is correct college with no other a priori information, and
P(A_H, B_~H) is the marginal probability of A recommending H and B something else. This last is summed over the two possibilities H is correct and H is wrong:
P(A_H, B_~H)= P(A_H, B_~H|H) * P(H) + P(A_H, B_~H|~H) * P(~H).

We know
P(H) = 1/N;
P(~H) = 1 – P(H) = (N-1) / N
P(A_H |H) = x, and
P(B_H|H) = y or P(B_~H|H) = 1-y. (x and y are 0.8 and 0.1 in the problem)

We can write
P(A_H, B_~H|H) = P(A_H |H) * P(B_~H|H) = x*(1-y)
since the counselors work independently. Likewise the second term is
P(A_H, B_~H|~H) = P(A_H |~H) * P(B_~H|~H). We need values for these.

P(A_H |~H) = ( 1 - P(A_~H |~H) ) / (N-1) = (1-x) / (N-1). [Prob. that H is wrong answer but A chooses it out of N-1 possibilities.]
P(B_~H|~H) = 1 - P(B_H|~H) = 1 – (1-P(B_~H|H)) / (N-1) = (N-2+y) / (N-1). [Prob. that H is wrong answer and B doesn’t give H. Note that B may still be wrong since there are many answers that aren’t H].

The hard part is over (whew!) and the rest is rewriting and simplifying. The answer is
P(H|A_H, B_~H) = x*(1-y)*(N-1) / ( N*(1-xy) + x + y - 2 ).
Sure hope I did that right! :rolleyes:

Plugging in x = 0.8 and y = 0.1 for this problem,
P(H|A_H, B_~H) = 0.72 * (N-1) / (0.92*N – 1.1).

This gives probabilities
N P
2 0.973
3 0.868
.
.
8 0.802
9 0.800
10 0.798
inf 0.783

So we definitely do better than A by also listening to B, so long as N<=8.

It’s interesting that we do worse by adding B’s information for N>9. If B were always 100% wrong, then his information would always aid us as can be seen by setting y=0 above.

After that I have no energy left to add counselor C!

Edit: corrected a couple of typos.
 
Last edited:
Sorry for another typo, this time in the chart N vs. P.
N P
2 0.973
3 0.868
.
.
9 0.802
10 0.800
11 0.798
inf 0.783

If we solve for the condition where adding B's guess helps over taking A's alone
P(H|A_H, B_~H) > P(H|A_H) = x
with the equation for the LHS in the previous post, we get the condition
N*y < 1.
Since y=0.1, the situation is improved for N<10 as in the table.
 
The Beauty behind this problem

Wow, I had an Aha! moment while thinking about this on the commute home last night. One can answer the question posed without equations.

The chance of randomly choosing the right one of N colleges is 1/N. Again call A’s probability of choosing correctly x, B’s y. Add B’s recommendation to A’s only if B’s performance is better than random, which depends on N relative to y.
1. If y = 1/N, then B’s performance is random and gives no useful information. Take A’s choice with probability x of being correct.
2. If y < 1/N, B is right less often, or wrong more often, than random, which is useful; the probability of success P exceeds x when A and B disagree. This is the case analyzed in the earlier post.
3. If y > 1/N, it means B is right more often than random, which is also useful. P exceeds x when A and B agree.
So the crossover point that came out of the Bayes analysis has a logical basis.

We can apply the same reasoning to counselor C.
4. If A and B agree, then C gives no useful information and can be ignored.
5. If A and B disagree, then C is always wrong which is useful for all values of N. There are two further possibilities:
a. If y < 1/N, we do even better than case 2 by considering C and having two disagreements. The Bayesian analysis for this case is left as an exercise for the reader. (lol, just kidding. I’ve always wanted to say that! :smile: )
b. If y > 1/N, B is no help when it disagrees with A, but we have C disagreeing with 100% certainty. Again we do better than x, according to the formula in the preceding post with y = 0.


For the sake of completeness, we write down the Bayesian result for case 3. It follows closely the earlier derivation of case 2, and most of the terms were already found there. The result is
P(H|A_H, B_H) = P(A_H, B_H|B) * P(H) / P(AB) = x * y * (N-1) / ( 1 - x – y + x*y*N )
and it’s easy to show that this exceeds A’s performance
P(H|A_H, B_H) > P(H|A)
whenever y > 1/N. The crossover is visible for both cases 2 and 3 in the attached plot.


Thanks for posting this problem! It has beauty and depth.
 

Attachments

Last edited:
How about a slight variation of the original problem...(i have quite some argument with my friends over this problem...heh)

A student has to pick a college from A, B, or C. The student has to see 3 different advisors for advice. The 1st advisor is correct 80% of the time; The 2nd advisor is correct 10% of the time; The 3rd advisor gives the same answer as 1st and 2nd if 1st and 2nd agreed with each other. Otherwise, if 1st and 2nd disagreed with each other, the 3rd advisor will always give you the wrong answer.

With the above information given, can the probability of the student picking the correct college exceed 80%?
 
  • #10
I like this problem. I'm going to take a different approach from marcusl, since that seems covered pretty well. I'll look for optimal approaches for a given number of colleges. Here I assume that when the right choice is not given, the selection is uniformly random between the remaining colleges.

If there is one college: (100% right)
* All three will recommend the same college; go there. (100% right)

If there are two colleges: (92% right)
* If A and B agree, go to the college A didn't recommend (9/13 ~= 69% right)
* Otherwise, go to the college C didn't recommend (100% right)

If there are three colleges: (81% right)
* 17/100: If A and B agree, go there (8/17)
* 11/200: If A and C agree, go to the college no one recommends (9/11)
* 155/200: Otherwise, go to the college A recommends (144/155)

So with these strategies, you can beat 80% for 1, 2, or 3 colleges. Generalizing:

Fix Q = 1/(n-1) and R = (n-2)Q. If there are 3 <= n <= 10 colleges: ()
* (8+18Q)/100: If A and B agree, go there (8/(8+18Q))
* (2QQ+18Q)/100: If A and C agree, choose a random college no one recommends (9Q/(9R+1))
* (18QQ+2Q+72)/100: Otherwise, go to the college A recommends (72/(18QQ+2Q+72))
(I'm having trouble making this one work; I'm sure I have a simple mistake or three.)

For n > 10 colleges:
* If A and B agree, go there (8/17)
* If A and C agree, go to the college B recommends (1/10)
* Otherwise, go to the college A recommends (8/10)
 
Last edited:
  • #11
Thanks wlkhoo, that's the question I was aimning for, thanks everyone for the help. So can it be solve for any number of colleges or would it have to be a fix number?
 
  • #12
Nice work, CRGreathouse. Your analysis is simple and you summarized all the combinations!
 
  • #13
My answer seem to confirm with CRGreathouse for 3 colleges.
 
Back
Top