Maximize chance of success based on opinion poll

[alexk]

Hi,

Here is my (imaginary) problem :

I have to make a medical diagnosis for a patient : is he A or B ?

I can not make inspection of the patient, and my only help is a poll of N medical experts that observed him.

Out of N experts, the track records for their past diagnoses are different:

• 22% of experts had a success rate of 40%

• 41% of experts had a succes rate of 53%

• 37% of experts had a success rate of 68%

Questions :

1 / How can I maximise my chance of giving the right medical diagnosis, based only on the source of medical experts ?
Should I take the most used answer of all N experts, or the most answered of n experts (n < N) , or another solution ... ?

2/ What if the diagnosis asked is not only A or B, but A or B or C ?

3/ What if my experts are the best Y experts out of 100, but I don't know the distribution of their success rate ?

Thanks very much for your suggestions !

 

[Valkarie]

1 / Taking the most used answer of all N experts would be the best option as it would give you the highest chance of giving the right diagnosis. If you are looking to increase your accuracy even further, you could look at taking the most used answer of n experts (n < N). This would give you a more focused and accurate result as you are only taking into account the opinions of the most reliable experts. 2/ If the diagnosis asked is not only A or B, but A or B or C, you could use a weighted average approach. This involves assigning weights to each answer based on the success rate for each expert. For example, if 40% of experts think the patient is A, 53% think he is B and 68% think he is C, the weights would be 0.4, 0.53 and 0.68 respectively. The weighted average of the responses can then be calculated and used to determine the most likely diagnosis for the patient.3/ If your experts are the best Y experts out of 100, but you don't know the distribution of their success rate, you could use a similar weighted average approach. You could assign weights to each answer based on the number of experts who answered it, rather than the success rate. For example, if 40% of the best Y experts think the patient is A, 53% think he is B and 68% think he is C, the weights would be 0.4, 0.53 and 0.68 respectively. The weighted average of the responses can then be calculated and used to determine the most likely diagnosis for the patient.