What are the chances of achieving a 42% probability in this scenario?

[Last_Exile]

I'm trying to understand this problem and could do with some help.

A small business is trying to measure their delivery performance in terms of "on-time, in-full & error-free".

It sent out product to 4 customers (A,B,C,D) and notes down what the performance was: 0 = not ok, 1 = ok.

(Apologies for not being able to do a table)

__________A_B_C_D
on time____1_0_1_1
in full______1_1_1_0
error free___0_1_1_1

Overall_____0_0_1_0

It is clear that only 1 out of 4 (25%) actually received goods on-time, in-full & error-free.

But then, a customer calls Sales and asks "What are the chances of my order being on-time in full and error-free?

The Sales guy thinks to himself:

Well, 3 out 4 (75%) were on time, 75% were in full & 75% were error-free so the chances are:

0.75 x 0.75 x 0.75 = 0.42

and tells the customer "There is a 42% chance your order will be on-time, in-full & error-free"

Can anybody explain why there is such a difference between the actual performance and the probability?


Thanks in advance.

 

[D H]

Suppose the company was able to improve its performance to 99.9% in each individual category and as a result garnered thousands of customers. The "Overall" metric will most likely be zero in this case. Think about how that metric is formed.

 

[Last_Exile]

I see what you are saying.

In using 0 or 1, I am forcing it to be either 0% or 100% where in fact each category could be 0 - 99.9%.

Does that mean, then, that the overall metric is not valid? Is the probability of receiving goods otifef really 42%?

 

[D H]

Apologies, first of all. I misread the overall stats. It is obviously the minimum score for a given customer. So my remark on a zero overall score as the number of customers becomes large is incorrect.

The correct answer depends on whether the individual scores are statistically independent. If they are independent, the salesman was basically correct ("basically" because four is far too small sample size). The salesman is of course wrong if some correlation exists among on-time, in-full, and error-free. Do you think these are uncorrelated statistics?

 

[Last_Exile]

I would say that there is little or no correlation between being on-time, in-full or error-free as one doesn't depend on any of the others.

So, it is more correct to say that the probability is 42% and the "actual" result of 25% is due to the small sample?

 

[D H]

In the real world, I suspect there *is* a correlation. As an analogy, "cheaper, better, faster" was a buzz-phrase in the 1990s. The problem: The three often do not go together. You can make something cheaper and better but it may take more time to do so, cheaper and faster at the expense of quality, and better and faster at a higher cost. The same may be true for on-time, in-full, and error-free.

 

[Last_Exile]

OK, I can understand that.

In trying to meet these targets people rush things, generate errors, forget to send all the order or end up sending it late...

However, this company isn't like that (they're quite good at what they do) and want some measure of further improvement and be able to give their customers a reliable answer to the question "what are the chances of my goods being on-time... etc"

 

[D H]

If the company is good at what they do, they should realize that such a correlation will exist and will push the overall probability of success down from p_on-time p _in-full p _error-free. In other words, if the company is good, what the salesman did was wrong. OTOH, a 75% success rate is incredibly lousy. For example, what if 75% of the planes flying in the sky had successful landings? Given such a lousy rate, the metrics might well be uncorrelated, making what the salesman did correct.

 

[Last_Exile]

Thanks for your comments so far - I must admit I'm still trying to get my head around this.

I suppose one way of thinking about it is to start off with a table (as above) but populated by zeroes i.e. nothing is on time, in-full or error-free.

As the company improves the table is randomly populated by 1's until eventually the overall measure also shows 1s.

If I understand what you have said so far, in the beginning (lousy performance) the Sales guy's method is correct (p.on-time p.in-full p.error-free) but as the company improves toward excellence the overall measure is correct.

So, this (overall) measure is not really useful until you've already achieved what you set out to do!

If a customer calls the Sales guy should always give the "best" answer!

 

[Last_Exile]

I have now figured out the answer to my original post.

The table I supplied was just a snapshot. Each row had a probability of 0.75 but I could have achieved that same result with a different mix of 1s & 0s and obtained a different overall result.

If I had gone to the trouble of mapping out all the ways in which 1s & 0s make 0.75 then I would have found that the actual overall probability would have been 42%