Understanding the Uniform Probability Distribution in Statistical Ensembles

[A. Neumaier]

OK, then let my try something completely different. I flip the coin twice, and I obtain the result:
heads, heads
What is the probability of getting heads? Can probability be assigned in that case?

It is in [0,1].

 

[Demystifier]

It is in [0,1].

So is there any case in science where one can assign definite probabilities, without performing an infinite number of experiments?

 

[stevendaryl]

In the absence of further knowledge both choices are rational. There is no way to compare the quality of the choices except by waiting for the consequences. To make choices, no concept of probability is needed.

It's not needed, but probabilities provide a coherent way to reason about uncertainties. That's one of the arguments in favor of the axioms of probability: If you express your uncertainties in terms of probability, then you have a principled way to combine uncertainties. If you don't, then you can become the victim of a "Dutch book" scam:

https://en.wikipedia.org/wiki/Dutch_book

 

[A. Neumaier]

So is there any case in science where one can assign definite probabilities, without performing an infinite number of experiments?

Assuming that probabilities have definite (infinitely accurate) values is as fictitious as assuming the length of a stick to have a definite (infinitely accurate) value. Science is the art of valid approximation, not the magic of assigning definite values.

One uses statistics to assign uncertain probabilities according to the standard rules, and one can turn these probabilities into simple numbers by ignoring the uncertainty. That's the scientific practice, and that's what theory,
and standards such as NIST, tell one should do.

 

[A. Neumaier]

probabilities provide a coherent way to reason about uncertainties.

Only about aleatoric uncertainty. This is the consensus of modern researchers in uncertainty quantification. Se the links given earlier.

 

[Demystifier]

Assuming that probabilities have definite (infinitely accurate) values is as fictitious as assuming the length of a stick to have a definite (infinitely accurate) value. Science is the art of valid approximation, not the magic of assigning definite values.

One uses statistics to assign uncertain probabilities according to the standard rules, and one can turn these probabilities into simple numbers by ignoring the uncertainty. That's the scientific practice, and that's what theory,
and standards such as NIST, tell one should do.

OK, then please use this scientific practice to determine probability in my post #90.

 

[A. Neumaier]

If you don't, then you can become the victim

You don't need to teach me how to reason successfully about uncertainty. Our company http://www.dagopt.com/en/home sells customized software that allows our industrial customers to save lots of money by making best use of the information available. They wouldn't pay us if they weren't satisfied with our service.

It is a big mistake to use probabilities as a substitute for ignorance, simply because with probabilities ''you have a principled way to combine uncertainties''.

 

[A. Neumaier]

OK, then please use this scientific practice to determine probability in my post #90.

Respectable scientists are no fools that would determine probabilities from the information you gave.

 

[Demystifier]

Respectable scientists are no fools that would determine probabilities from the information you gave.

OK, what is the minimal amount of information that would trigger you to determine probabilities? How many coin flips is the minimum?

 

[stevendaryl]

Only about aleatoric uncertainty. This is the consensus of modern researchers in uncertainty quantification. Se the links given earlier.

I think there are times when the different types of uncertainty have to be combined. For example, if you're taking some action that's never been done before, such as a new particle experiment, or sending someone to Mars, or whatever, some of the uncertainties are statistical, and some of the uncertainties are epistemic--we may not know all the relevant laws of physics or conditions.

So suppose that there are two competing physical theories, theory A and theory B. If A is correct, then our mission has only a 1% chance of disaster, and a 99% chance of success. If B is correct, then our mission has a 99% chance of disaster and 1% chance success. But we don't know whether A or B is correct. What do we do? You could say that we should postpone making any decision until we know which theory is correct, but we may not have that luxury. It seems to me that in making a decision, you have to take into account both types of uncertainty. But how to combine them, if you don't use probability? I guess you could say that you're just screwed in that case, but surely there are extreme cases where we know what to do: If A is an accepted, mainstream, well-tested theory and B is just somebody's hunch, then we would go with A.

 

[A. Neumaier]

OK, what is the minimal amount of information that would trigger you to determine probabilities? How many coin flips is the minimum?

Are you so ignorant about statistical practice? It depends on the accuracy you want.

 

[stevendaryl]

Respectable scientists are no fools that would determine probabilities from the information you gave.

If they have the luxury of performing more tests, then they can put off making any kind of decision until they have more data. But at some point, you have to make a decision based on the data that you have.

 

[Demystifier]

Are you so ignorant about statistical practice? It depends on the accuracy you want.

You are smart, but I am smart too.
I want the minimal possible accuracy that will trigger you to assign some definite numbers as probabilities.

 

[stevendaryl]

Are you so ignorant about statistical practice? It depends on the accuracy you want.

There is never a point when you know that your probability estimate is accurate. There is never a point when you can say with certainty: "The probability of heads is between 45\% and 55\%."

 

[Demystifier]

There is never a point when you know that your probability estimate is accurate. There is never a point when you can say with certainty: "The probability of heads is between 45\% and 55\%."

In other words, all you have is the probability of probability of probability of probability of ...

 

[A. Neumaier]

sending someone to Mars [...] What do we do?

We developed uncertainty software for ESA - see Robust and automated space system design for a related publication.

The way of arriving at a design that is acceptable to all parties that have a say is a far more complex process than assigning of faulty probabilities to epistemic uncertainties. Idealized textbook decision procedures are as irrelevant there as are the textbook state reduction recipes when doing a real measurement of a complex process.

 

[A. Neumaier]

There is never a point when you can say with certainty

Physics is never about certainty but about the art of valid approximation.

We treat photons as massless not because we know it for certain but because the mass is known to be extremely small.

Every argument in physics that demands the exact knowledge of the numbers involved is extremely suspicious.

 

[stevendaryl]

We developed uncertainty software for ESA - see Robust and automated space system design for a related publication.

The way of arriving at a design that is acceptable to all parties that have a say is a far more complex process than assigning of faulty probabilities to epistemic uncertainties.

That doesn't mean it is more accurate or more rational. The actual process is as much about psychology (reassuring all interested parties) as it is dealing with uncertainty.

 

[stevendaryl]

Physics is never about certainty but about the art of valid approximation.

But there is no point where you know that your approximation is valid.

 

[A. Neumaier]

You are smart, but I am smart too.
I want the minimal possible accuracy that will trigger you to assign some definite numbers as probabilities.

I'll never assign definite probabilities. probabilities are always estimates, with an intrinsic uncertainty.

 

[A. Neumaier]

But there is no point where you know that your approximation is valid.

This is why physics is slightly in flux, and why these are sometimes controversies.

 

[stevendaryl]

I'll never assign definite probabilities. probabilities are always estimates, with an intrinsic uncertainty.

It doesn't make any difference if you include uncertainty in the probability, because the assignment of uncertainty is itself uncertain.

 

[A. Neumaier]

That doesn't mean it is more accurate or more rational. The actual process is as much about psychology (reassuring all interested parties) as it is dealing with uncertainty.

Yes, as everywhere in life. Including in science.

You need to convince your peers that you did your study according to scientific standards. Claiming a probability based on two coin tosses will convince no one.

 

[Demystifier]

I'll never assign definite probabilities. probabilities are always estimates, with an intrinsic uncertainty.

OK, then just estimate probabilities, with the lowest possible accuracy, for the lowest possible number of coin flips. Can you perform such a lowest possible task?

 

[A. Neumaier]

the assignment of uncertainty is itself uncertain.

Indeed. There is no certainty about real numbers obtained from experiment. It is never like $\pi$ or $e$ or $\sqrt{2}$.

 

[A. Neumaier]

OK, then just estimate probabilities, with the lowest possible accuracy, for the lowest possible number of coin flips. Can you perform such a lowest possible task?

Play this game by yourself, using a book on elementary statistics!

 

[stevendaryl]

OK, then just estimate probabilities, with the lowest possible accuracy, for the lowest possible number of coin flips. Can you perform such a lowest possible task?

You know how it's really done. People pick a cutoff C-- 5% or 1% or 0.1% or whatever. Then they flip the coin often enough so that they can say:

If the probability of heads were outside of the range p \pm \delta p, then the probability of getting these results would be less than C.​

So, relative to the cutoff choice C, they conclude that the probability is in the range p \pm \delta p. That's not actually what you want to know, but it's the best you can get, using frequentist methods.

 

[Demystifier]

Play this game by yourself, using a book on elementary statistics!

Are you a politician? (Never giving a direct answer to a tricky question.)

 

[A. Neumaier]

to a tricky question

To a physically meaningless question.

 

[Demystifier]

That's not actually what you want to know, but it's the best you can get, using frequentist methods.

Sure, but I want to force Neumaier to admit that sometimes Bayesian methods make more sense. But he's smart, he understands what I am trying to do, so that's why he doesn't answer my questions.