Understanding the Uniform Probability Distribution in Statistical Ensembles

[bananabandana]

Homework Statement

Confused about what a statistical ensemble actually means. Why does the ensemble have to have a uniform probability distribution at equilibrium? [If my definition of an ensemble is correct]

The Attempt at a Solution

This is what I understand so far: [/B]

• For any given macrostate, there is going to be an associated set of microstates ( a region in phase space)
• If we look at a great number of systems started off with the same macrostate, under the same conditions, then if we looked simultaneously at all of them some time later you would have say, five in microstate 1, 2 in microstate 2 etc. etc.
• So in this way you have a probability distribution for the microstates [as a function of time] - is this what an ensemble is?
• But then why is it necessarily true that at equilibrium the probability distribution must be uniform? I.e the ensemble doesn't change with time?
• I can understand the logic for an isolated system - but why does it hold in general?

Thanks!

 

[Demystifier]

I think that that the concept of ensemble in statistical physics creates more confusion than explanation. Most ideas of statistical physics can be more easily understood without it. The central concept one needs to start from is probability, and if one understands what probability is then the concept of ensemble is superfluous. The concept of ensemble is created for those who are confused with the concept of probability, with the motivation to better explain what probability "really is". However, the explanation based on ensemble can easily create even more confusion.

That being said, one can concentrate on your questions which do not mention ensemble. The entropy can be defined in terms of probability (without referring to ensembles) as
$$S=-\sum_x P(x) ln P(x)$$
where $x$ are points in the phase space of physical interest and $P(x)$ is the corresponding probability distribution. Depending on the physical context, the states $x$ can be either (fine-grained) microstates or (coarse-grained) macrostates. By definition, equilibrium probability distribution is the one that maximizes entropy. So it is a matter of relatively straightforward calculation that entropy (defined by equation above) is maximal when probability is uniform.

Or perhaps you wanted to know where does the definition "equilibrium probability distribution is the one that maximizes entropy" comes from?

 

[A. Neumaier]

if one understands what probability is then the concept of ensemble is superfluous.

But to make sense of the concept of probability in the spirit of Kolmogorov one needs the notion of realizations of random variables. The ensemble is just the collection of all conceivable realizations.

why is it necessarily true that at equilibrium [...] the ensemble doesn't change with time?

Equilibrium is defined through stationarity in time. Thus the ensemble is time-independent by definition. If there are changes in time, is is a sure sign of lack of equilibrium.

 

[A. Neumaier]

the ensemble have to have a uniform probability distribution at equilibrium?

Different ensembles have different distributions. The distribution of the canonical ensemble is not uniform.

 

[N88]

I think that that the concept of ensemble in statistical physics creates more confusion than explanation. Most ideas of statistical physics can be more easily understood without it. The central concept one needs to start from is probability, and if one understands what probability is then the concept of ensemble is superfluous. The concept of ensemble is created for those who are confused with the concept of probability, with the motivation to better explain what probability "really is". However, the explanation based on ensemble can easily create even more confusion.

That being said, one can concentrate on your questions which do not mention ensemble. The entropy can be defined in terms of probability (without referring to ensembles) as ...

"The central concept one needs to start from is probability, and if one understands what probability is."

What is your understanding of probability, please?

 

[Demystifier]

What is your understanding of probability, please?

It depends on the context, so the question is too general. Can you ask a more specific question?

 

[bhobba]

"The central concept one needs to start from is probability, and if one understands what probability is." What is your understanding of probability, please?

It often leads to heated argument but my understanding is Kolmogorov's axioms
http://www.econ.umn.edu/undergrad/math/Kolmogorov's Axioms.pdf

Like QM itself you can have different interpretations - frequentest, Baysian, Decision theory etc etc.

Interestingly much of QM interpretations is simply an argument about probability
http://math.ucr.edu/home/baez/bayes.html

I hold to the ignorance ensemble interpretation of QM which is ensemble, frequentest in its interpretation of probability etc etc - it all really means the same thing. Copenhagen, Bayesian etc etc (I really can't tell the difference) is Bayesian. Many worlds is decision theory based - at least in modern times.

Chose whatever you like - but for heavens sake don't worry about it - all are equally valid, or not valid - its meaningless really - just a personal preference thing or sometimes a specific choice makes solving a problem easier.

I did a degree in applied math where I had to do mathematical statistics 1a, 1b, 2a, 2b, and as an elective also 3a, 3b. We would seamlessly choose between different views such as frequentest or Baysian purely on the problem. Why not do the same in QM?

Thanks
Bill

 

[N88]

It depends on the context, so the question is too general. Can you ask a more specific question?

Demystifier, I was asking in the context of your statement: "I think that that the concept of ensemble in statistical physics creates more confusion than explanation. Most ideas of statistical physics can be more easily understood without it. The central concept one needs to start from is probability, and if one understands what probability is then the concept of ensemble is superfluous."

For I take an ensemble to be the starting point for understanding probability.

PS: Thanks Bill; I'm also a fan of Ballentine.

 

[Demystifier]

Demystifier, I was asking in the context of your statement: "I think that that the concept of ensemble in statistical physics creates more confusion than explanation. Most ideas of statistical physics can be more easily understood without it. The central concept one needs to start from is probability, and if one understands what probability is then the concept of ensemble is superfluous."

For I take an ensemble to be the starting point for understanding probability.

PS: Thanks Bill; I'm also a fan of Ballentine.

Then let me use an example. Suppose that you flip a coin, but only ones. How would you justify that the probability of getting heads is $p=1/2$? Would you use an ensemble for that?

 

[Mentz114]

Then let me use an example. Suppose that you flip a coin, but only ones. How would you justify that the probability of getting heads is $p=1/2$? Would you use an ensemble for that?

It would be premature to try to assign a probability based on one trial ( I hope you don't gamble).

I should say though, that if I knew the phase space of the coin tossing Hamiltonian, I could use that to get a probability.

 

[Demystifier]

It would be premature to try to assign a probability based on one trial

How about zero trials? Even without flipping a coin I would predict that probability of getting heads is p=1/2.

 

[Mentz114]

How about zero trials? Even without flipping a coin I would predict that probability of getting heads is p=1/2.

You would be making an assumption. Not all coins are 'fair'.

Being serious for a moment - if we have a Hamiltonian we can define an ensemble without reference to probability. Probability is not fundamental - likelihood ( the number of ways something can happen) comes first ( in physics anyway).

 

[Demystifier]

Probability is not fundamental - likelihood ( the number of ways something can happen) comes first ( in physics anyway).

I agree with that. But do we necessarily need ensembles for that purpose?

 

[bhobba]

Probability is not fundamental - likelihood ( the number of ways something can happen) comes first ( in physics anyway).

We are delving into rather obvious circularity here - which is why the axiomatic approach is best for clarity.

Thanks
Bill

 

[stevendaryl]

It often leads to heated argument but my understanding is Kolmogorov's axioms
http://www.econ.umn.edu/undergrad/math/Kolmogorov's Axioms.pdf

I would take those axioms to define what it means to be a probability function---it's any function on subsets of events such that blah, blah, blah. But it doesn't say what it means to say that something has probability X, because for any set of possibilities, there are infinitely many different probability functions.

 

[A. Neumaier]

We are delving into rather obvious circularity here - which is why the axiomatic approach is best for clarity.

The axiomatic approach only tells one what is permitted to do with probabilities, not what they are.

Various items on the meaning of probability can be found in Chapter A3: Classical probability of my theoretical physics FAQ.

 

[stevendaryl]

You would be making an assumption. Not all coins are 'fair'.

Being serious for a moment - if we have a Hamiltonian we can define an ensemble without reference to probability. Probability is not fundamental - likelihood ( the number of ways something can happen) comes first ( in physics anyway).

Well, as Bill says, there is something a little circular about saying that

If there are N possibilities, then each possibility has likelihood \frac{1}{N}.

That conclusion assumes that each possibility is equally likely. So you need some notion of likelihood to start with. We might say that "If I throw a pencil, there are three possibilities: It could land on its side, or it could land on its point, or it could land on the eraser." But obviously, those three possibilities don't all have probability 1/3.

It's possible that a lot of probability can be derived from some assumption along the lines of:

If there is a symmetry relating all N possibilities, then they are all equally likely.

 

[bhobba]

The axiomatic approach only tells one what is permitted to do with probabilities, not what they are.

Hmmmm - yes - but its a bit more complicated

From Feller - An Introduction To Probability Theory And Its Applications page 3
'We shall no more attempt to to explain the true meaning of probability than the modern physicist dwells on the real meaning of mass and energy or the geometer discusses the the nature of a point. Instead we shall prove theorems an show how they are applied'

It is the 'intuition' you build up in seeing how its applied that is its real content. In particular you need to see concrete examples of this thing called event in the axioms.

Thanks
Bill

 

[A. Neumaier]

If I buy a lottery ticket, there are two possibilities: Either I win the jackpot, or I don't. The jackpot is worth many thousand times the lottery ticket. Assigning equal probabilities (or likelihoods) I should buy a dozen tickets and win almost with certainty.

If there is a symmetry relating all N possibilities, then they are all equally likely.

But this principle is almost never applicable. Coins, for example, are not symmetric, neither are dice.

It therefore requires some knowledge of physics to assign correct probabilities to a physical system that is not measured.

 

[A. Neumaier]

It is the 'intuition' you build up in seeing how its applied that is its real content. In particular you need to see concrete examples of this thing called event in the axioms.

Yes. This is the analogue of shut-up-and-calculate in applied mathematics.

 

[stevendaryl]

I agree with that. But do we necessarily need ensembles for that purpose?

I may have been overly influenced by Bayesianism, but it seemed to me that ensembles (and the associated frequentist probability) doesn't actually help in understanding probabilities. You can understand "A coin toss has 50/50 chance of resulting in heads or tails" in terms of repeated trials as follows:

"A coin toss has a 50/50 chance of heads or tails" means "Tossing a coin 100 times will produce 50 \pm 5 heads and 50 \mp 5 tails with probability 99%" (or whatever the number is). But you've just defined the probability for one event (tossing a single coin) in terms of the probability for a different event (tossing 100 coins). You haven't explained anything.

 

[stevendaryl]

But this principle is almost never applicable. Coins, for example, are not symmetric, neither are dice

Well, subjective (Bayesian) probability doesn't have this problem, because there is a symmetry in our knowledge about the coins. That is, we don't have any reason to prefer one side of a coin over another.

It therefore requires some knowledge of physics to assign correct probabilities to a physical system that is not measured.

Physics by itself isn't good enough either. Physics allows you to deduce (in principle---in practice, it's often too complicated) probabilities for final states from assumed probability distributions on initial states. But physics alone doesn't tell us the probabilities of the initial states.

 

[Mentz114]

Well, as Bill says, there is something a little circular about saying that

If there are N possibilities, then each possibility has likelihood \frac{1}{N}.

That conclusion assumes that each possibility is equally likely. So you need some notion of likelihood to start with. We might say that "If I throw a pencil, there are three possibilities: It could land on its side, or it could land on its point, or it could land on the eraser." But obviously, those three possibilities don't all have probability 1/3.

It's possible that a lot of probability can be derived from some assumption along the lines of:

If there is a symmetry relating all N possibilities, then they are all equally likely.

I don't know what you mean. There's no mention of phase space.

I mean something very different by 'likelihood'.

If you throw 2 dice, the various outcomes depend on the number of ways they can happen. And they are not the same. Probability is normalized likelihood and likelihoods are raw phase-space volumes.

 

[bhobba]

Well, subjective (Bayesian) probability doesn't have this problem, because there is a symmetry in our knowledge about the coins. That is, we don't have any reason to prefer one side of a coin over another.

As I said one flicks between different interpretations depending on the problem. You can't use a frequentest view to assign a reasonable a-priori probability to a coin - but in the Bayesian view its rather trivial then one uses Bayesian inference to update the probabilities.

Thanks
Bill

 

[bhobba]

If you throw 2 dice, the various outcomes depend on the number of ways they can happen. And they are not the same. Probability is normalized likelihood and likelihoods are raw phase-space volumes.

I think you are introducing a Bayesian reasonable a-priori view here. This stuff is notoriously slippery and circular.

Thanks
Bill

 

[A. Neumaier]

Well, subjective (Bayesian) probability doesn't have this problem, because there is a symmetry in our knowledge about the coins. That is, we don't have any reason to prefer one side of a coin over another.

Subjective probability isn't good either since one can choose arbitrary probabilities that have nothing to do with the real situation. As it is subjective, any choice is as good as any other. I can prefer head 3 times as much as tail because it shows the value of the coin. Your argument not to prefer one side of the coin is based on reason, which is a subjective preference. (Most people are often unreasonable.)

What you are effectively saying is that one needs reason to determine the probabilities. I agree.

But the only acceptable reasons to predict the later frequencies are physical - any other reasons don't matter!

Physics by itself isn't good enough either. Physics allows you to deduce (in principle---in practice, it's often too complicated) probabilities for final states from assumed probability distributions on initial states. But physics alone doesn't tell us the probabilities of the initial states.

Experiment and data collection is part of physics. They determine the missing information in the physical models.

If you can see that a physical system is in equilibrium you know that you only need to determine a few numbers to determine the full density operator and hence all probabilities. If you can see that the only relevant degree of freedom is the polarization, you need to determine just four numbers - the components of the Stokes vector. And so on.

Theoretical physics tells you precisely which kind of information you need to determine the probabilities, Experimental physics tells you what are the reliable ways to obtain this information.

 

[bhobba]

Probability is normalized likelihood and likelihoods are raw phase-space volumes.

The difference between probability and likelihood is exactly what? Please be precise. I think you will find its very very slippery just like pinning down exactly what a point is rather slippery. That's why the axiomatic method was developed - it wasn't just so pure mathematicians could while away their time.

Thanks
Bill

 

[stevendaryl]

If you throw 2 dice, the various outcomes depend on the number of ways they can happen. And they are not the same. Probability is normalized likelihood and likelihoods are raw phase-space volumes.

But you're making the assumption that equal volumes in phase space are equally likely. I guess you could say that that's the way you're defining "likelihood", but why phase space? For a single particle, you could characterize the particle's state (in one-dimension, for simplicity) by the pair p, x, where p is the momentum. Or you could characterize it by the pair v, x, where v is the velocity. If you include relativistic effects, v is not linearly proportional to p, so equal volumes in p,x space don't correspond to equal volumes in v,x. So why should one be the definition of "equally likely" rather than the other?

 

[bhobba]

But you're making the assumption that equal volumes in phase space are equally likely.

Exactly. Based on intuition and experience we all make reasonable assumptions.

Thanks
Bill

 

[stevendaryl]

Subjective probability isn't good either since one can choose arbitrary probabilities that have nothing to do with the real situation. As it is subjective, any choice is as good as any other. I can prefer head 3 times as much as tail because it shows the value of the coin. Your argument not to prefer one side of the coin is based on reason, which is a subjective preference. (Most people are often unreasonable.)

But the point is that subjective probability is subjective, so the fact that different people use different probabilities is not a problem.

I guess intuitively we feel that some people's subjective probabilities are more accurate than other people's, but you would need a nonsubjective notion of probability to make such a judgement.

But the only acceptable reasons to predict the later frequencies are physical - any other reasons don't matter!

Physics alone can't tell you anything about probabilities unless you know the initial conditions exactly. If you don't know them exactly, then in probabilities you compute must be weighted by your notion of likelihood of initial conditions. So you can't avoid subjective probabilities, it seems to me.

 

[A. Neumaier]

you would need a nonsubjective notion of probability to make such a judgement.

Of course. But physics is based on an objective notion of probability defined as expected relative frequency - with expectations checkable by experiment within the standard statistical limits.

Physics alone can't tell you anything about probabilities unless you know the initial conditions exactly.

This is simply false.

We never know the initial conditions exactly and nevertheless make very useful predictions using the physical laws and reliably collected data.

We know the probability for decay of all familiar radioactive substances objectively to a fairly high accuracy. We predict probabilities for the daily weather and companies depending on whether pay a lot for accurate prognosis. We can calculate predictions for probabilities of quantum optics experiments to the point that we can reliably refute the Bell inequalities. And so on. All this is done using physics and slightly inaccurate knowledge to get objective (though a little approximate) probabilities.

Nowhere is the slightest use made of subjective probabilities.

Subjective judgments (and in particular subjective probabilities) have no place at all in physics. Their reasonable place is constrained for making value judgments about the relevance or success likelihood of what we do, priority judgments about what we should do, choices about which physical system to study in which detail, which part of a scientific study to make public, etc.. Every other use of subjectivity is - from the scientific point of view - a blunder.

 

[bhobba]

Subjective judgments (and in particular subjective probabilities) have no place at all in physics.

Hmmmmm. A Copenhagenist might argue that one.

I think Jaynes was a physicist.
http://bayes.wustl.edu/etj/prob/book.pdf

My view is its malleable - chosen purely for utility.

Thanks
Bill

 

[stevendaryl]

This is simply false

No, it's simply true.

We never know the initial conditions exactly and nevertheless make very useful predictions using the physical laws and reliably collected data.

Subjective probability is used all the time to make useful and accurate predictions.

We know the probability for decay of all familiar radioactive substances objectively to a fairly high accuracy.

Once again, what I said was that to make objective probabilistic predictions in physics, you have to know the initial states. We don't know exactly the initial states of atoms. We make a guess, and that guess is good enough for most purposes.

We predict probabilities for the daily weather and companies depending on whether pay a lot for accurate prognosis.

Subjective does not mean useless. Subjective probabilities can be used for useful and accurate predictions.

 

[A. Neumaier]

Subjective probability is used all the time to make useful and accurate predictions.

Probabilities that lead to accurate predictions are objective, not subjective. For objectivity is what agrees with Nature.

With your use of the notion ''subjective'' everything physicists do, and all science is subjective, and the term (and its opposite ''objective'') lose their traditional meaning.

 

[bhobba]

Probabilities that lead to accurate predictions are objective, not subjective.

Baysisan inference - how does that fit? It can be done in a frequentest way but its not natural.

Thanks
Bill

 

[stevendaryl]

Subjective judgments (and in particular subjective probabilities) have no place at all in physics

That's completely false. We can't make any predictions at all without making assumptions that are subjective. You have to assume that your theory is correct, in the first place. You have to assume that your measurement devices worked correctly. You have to assume that you've accounted for all the relevant causal effects. You have to assume that records of past measurements were accurately recorded. There are countless assumptions that everyone must make in order to do the simplest sort of reasoning in physics. Most of those assumptions are completely subjective. You can certainly try to check your assumptions by repeating your measurements, and double-checking everything, but it's subjective whether you've repeated things enough times, whether you've double-checked enough times.

It is impossible to get along in the world without subjective judgments.

 

[A. Neumaier]

to make objective probabilistic predictions in physics, you have to know the initial states.

The S-matrix gives objective probabilities for the outcomes given the input. The input is very accurately known in collision experiments - so accurate that they can check whether the scattering predictions come true or would represent violations of the standard model.

 

[stevendaryl]

Probabilities that lead to accurate predictions are objective, not subjective.

Whether a prediction is "accurate" or not is subjective. You predict that a coin toss has a 50% chance of resulting in heads. You toss 100 coins, and get 53 heads. Was that an accurate prediction, or not? It's not 50%. At some point, you're going to make a subjective decision that your statistics agree close enough with your predictions, and then you'll declare the predictions accurate.

 

[stevendaryl]

The S-matrix gives objective probabilities for the outcomes given the input.

The S-matrix makes asymptotic predictions: Some number of particles come in from infinity, where it's assumed that there are no interactions, collide and then the product particles go out to infinity. In the real world, we don't have particles coming in from infinity, and particles are always interacting. So to compare the S-matrix to actual experiments requires judgment. I claim that there is a subjective element to that judgment, inevitably.

 

[stevendaryl]

With your use of the notion ''subjective'' everything physicists do, and all science is subjective, and the term (and its opposite ''objective'') lose their traditional meaning.

It's a subjective judgment to call something objective. I know that's unsatisfying, but that's the way it is.

 

[stevendaryl]

It's a subjective judgment to call something objective. I know that's unsatisfying, but that's the way it is.

I can see that this has gotten into a philosophical discussion about the meaning of probability and objectivity, and that's probably off-topic. So I will refrain from further replies on this topic.

 

[A. Neumaier]

Bayesian inference - how does that fit? It can be done in a frequentest way but its not natural.

Bayesian inference if done in an objective manner, means to account for prior information in the likelihood function in a roundabout way. One adds extra prior terms that reflect (in a frequentist interpretation) what would have been obtained from data equivalent to the assumed knowledge. If the assumed knowledge (i.e., the prior) is true knowledge, the resulting Bayesian prediction is more accurate than without the prior; if the prior represents prejudice only, the resulting Bayesian prediction is heavily biased towards the prejudice unless a huge amount of data are present to cancel it.

For example, the Kalman filter for updating a Gaussian probability model is Bayesian in form as the current model is updated each time an additional data set comes in. However, if one considers the whole data stream as the data, it can be seen (when started with an improper prior at time zero) to be an optimal model according to the purely frequentist Gauss-Markov theorem for the estimation of linear models. The same holds for REML (restricted maximum likelihood), which is in spirit Bayesian but can be fully treated in a purely frequentist framework.

Thus it is only a matter of presentation and subjective preference whether to take a Bayesian or a frequentist view. Bayesian statistics is not intrinsically related to a subjective view of probability. It is a mathematical technique that is used in statistical practice in a shut-up-and-calculate way like quantum mechanics in physical practice.

In case you think I might not understand what I am talking about: As part of my work at the University of Vienna, I give regularly courses on statistical data analysis. I have written a big survey article about regularization (the abstract version of Bayesian inference in linear models) in SIAM Review 40 (1998), 636-666. I have worked on the Bayesian (REML) estimation of large animal breeding models; algorithms based upon my work are used all over the world to decide on animal breeding.

 

[bhobba]

In case you think I might not understand what I am talking about:

You obviously do. The initial probability, how is that arrived at in a frequentest view?

Take for example a coin. You start with it at 50-50 then flip the coin to update. In a frequentest view why would you start at 50-50?

Thanks
Bill

 

[A. Neumaier]

It's a subjective judgment to call something objective.

As everything is subjective according to your usage of the word, it is meaningless to apply the adjective to anything, as it has no discriminative value. Your usage is far from how everyone else uses the word.

Is there anything that, according to you, fully deserves being called objective?
If not, why do you think the language contains such a term?
Why is science generally considered to collect objective knowledge?

the meaning of probability and objectivity, and that's probably off-topic. So I will refrain from further replies on this topic.

The topic is ''what is an ensemble?'' and this is essentially synonymous with ''what is probability?'' It has a large physical (objective) aspect and a small philosophical (subjective) aspect. You are pulling the weight fully to the subjective side, but this is your subjective bias.

 

[Mentz114]

But you're making the assumption that equal volumes in phase space are equally likely. I guess you could say that that's the way you're defining "likelihood", but why phase space? For a single particle, you could characterize the particle's state (in one-dimension, for simplicity) by the pair p, x, where p is the momentum. Or you could characterize it by the pair v, x, where v is the velocity. If you include relativistic effects, v is not linearly proportional to p, so equal volumes in p,x space don't correspond to equal volumes in v,x. So why should one be the definition of "equally likely" rather than the other?

Because physics is about phase and configuration space. Most of what you've been saying is off topic. You're moving the goalposts around wildly so I don't know what you are trying to say.

Have a look at this
https://en.wikipedia.org/wiki/Phase_space_formulation
and this
https://web.stanford.edu/~peastman/statmech/phasespace.html
and
http://arxiv.org/abs/1003.0772
and
http://www.springer.com/us/book/9780792337942

 

[A. Neumaier]

The initial probability, how is that arrived at in a frequentest view?

Take for example a coin. You start with it at 50-50 then flip the coin to update. In a frequentest view why would you start at 50-50?

You wouldn't unless you have good reasons to assume that the coin is almost fair.

In both the frequentist and the Bayesian case one starts with a prior count $H_0$ of heads and $T_0$ of tails. Then you flip a number of times and find $H$ heads and $T$ tails. You update the frequencies and get $H'=H_0+H$ and $T'=T_0+T$. Then you estimate the probability for head as $P_H:=H'/(H'+T')$.

If one initially knows nothing at all - in fact, unknown to everybody, someone prepared the coin so that both sides show head, and the experimenters see only the result, not the act of falling! -, the Bayesian starts with the unwarranted assumption [using an allegedly ''uninformative prior'', but still a prejudice] that $H_0=T_0>0$ (with a value that depends on how strongly the prior is believed to be true) while the frequentist puts correctly $H_0=T_0=0$. It takes the Bayesian estimate a long time to realize that the coin was forged, while the frequentist gets the answer correct from the start. This shows the bad influence of a prejudice. (A real person would soon be suspicious about the coin, but a true Bayesian - following objective shut-up-and-calculate techniques rather than being subjective) will be unable to do that.

On the other hand, if the coin is known to be almost fair (because it looks like many other coins that have been tried before), both Bayesian and frequentist will assign $H_0=T_0>0$ - the frequentist by making a (somewhat subjective) estimate of how many equivalent coin flips the prior knowledge is worth, and checks during the computation whether the assumed estimate has a large effect on the result. (In technical terms, this is a regularization parameter. There are a number of ways this parameter can be objectively chosen under appropriate assumptions.) I have no idea how a true Bayesian would assigns the actual value of $H_0=T_0>0$ since probability theory gives no hints. In practice, there is no difference between the two; it is shut-up-and-calculate according to recipes taken from the literature.

If there are enough data and the prior is not weighted too much, the result is indifferent to the value of the prior.

 

[A. Neumaier]

Jaynes was a physicist.

But he was mistaken about his subjective interpretation of physics. His interpretation only works because he knew already (from half a century of prior objective physics) which subjective assumptions he has to make to get it objectively correct. If he would assume in place of the subjective knowledge of $\langle H\rangle$ (which Nature happens to make use of) the subjective knowledge of $\langle H^2\rangle$ (which Nature abhors) he would have obtained in place of the canonical ensemble a ridiculously wrong ensemble. And even with the canonical ensemble, if he would know subjectively the wrong value of $\langle H\rangle$ (which is very well possible since in a subjective, stevendaryl-type of physics, no one specifies objectively what it means to have knowledge, then Jaynes would assign an equally wrong value for the temperature.

This proves that even in the context of the maximum entropy principle, only knowledge of the objectively correct information produces a reliable physical model and enables reliable physical predictions. Again, there is nothing subjective in the physics. Subjective deviations from the objective reality lead here (as always) to inaccurate or even grossly wrong predictions.

 

[Mentz114]

The difference between probability and likelihood is exactly what? Please be precise. I think you will find its very very slippery just like pinning down exactly what a point is rather slippery. That's why the axiomatic method was developed - it wasn't just so pure mathematicians could while away their time.

Thanks
Bill

bhobba,

I mean as in the likelihood function defined here.

https://en.wikipedia.org/wiki/Maximum_likelihood

 

[A. Neumaier]

I mean as in the likelihood function defined here.

https://en.wikipedia.org/wiki/Maximum_likelihood

Then it is the logarithm of the probability density with respect to a prior measure. This is surely less fundamental than the notion of probability, which is independent of a prior measure.

 

[stevendaryl]

As everything is subjective according to your usage of the word, it is meaningless to apply the adjective to anything, as it has no discriminative value. Your usage is far from how everyone else uses the word.

Is there anything that, according to you, fully deserves being called objective?

No. I don't. I think that it's a short-cut in reasoning. To take into account all the ways that our judgments are influenced by unproved assumptions is intractable and inconvenient. So it's useful to be able to have cut-offs, where you treat sufficiently unlikely possibilities as if they were impossibilities. So the kind of reasoning that people typically do is a rule of thumb. It's subjective, but it's not consciously subjective.