Action, entropy and probability - measures (fuzzy reflections)
We have multiplied our focuses now, so i'll try to comment in bits. I decided to try to be brief, or at brief as possible and I appeal to your intuition.
The following is my personal reflections upon your reflection so to speak.
I guess what I am aiming at is that the concepts entropy, probabilitiy and action are in my thinking very closely related measures. But since I am partly working on my own thinking still, and due to my regularized response I am brief in order to just hint the thinking without attempt to explain or argue in detail.
marcus said:
BTW one thing I don't think we have mentioned explicitly (although it is always present implicitly)
is the classical principle of least action. As an intuitive crutch (not a formal definition!
) I tend
to think of this as the principle of the "laziness of Nature" and I tend to think of "action" in this mathematical
context as really meaning bother, or trouble, or awkward inconvenience.
So it is really (in an intuitive way) the principle of least bother, or the principle of least trouble.
what Feynman seems to have done, if one trivializes it in the extreme, is just to put the square root of minus unity, the imaginary number
i
in front of the bother.
In that way, paths (whether thru ordinary space or thru the space of geometries) which involve a lot of bother cause the exponential quantity to WHIRL AROUND the origin so that they add up to almost nothing. The rapidly changing phase angle causes them to cancel out.
It is that thing about
e^{iA}
versus
e^{-A}
where A is the bother. The former expression favors cases with small A because when you get out into large A territory the exponential whirls around the origin rapidly and cancels out. The latter expression favors cases with small A in a more ordinary mundane way, simply because it gets exponentially smaller as A increases.
I analyse this coming from a particular line of reasoning, so I'm not sure if it makes sense to you but.
Some "free associative ramblings"...Equilibrium can be static or dynamic. Ie. equlibrium can be a state or a state of motion. How do we measure equilibrium? Entropy? entropy of a state vs entropy of a state of motion? Now think away time... we have not define time yet... instead picture change abstractly without reference ot a clock, as something like "uncertainty" and there is a concept similar to random walks. Now this random walk tends to be self organized and eventually a distinguishable preferred path is formed. This is formed by structures forming in the the observers microstructure. Once expectations of this path is formed, it can be parametrized by expected relative changes. Of course simiarly preferred paths into the unknown are responsible for forming space! I THINK that you like the sound of this, and in this respect I share some of the visions of behind the CDT project. But I envision doing away with even more baggage than they do. Of course I ahve not complete anything yet, but then given the progress that others have accomplished the last 40 years with a lot of funding I see no reason whatsoever to excuse myself at this point...
Simplified, in thermodynamics a typical dynamics we see is simple diffusion. The system approach the equilibrium state (macrostate) simply basically by a random walk from low to high entropy or at least that is what one would EXPECT. In effect I see entropy as nothing but a measure of the the prior preference for certain microstates, which is just another measure of the prior probability to find the microstructure in a particular distinguishable microstate.
Traditionally the entropy measure is defined by a number of additional requirement that some feel is plausible. There are also the axioms of cox, that some people like. I personally find this somewhat ambigous, and think that it's more useful to directly work with the "probability over the microstates". The proper justification of a particular choice measure entropy is IMO more or less physically equiuvalent to choosing measures for the microstates. It's just that the latter feels cleaner IMO.
I've come to the conclusion that to predict things, one needs to make two things. First to try to find the plausability(probability) of a transition, ie. given a state, what are the probability that this state will be found in another state? Then if one considers the concept of a history, one may also try to parametrize the history of changes. What is a natural measure of this? some kind of itme measure? maybe a relative transition probability?
I think there is a close connection with the concept of entropy, and the concept of prior transition probability. And when another complication is added there is a close relation to the action and transition probabilities.
I find it illustrative to take a classical simple example to illustrate how various entropy measures relate to transition probabilities.
Consider the following trivial but still somewhat illustrative scenario:
An observer who can distinguish k external states, and from this history and memory record, he defines a prior probability over the set of distinguishable states, define by the relative frequency in the memory record. We can think of this memory structure as defining the observer.
Now he may ask, what is the probability that he will draw n samples according to a particular frequency distribution?
This is the case of the multinomial distribution,
<br />
P(\rho_i,n,k|\rho_{i,prior},k) = n! \frac{ \prod_{i=1..k} \rho_{i,prior}^{(n\rho_{i})} }{ \prod_{i=1..k} (n\rho_{i})! }<br />
now the interesting thing is that we can interpret this as the probability (in the space of distributions) of seeing a transition from a prior probability to a new probability. And this transition probability is seen to be related to the relative entropy of the probability distributions. This is just an example so I'll leave out the details and just claim that one can find that
<br />
P(\rho_i,M,k|\rho_{i,prior},k)= w e^{-S_{KL}}<br />
Where
<br />
w = \left\{ M! \frac{ \prod_{i=1..k} \rho_{i}^{(M\rho_{i})} }{ \prod_{i=1..k} (M\rho_{i})! } \right\} <br />
S_{KL} is the "relative entropy", also called Kullback-Leibler divergence or information divergence. It is usually considered a measure of the missing relative information between two states. The association here is that the more relative information that's missing the more unlikely is the transition to be observed. The other association is that the most likely transition is the one that minimizes the information divergence, this smells like action thinking.
w can be interpreted as the confidence in the final state. w -> 1, as the confidence goes to infinity. The only thing this does is hint the principal relation between probability of probability, and the entropy of the space of spaces etc. It's an inductive hierarchy.
M is the number of counts,
loosely associative to "inertia" or information content, or the number of distinguishable microstates. Strictly this is unclear, but let it be an artistic image of a vision at this point :)
One difference between thermodynamics and classical dynamical systems is that in thermodynamics that equilibrium state is usually a fixed macrostate point, in dynamical system the equilibrium is often say a orbit or steady state dynamical pattern. it should be conceptually clear here how the notion of entropy is generalized.
So far this is "classical information" and just loose associative inspiring reflection.
One should also note that the above is only valid given the prior distribution, and this is emergent from the history and memory of the observer the probabiltiy is relative to the observers history - so it's a conditional probabiltiy first of all. Second, the probability is updated gradually, so strictly speaking the entropy formula only makes sense in the differential sense, since technically after each new sample, the actions are updated!
So what about QM? What is the generalization to QM and how does the complex action and amplitudes enter the picture? I am working on this and I don't have the answer yet! but the idea, that I for various reasons think will work is that the trick on howo make QM consistent with this is to consider that the rention of the information, stored inthe observers microstructure, may be done in different, and generally unknown ways! because the microstructure of an observer could be anything. They question is mainly, what are the typical microstructures we encounter in nature? For example, is there some plausabilit arguments as to what the elementary particles have the properties they have?
The idea is that there are internal equilibration processes going on, in parallell to the external interactions, in a certain sense I association here to datacompression and _learning_. Given any microstructure I am looking to RATE different transformations in the whole and parts of the microstructure. Now we are closing up on sometihng that might look like actions. The actions are association to transformations, and each transformation have a prior probability, that is updated all the way as part of the evolution and state change. In a certain sense the state change may be considered as the superficial changes, and the evolutionary changes of the microstructure and action rations is a condensed form that evolves slower.
I can't do this yet, but if I am right and given time I think I can be able to show how and why these transformations give rise to something that can be written in complex amplitudes. It basically has to do with retention of conflicting information, that implies a nontrivial dynamics beyond.
What is more, the size of the memory record (complexity of the microstructure) is clearly related to the confidence in the expectations. Because we know loosely speaking from basic statistics the confidence level increases as the data size increases - this, in my thinking is related to proto-ideas of intertia. Note that I am never to to assume that there is a connection, my plan is to show that the information capacity itself possesses intertia! This also contains the potential to derive some gravity action from first principles. The complex part is exactly the part that all of these things we circle really are connected. I'm trying to structure it, and eventually some kind of computer simulations is also in my plan.
I wrote reflections in a few settings, and there is probably no coherent line of reasoning the builds the compelxity here, but this is my reflection on the action stuff. This can be made quite deep and it's something I an processing ongoingly.
/Fredrik
