Is Information Theory Driving Astrophysics in the 21st Century?

  • #31
Go ahead, flatter me!

Chronos said:
I had no idea Chris was so well versed, and appreciative of the power of IT.

:smile:

IT was my first mathematical interest. People who know me only from my public postings seem to assume I only know about general relativity, but a page count of my personal notes and a title count of my mathematical library suggests that only 1/15 to 1/12 of my mathematical knowledge directly concerns gtr. Of course, perhaps the most wonderful aspect of mathematics is that techniques valuable in one area often turn out to be valuable in others; for example, my interest in perturbation theory doesn't arise from gtr but from my interest in symmetries of differential equations; however, it is very useful in gtr!

Chronos said:
That is awesome, as are the references. Give me about 2 weeks to bone up before daring to comment. I thought modern scientists were virtually ignoring this, IMO, extraordinary resource. I stand [actually jumping up and down] corrected.

:smile:
 
Physics news on Phys.org
  • #32
Told ya!

Fra said:
I checked some papers and stuff relating to to Boltzmann and also discussion by Einstein, and it seems they indeed used the term complexions and complexion number

People sometimes seem to assume I am making the whole thing up! :rolleyes:

Fra said:
Anyway, since I'm not seeing it from the point of pure math, boltzman and einstein seems to be using (my interpretation from context) it pretty much synonymous or a slight generalisation to the notion of the set of distinguishable microstates consistent with the macrostate

It will certainly help to extract the pure math. Suppose we have some set X. A finite partition of X, written \pi, is a decomposition of X into r disjoint blocks A_j \subset X such that X = \cup_{j=1}^r A_j (perhaps needless to say, "disjoint" means A_j \cap A_k = \emptyset, \; j \neq k). You can say that the elements of X are microstates each of which gives rise to a unique macrostate, with the macrostates corresponding to the blocks of the partition. Or you can say that the blocks are the preimages of some function f:X \rightarrow \mathbold{R}; for example, a function assigning an "energy" to each element (taking a different value on each block). My point is that none of this need have anything to do with physics.

Fra said:
and the complexion number is the number of possible distinguishable microstates or "possibilities" consistent with the constraints or macrostate. This is like the boltzmanns entropy except I suppose the microstate is generalised beyond the mechanical analogue.

Right, if you follow up my citation of the expository paper by Brian Hayes in American Scientist, and if you recall the fundamental orbit-stabilizer relation from elementary group theory (see any good book on group theory, for example Neumann, Stoy, and Thompson, Groups and Geometry), you should be able to see that the natural action by S_n on an n-set X induces an action on the set of partitions of X, and the size of the orbit of \pi is then
\frac{n!}{n_1! \, n_2! \dots n_r!}
while the orbit itself is the coset space
S_n/\left( S_{n_1} \times S_{n_2} \dots \times S_{n_r} \right)
where the partition is X = \cup_{j=1}^r A_j with |A_j| = n_j. Here, the stabilizer of \pi is a subgroup of S_n which is isomorphic to the external direct product S_{n_1} \times S_{n_2} \dots \times S_{n_r}; in other words, the stabilizer is a Young subgroup, a kind of "internal direct product" of subgroups which are themselves symmetric groups.

If we are thinking of a function and we take \pi to be the partition of X into preimages, then the stabilizer consists of those permutations which respect the partition, i.e. don't map any point to a point lying in another preimage.

Here, the complexion is the coset space; that is, the orbit of \pi[/itex] under the induced action by S_n on the partitions of X. (This action can carry a given partition into any other partition with the same block sizes.) In Boltzmann&#039;s work the complexion of \pi is the set of microstates corresponding to a given macrostate (e.g. having a given energy value), and to get an &quot;subadditive&quot; measure of the &quot;variety&quot; of the sizes of the blocks, we use the logarithm of the size of the complexion as the Boltzmann entropy. Indeed, for finite complexions the logarithm of the sizes of the complexions is always a generalized Boltzmann entropy. As I already remarked, entropies are essentially dimensions, so we should expect that in another famously tractable case of group actions, finite dimensional Lie groups of diffeomorphisms, the dimension of the cosets (which are finite dimensional coset spaces) will behave as entropies, and they do.<br /> <br /> From this point of view, the Boltzmann entropy of a function (or if you prefer, of the partition into preimages induced by this function) measures the &quot;asymmetry&quot; of the partition. If you are familiar with Polya enumeration theory, you are already familiar with the idea that among geometric configurations consisting of k points in some finite space, the more symmetrical configurations have <i>smaller</i> orbits under the symmetry group, while the more asymmetric configurations have <i>larger</i> orbits. A good example is D_n acting on a necklace strung with n beads.<br /> <br /> Incidently, from the perspective of the theory of G-sets (sets equipped with an action by some specific group G; this category is analgous to the category of R-modules where R is some specific ring), the induced action on partitions is remarkable in that it satisfies an analogue of the primitive element theorem: every intersection of stabilizers of individual partitions, G_\pi, \, G_{\pi^\prime}, \dots, is the stabilizer of some partition. In general, it is certainly not true that every intersection of point stabilizers G_x, \, G_{x^\prime}, \dots is the stabilizer of some point! <br /> <br /> In general, given any action by some group G on some set X, there are many interesting &quot;induced actions&quot; one can consider, several of which have &quot;regularizing&quot; properties, in the sense of improving the behavior in some respect. In the induced action on partitions we altered the set being acted on, but we can also alter the group which is acting. For example, it is easy to define the <i>wreath product</i> of two actions (by G on X and by H on Y) and the result is an action by G \wr H on X \times Y, in which we take copies of Y indexed by X, thinking of the copies of Y as <i>fibers</i> sitting over the <i>base</i> X, and let copies of H independently permute the copies of Y and let G permute these fibers. <br /> <br /> In the case of finite permutation groups this gives a direct connection between Polya enumeration and complexions. Namely: the <i>pattern index</i> enumerates the conjugacy classes of pointwise stabilizers but forgets the lattice structure. For example, let us compare the natural permutation actions by the transitive permutation groups of degree five. Then C_5, \, D_5 both give pattern index 1,1,2,2,1,1 while F_{5:4}, \, A_5, \, S_5 give pattern index 1,1,1,1,1,1. These numbers correspond to the stabilizer lattice; for example the stabilizer lattice for C_2 \wr D_5 (now writing fiber first, as appropriate for right actions), acting in the wreath product action on the subsets of our 5-set, starts with G=C_2 \wr D_5 at the top, which covers a conjugacy class of five index ten subgroups (the stabilizers of the five points), which covers two classes of index four sugroups (two distinct types of five pairs of points), which each cover two conjugacy classes of index two subgroups (two distinct types of five triples of points), which each covers a single conjugacy class of index two subgroups (five quadruples of points), which covers a congacy class consisting of a unique index two subgroup (the trivial subgroup). Note that 10 \cdot 4 \cdot 2 \cdot 2 \cdot 2 = 320 = | C_2 \wr D_5 |. In more complicated cases, one really requires a Hasse graph to depict the stabfix lattice (modulo conjugacy). One thing I find useful is to attach to the edge from C down to C^\prime not only the stabilizer subgroup index, but also a symbol m/n indicating that each subgroup belonging to class C contains m subgroups belonging to class C^\prime, while each subgroup belonging to class C^\prime is contained in n subgroups belonging to class C. These integer ratios (not neccessarily in lowest terms!) together with the Hasse diagram describe the incidence relations among the fixsets.<br /> <br /> <blockquote data-attributes="" data-quote="Fra" data-source="post: 1431169" class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch"> <div class="bbCodeBlock-title"> Fra said: </div> <div class="bbCodeBlock-content"> <div class="bbCodeBlock-expandContent js-expandContent "> you thus get around the issue of defining probability directly in terms of measurements, instead one has to be able to infer _distinguishable states_ microstructure from input, or the constructs is still uncelar (ie any &quot;hidden&quot; microstructures are not acceptable). This is to the limit of my ignorance not trivial either, in particular as the complexity and memory sizes vary - THIS is the keys where I think many interesting things. Oddly enough, I think this is very interesting, even for an amateur. </div> </div> </blockquote><br /> I am not sure I see what you are getting at.<br /> <br /> <blockquote data-attributes="" data-quote="Fra" data-source="post: 1431169" class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch"> <div class="bbCodeBlock-title"> Fra said: </div> <div class="bbCodeBlock-content"> <div class="bbCodeBlock-expandContent js-expandContent "> Chris will probably get upset if this is not what he meant, so I explicitly declare that this does not necessarily have any relation to it. I still wear my ignorance with pride :) </div> </div> </blockquote><br /> Better to just say &quot;if I understand you correctly&quot; over and over, until we agree that you do understand correctly what I said. (This policy is symmetrical, of course.)<br /> <br /> <blockquote data-attributes="" data-quote="Fra" data-source="post: 1431169" class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch"> <div class="bbCodeBlock-title"> Fra said: </div> <div class="bbCodeBlock-content"> <div class="bbCodeBlock-expandContent js-expandContent "> But it is more or less what I referred to in post 15. I don&#039;t follow Chris all steps. He is a matematician, I am not. This alone explains the communication issues. </div> </div> </blockquote><br /> Well, my posts have only been sketches. There is a lot to say so if I tried to fill in all the background for a general audience and write out all the arguments, I&#039;d quickly have a book (my notes on this stuff are in fact more extensive than my notes on gtr).
 
Last edited:
  • #33
A quick comment
Chris Hillman said:
People sometimes seem to assume I am making the whole thing up! :rolleyes:

I want to add quickly that other misconceptions aside, in no way do I want you to think that I think that you were "making anything up" in any way - your posts tells me that you are very likely to be someone that knows a lot about the various formalisms relating to this and it was equally extremely likely that you had something interesting to say!

What I wasn't sure about though what your remarks had for bearing on my interests (speaking for myself, I do not speak for any other participants in this thread). Your terminology was unusual to me, but then you wrote yourself you are a matematician to training and inclination - and I am not (so I figure I am pretty ignorant relative to your position). I wasn't sure if you used matematical terms for something that I would label something else. Also I don't know you since before, except seeing your name adding sophisticated comments on GR in posts, it makes it harder to understand. You also appeared certain that I was wrong in any impressions, which was amazing since I never explained my application in detail, but it's in either case quite different from shannons problem from my viewpoint. But wether this difference is due to ignorance or something more substantial, is by construction impossible for me to know, or what's the difference? I figure everything may be due to my ignorance, which is also the very problem under examination (to me at least).

I tried to understand the meaning of your message, which is also why I asked about your opinions of QG, and what your interests were, so I could get an image of you(the sender), so as to better guess the meaning :smile:

/Fredrik
 
  • #34
Fra said:
I want to add quickly that other misconceptions aside, in no way do I want you to think that I think that you were "making anything up"

Sorry, I didn't mean you, or anyone participating in this thread (unless I have encountered someone before under a different "handle").

I don't understand the rest of your post.
 
  • #35
Chris Hillman said:
It will certainly help to extract the pure math.
...
My point is that none of this need have anything to do with physics.

Yes, this makes sense and your point it well taken. I guess I might ask, extract the pure math of what? :) Perhaps this explains in a nutshell my angle. I see many ways of coming up with mathematics, unless I know what how it relates to reality.

I'm sort of trying to apply this to what I percept as reality, I am trying to find a relation between reality and a mathematical formalism. My focus in all my comments is on this latter part and perhaps more importanly in the relation between mathematics and science that sort of also induces a reality to mathematics too. I admit that this touches not only physics but also philosophy of physics and scientific method.

Chris Hillman said:
Right, if you follow up my citation of the expository paper by Brian Hayes in American Scientist, and if you recall the fundamental orbit-stabilizer relation from elementary group theory (see any good book on group theory, for example Neumann, Stoy, and Thompson, Groups and Geometry), you should be able to see that the natural action by S_n on an n-set X induces an action on the set of partitions of X, and the size of the orbit of \pi is then
\frac{n!}{n_1! \, n_2! \dots n_r!}
while the orbit itself is the coset space
S_n/\left( S_{n_1} \times S_{n_2} \dots \times S_{n_r} \right)
where the partition is X = \cup_{j=1}^r A_j with |A_j| = n_j. Here, the stabilizer of \pi is a subgroup of S_n which is isomorphic to the external direct product S_{n_1} \times S_{n_2} \dots \times S_{n_r}; in other words, the stabilizer is a Young subgroup, a kind of "internal direct product" of subgroups which are themselves symmetric groups.

...

together with the Hasse diagram describe the incidence relations among the fixsets.

I roughly get what your saying and it all looks familiar, but I admit that due to other things that I do, I am currently rusty with the formal algebra, so I can not on top of my head give a response at your level here. You are describing in a stronger formal way things that what I understand really isn't that terribly complicated from a conceptual point of view.

From my point of view, "an energy partition" is quite fuzzy to start with, since what is energy? Unless the notion of energy is defined, the entire construct is compromised. I am trying to define all notions in terms of information at hand. Meaning that the part of history that has not dissipated from the observer memory is at hand.

I am trying to find a way to attach everything to principal feedback/experiment. This includes the spaces that the observables sit in. At this point however, the ideas are not mature enough to be explained in the exact way that I think you would want in order to understand what I am saying. I have been appealing to your intuitive understanding to communicate.

Chris Hillman said:
I am not sure I see what you are getting at.

Chris Hillman said:
Well, my posts have only been sketches. There is a lot to say so if I tried to fill in all the background for a general audience and write out all the arguments, I'd quickly have a book (my notes on this stuff are in fact more extensive than my notes on gtrc

I'll point out that I am not starting by looking among existent formalisms, I try to use physical intuition guided by my view of the scientific method to identify/invent/learn the formalism I need. This method in fact reflects my philosophy which I am at the same time testing.

I realize that you can't write a book on here, and you already supplied plenty of information so I don't ask for more. Also I think that if I were to define my application in your preferred language, I would have to quite some time to refresh my group theory skills and perhaps more importantly, make sure my own ideas mature, so I can describe them in the receivers preferred base.

I'll start by nothing that my starting point is more like the "microstate approach". But I'm looking for howto define the notion of distinguishable state. So far my best shot is to consider a boolean state to be the simplest possible observable. Either two states are distinguished, or they are not.

But at the same time, an observable is constrained in complexit by the observers memory. A light observer can no fit a complex observation in memory at once. This suggest a non-unitary evolution. I also try to keep relations all the way, so that the priors are induced in all higher constructs from first principles. This way there should never have to be a case of missing strategy.

I am also in this picture working on implementing mass in terms of information capacity. And try to deduce interial phenomena from the intertial phenomena already existing in information thinking, namely the mass of the prior, imposes a interia to ANY incoming information.

Anyway, I could expand this too... but I'm working on it and it's hard to convey something that is not mature. I figure that due to what I suspect your excellent expertise in GR as well as these information formalisms... you may already have some interesting reflections on this? I hope you note that the nature of my question is a philosophical one. To translate it into a strict mathematical problem, assumptions need to be made.

Many problems here...

Howdo we identify the microstructure of reality in a scientific way?
Is it strings? :)
what is it? and more importantly, how can we infer a guess in the spirit of "minimum speculation", thish relates also to the various maximum entropy principles and entropy dynamics that other people work on. It's related, but I find it's difficulty to find a satisfactory answer.

I want to find an induction principle (like Ariel Caticha) that by construction works by the minimum speculation principle, that generates a guide for betting. This togther with the unavoidable element of uncertainty I want to use to infere the laws of physics and probably also to guess the most likely simplest possible AND distinguishable mictrostructure.

I offer my apologees in advance in case this is message appears scrambled.

/Fredrik
 
Last edited:
  • #36
Fra said:
I want to find an induction principle (like Ariel Caticha) that by construction works by the minimum speculation principle, that generates a guide for betting. This togther with the unavoidable element of uncertainty I want to use to infere the laws of physics and probably also to guess the most likely simplest possible AND distinguishable mictrostructure.

An addiotnal thing that may or many not clarify what I tried to convey:

The obvious approach I've found is bayesian reasoning and bayes rule or the "analog of bayes approach" in any non-probabilistic approach but there is still some unsatisfactory things there... The choice of initial prior and howto infer the probability space it self, this leads to inductive principles to the probability space itself / alternatively the microstructure... so we get like an induction principle, not only for the "probability distribution" or state of the microstructure (am speaking loosely here), but also for the probabilti pace / microstructure itself. Giving a non-linear and quite complex feedback.

Chris, do you have any direct pointers for such specific application, assuming I made myself understood in the first place :)

I have initiated a construct on my own, but progress is slow.

The idea is that the simplest answer to the question what is the microstructure of reality is that we don't know. Then I question that answer, and want a better one. It seems an arbitrary guess is better than no guess - so someone may exlaim STRINGS (better than no answer). Then again, I question that answer and want a motivation for the guess. How can we formalise an inductive reasoning when the underlying mictrostructure is part of the unknown? Not only is the state of the mictrostructure unknown, the mictrostructure itself is unknown!

That is my problem, made short.

Any ideas relating to that, would be highly appreciated.

/Fredrik
 
Last edited:
  • #37
Addition to answer the estimated comments.

This induction suggest a nonlinear thing or an algorithm that can be approximated one level at a time by wildly assuming a mictrostructure of the landscape of the world of misctrostructures at the desired level. But what kind of memory capacity in natur eis needed to reflect this degree of sophistication? This is where I see an exploit to rule out overly complex things, by trying to assign information also to the framework, effectively giving also the framework "mass"/"info capacity".

This kind of reasoning seems to naturally lead to at least gravity like phenomena. This is why I figured this "problem" should be dead on to someone like you, except that perhaps it's too much potty philosophy for your taste? Comment?

/Fredrik
 
  • #38
Fra said:
I guess I might ask, extract the pure math of what? :) Perhaps this explains in a nutshell my angle. I see many ways of coming up with mathematics, unless I know what how it relates to reality.

I'm sort of trying to apply this to what I percept as reality, I am trying to find a relation between reality and a mathematical formalism.

Define "reality".

Fra said:
From my point of view, "an energy partition" is quite fuzzy to start with, since what is energy? Unless the notion of energy is defined, the entire construct is compromised

Maybe you misunderstood something?

Boltzmann entropies are a property of finite partitions; since any function induces a partition into preimages, we can associated a Boltzmann entropy with any function on a finite set. Neither the set nor the function need have any physical interpretation.

Fra said:
Meaning that the part of history that has not dissipated from the observer memory is at hand.

Assuming that you are indeed responding to what I wrote, none of what I said involves any notion of an "observer".

Fra said:
Also I think that if I were to define my application in your preferred language, I would have to quite some time to refresh my group theory skills

The undergraduate text by Neumann et al. and the undergraduate text by Cameron are more than enough background.

Fra said:
it's hard to convey something that is not mature...I offer my apologees in advance in case this is message appears scrambled.

None of your posts #35-37 make sense to me, so allowing your ideas to mature while you do some reading seems advisable.
 
  • #39
Hmmm Interesting, I guess this settles it for now at least.

I appreciate your feedback. It was interesting to see how remarkably poor my communication performed.

But thanks for the responses and your time! Even a misunderstood communication is feedback, I obviously need to work out my ideas in more detail. I guess I was completely mistaken that the problem I consider would be easily conveyed to someone with your background. But that wasn't so.

/Fredrik
 
  • #40
I suspect that you need to acquire much more background to refine your thinking, and also to understand how to explain your refined thinking to others. Your posts also seemed rather "manic" to me.
 
  • #41
To the others interested in this thread, please keep up the discussions. Don't let my communiciation to Chris halt this thread! I'd personally like to see more reflections on this topic in relation to physics.

It would be interesting to hear the views/thinking of everyone else, fuzzy or not. Just spit it out and see what happens :)

/Fredrik
 
  • #42
It might be interesting to analyze radio communications using information theory. A simple ping [morse code] model would probably be manageable.
 
  • #43
Been there, done that

Chronos said:
It might be interesting to analyze radio communications using information theory. A simple ping [morse code] model would probably be manageable.

Sigh.. I refer you to Shannon 1948 (see link in second paragraph of http://www.math.uni-hamburg.de/home/gunesch/Entropy/infcode.html). Isn't anyone out there planning to follow my advice and read this paper, which is by common consent one of the greatest scientific papers of all time? And also one of the funniest? I mean, what's not to love?
 
  • #44
I don't know what Chronos application is but if we are talking about say ordinary communication like TV, radio etc... Shannon indeed seems like a good and right thing to start with. This is not quite what I was into that's why I didn't comment.

I got a feeling that people dare not say anything in this thread now out of fear to seem like fools relative to Chris massive expertise :)

Shannons paper http://plan9.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf is 55 pages, and in the introduction it says...

Shannon - A Mathematical Theory of Communication said:
The recent development of various methods of modulation such as PCM and PPM which exchange bandwidth for signal-to-noise ratio has intensified the interest in a general theory of communication. A basis for such a theory is contained in the important papers of Nyquist1 and Hartley2 on this subject.

...

The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.

As far as I see it, this problem description seems to take for granted that the logical problem of defining the comparasion of the message sent by the sender and the message received by the receiver is solved. How would you make this comparasion, without communication in the first place, and where do you get that trusted communication channel from, and how is that selected?

The way I see it, this is a bit analogous to the problem of
1) how two different observers in different frames can compare vectors? There is obviously need for some kind of communication? And starting from no prior common references, how can this be done?
2) how does two different observers - starting out with no common references - in general compare their observations? they need to communicate indeed. But I think a secure communication is something emergent. From what I can see, this is not a problem Shannon is treating.

So it seems Shannon mainly has in mind what he calls the engineering problem, like TV, radio and the likes, in which case the trusted channels are easily selected.

/Fredrik
 
Last edited by a moderator:
  • #45
There is a certain amount of fear, and respect for Chris. I was hoping for a pedagogical review. The 'ping' model I proposed has already been done - as I am sure Chris knows. The links Chris gave, unfortunately, were mostly broken and the 'Shannon' link is in PS format - which is unfortunate for me. I read Shannon's paper about 20 years ago, but would not object to reading it again.
 
  • #46
The last link I supplied is a pdf, and it's the very same paper as the postscript file.

Here is also a simple quick&dirty online postscript to pdf converter:
http://www.ps2pdf.com/convert/convert.htm

/Fredrik
 

Similar threads

  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 15 ·
Replies
15
Views
5K
  • · Replies 105 ·
4
Replies
105
Views
15K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 2 ·
Replies
2
Views
5K
  • · Replies 21 ·
Replies
21
Views
5K
  • · Replies 11 ·
Replies
11
Views
4K
  • · Replies 50 ·
2
Replies
50
Views
10K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 10 ·
Replies
10
Views
3K