PeterDonis said:
These statements only apply to the entire universe. They do not apply to tiny subsystems of the entire universe.
Yes, agreed. Which leads us to the topic: what "paradigm" does make sense for an agent, the Newtonian paradigm does not? (this is what I have been talking about)
By paradigm I mean choice of abstractions or mathematical methods for representing theoryspace, states and changes.(Examples of "paradigms" are say ODE/PDE, and boundary/intiial value problems. Another paradigm is ABM Agents based modelling. Often the same phenomenan can be described in BOTH paradigms, but perhaps sometimes one of them makes the logical clearer.)
As Smolin argues in several papers and books, this paradigm does make sense and have extreme power, but mainly when you study small subsystems at a short timescale. There is no need to deny this, this possibly even the best way to do it in this case. But open problems in physics, such as trying to unify interactions at very different energy ranges perhaps? could be easier in another paradigm. Finetuning and renormalisation problems are IMO symptoms related to trying to use this successful paradigm outside it's corroborated domain of fitness.
I totally agree that all this is fuzzy, but i wouldn't expect any of this to be easy to express or discuss. Step 1, is to for a second to at least be aware of that most of current physics are forged from this paradigm, and ask if it is good?
From the perspective of defining the "initial information" in these, seen as learning, both specifying initial conditiions, state space structure and the dynamical laws, implies prior information in the wider sense.
PeterDonis said:
Information can get transferred between subsystems even if, over the entire universe, the total information is constant.
A conservative mainstream answer then is that the agent (seen as a subsystem of a closed system) is described in principle by taking the master description of the whole universe and simply averaging out the environment. But this paradigm rests on the same abstractions as for the whole universe, which you then just "reduce". This "solution", is the problem as it contains a finetuning problem. Ie. the "explanation" requires alot of finetuning. In other words, the "explanation" by construction requires an a priori improbable premises.
The more pragmatic conservative approach is to simply consider the theory for the subsystem as an effective theory. This is of course totally fine and it's the standard view I would say, but how can we understand the emergence and relations of the difference theories, and most important, how can be guess the interaction of two agents that each encode two different effective theories - if we do not understand the emergent theories in a deeper way? The standard view is still that we can via various renormalisations ans phase transtiions infer an effective low energy theory given that the ultimate high energy theory is known. But what about the reverse? And the othre problem is that knowing the high energy theory again is too complex to encode an agent. If we need to encode all this in som gods view or external "superagent" - how does that help us understand the unification and emergence of interactions?
Note: discussing the measures of "information", "entropy" etc is interesting, but that i fear can be it's own discussion. I admit that there are multiple cans of worms here, but opening them all at ones gets messy. Just a short note: I think the shannon type of "information" is for reasons like above, not a satisfactory measure; as it relies on a fixed background structure (classical bits for example). I think the measure of information that an agent itself can define, withouyt referencing external references, is necessarily relative somehow, I'll stop there.
Edit: A note here that is also revealing the two perspectives. do we abstract the "Agent/Observer" as a subsystem of the close universe? Or do we abstract hte agent from the inside as something, learning and struggling in an unknown environmment? This differences in perspectives makes a big difference. Compare the task of riemann geometry to define curvatures in terms of intrinsic curvatures only. Rather than imagining the curved surface embedded in an flat enviroment. The perspective with intrinsic vs extrinsic information theory is the same here. The conventional theory is extrinsic and explains the agent by seeing it from a bigger embedding. This is just as insatisfactory as describing curvature of a surface in terms of extrinsic (intrisically non-measuralbe) quantitites. This is not a perfect analogy, but it illustrates the point. All the "measures" and "measurements" of the agent must be defined in terms of intrisiccally available things, in order to avoid blind finetuning of external parameters.
/Fredrik