- #1
Jarek 31
- 158
- 31
- TL;DR Summary
- local vs global entropy maximization?
To choose random walk on a graph, it seems natural to to assume that the walker jumps using each possible edge with the same probability (1/degree) - such GRW (generic random walk) maximizes entropy locally (for each step).
Discretizing continuous space and taking infinitesimal limit we get various used diffusion models this way.
However, looking at mean entropy production: averaged over stationary probability distribution of nodes, its maximization leads to usually a bit different MERW: https://en.wikipedia.org/wiki/Maximal_entropy_random_walk
It brings a crucial question which philosophy should we choose for various applications - I would like to discuss.
GRW
- uses "local" approximation of (Jaynes) https://en.wikipedia.org/wiki/Principle_of_maximum_entropy
- has no localization property (nearly uniform stationary probability distribution),
- has characteristic length of one step - this way e.g. depends on chosen discretization of a continuous system.
MERW
- is the one maximizing mean entropy, "most random among random walks",
- has strong localization property ( https://en.wikipedia.org/wiki/Anderson_localization ) - stationary probability distribution exactly as quantum ground state,
- is limit of characteristic step to infinity - is discretization independent.
Simulator of both for electron conductance: https://demonstrations.wolfram.com/ElectronConductanceModelsUsingMaximalEntropyRandomWalks/
Diagram with example of evolution and stationary denstity, also some formulas (MERW uses dominant eigenvalue):
Discretizing continuous space and taking infinitesimal limit we get various used diffusion models this way.
However, looking at mean entropy production: averaged over stationary probability distribution of nodes, its maximization leads to usually a bit different MERW: https://en.wikipedia.org/wiki/Maximal_entropy_random_walk
It brings a crucial question which philosophy should we choose for various applications - I would like to discuss.
GRW
- uses "local" approximation of (Jaynes) https://en.wikipedia.org/wiki/Principle_of_maximum_entropy
- has no localization property (nearly uniform stationary probability distribution),
- has characteristic length of one step - this way e.g. depends on chosen discretization of a continuous system.
MERW
- is the one maximizing mean entropy, "most random among random walks",
- has strong localization property ( https://en.wikipedia.org/wiki/Anderson_localization ) - stationary probability distribution exactly as quantum ground state,
- is limit of characteristic step to infinity - is discretization independent.
Simulator of both for electron conductance: https://demonstrations.wolfram.com/ElectronConductanceModelsUsingMaximalEntropyRandomWalks/
Diagram with example of evolution and stationary denstity, also some formulas (MERW uses dominant eigenvalue):
Last edited by a moderator: