What is the true nature of randomness and its implications in finance?

[Descartz2000]

It seems to me that when 'randomness' is referred to, the explanation is always describing what 'random' behavior is not. I would equate 'random' with: no history, no dependence on intial conditions, no pattern, no predictable order. It only argues that 'random' behavior or outcomes are based on something other than what is listed above. So, my question is: what 'is' randomness, without descriptions of what it is not?

 

[Hurkyl]

In probability theory, a "random variable" is simply a probability measure on some space -- a function that, among other properties, assigns a real number from 0 to 1 to each "event" of the measure space.

(e.g. one might consider the discrete space {1,2,3,4,5,6}, and the measure P that assigns to each subset S the number |S|/6)



In complexity theory, the "Kolmogorov complexity" of a string is its shortest representation (according to some reasonable scheme). It turns out (I hope I state this properly) that for any computable property Q, the proportion of incompressible strings that have property Q is the essentially the same as the proportion of all strings that have that property.

In particular, this means incompressible strings have no "features" that make them stand out from other strings, and so this makes for a good measure of randomness.

Unfortunately, this system is only really applies in bulk -- it cannot be used in any reasonable way to talk about individual strings.

 

[junglebeast]

Randomness is simply a word that is defined in terms of a negative. There are many such words -- such as "unkind", "antisocial," "atypical"...the list goes on. By definition, these words are defined in terms of the lack of something else. So what?

Randomness exists mostly only in Utopian statistical theory. The only potential source of randomness so-far known to man is in quantum theory...so this could be used as a positive example of randomness, but it doesn't change the fact that it is impossible to re-define a word that is defined in terms of a negative in terms of positive(s).

 

[ideasrule]

It seems to me that when 'randomness' is referred to, the explanation is always describing what 'random' behavior is not. I would equate 'random' with: no history, no dependence on intial conditions, no pattern, no predictable order. It only argues that 'random' behavior or outcomes are based on something other than what is listed above. So, my question is: what 'is' randomness, without descriptions of what it is not?

Alright, but how about a system that behaves almost exactly like a system that has no history, no dependence on initial conditions, etc? Chaotic systems have such sensitive dependence on initial conditions that, for all practical purposes, they behave like a random system would.

 

[kote]

It seems to me that when 'randomness' is referred to, the explanation is always describing what 'random' behavior is not. I would equate 'random' with: no history, no dependence on intial conditions, no pattern, no predictable order. It only argues that 'random' behavior or outcomes are based on something other than what is listed above. So, my question is: what 'is' randomness, without descriptions of what it is not?

A random event has no deterministic cause - not just no cause that we know of, but no cause period that can exactly explain the effect.

 

[DaveC426913]

A random event has no deterministic cause - not just no cause that we know of, but no cause period that can exactly explain the effect.

How can such an event occur at all then?

 

[Descartz2000]

Alright, but how about a system that behaves almost exactly like a system that has no history, no dependence on initial conditions, etc? Chaotic systems have such sensitive dependence on initial conditions that, for all practical purposes, they behave like a random system would.

macro chaotic systems are deterministic in principle, just not in practice. But, this doesn't seem to be 'random' as it is defined here, as future behavior is based on the initial conditions (in chaos theory), it's just not predictable. I think true randomness can have no ties to initial conditions, if it did, and the behavior of the system is dependent on these initial conditions, then this seems to be the cause, and therefore the system becomes determined (in principle).

 

[JoeDawg]

How can such an event occur at all then?

If there was a 'how', it wouldn't be random.

 

[kote]

How can such an event occur at all then?

In addition to JoeDawgs response... who said random events occur at all? :)

Besides many (but not all) quantum physicists, of course.

 

[kote]

macro chaotic systems are deterministic in principle, just not in practice. But, this doesn't seem to be 'random' as it is defined here, as future behavior is based on the initial conditions (in chaos theory), it's just not predictable. I think true randomness can have no ties to initial conditions, if it did, and the behavior of the system is dependent on these initial conditions, then this seems to be the cause, and therefore the system becomes determined (in principle).

Correct... unless you consider that random events may be bound by initial conditions. Events with no definite cause may still be bound by preconditions, even if they are not determined by them. Such situations are proposed by some interpretations of quantum mechanics, for example.

 

[DaveC426913]

If there was a 'how', it wouldn't be random.

My point is, it could not occur in the first place. A penny cannot fall - randomly or otherwise - if it has not first been tossed.

 

[kote]

My point is, it could not occur in the first place. A penny cannot fall - randomly or otherwise - if it has not first been tossed.

It seems, then, that you believe in deterministic causation in nature. In this case, no event can be truly random. We can still have the word "random" and use it approximately in 2nd order macroscopic statements of cause and effect. We say that a coin flip is random, but it's understood that if we could have complete knowledge of the toss it would be, in principle, possible to determine the outcome ahead of time. This is the classical mechanized world view.

QM is more fuzzy about random events occurring at the basic physical causal level.

 

[DaveC426913]

It seems, then, that you believe in deterministic causation in nature. In this case, no event can be truly random. We can still have the word "random" and use it approximately in 2nd order macroscopic statements of cause and effect. We say that a coin flip is random, but it's understood that if we could have complete knowledge of the toss it would be, in principle, possible to determine the outcome ahead of time. This is the classical mechanized world view.

QM is more fuzzy about random events occurring at the basic physical causal level.

I believe that the reason for decay of a nucleus will eventually be explainable. If the decay or any other random subatomic event truly had no causal event, then what is to stop it from decaying into multiple hydrogen atoms, or into a pot of petunias? See, there are causal rules around decay; it happens virtually the same way every time. The fact that the time delay (of an individual nucleus) doesn't seem to follow a rule suggests simply that we don't know it yet.

 

[kote]

I believe that the reason for decay of a nucleus will eventually be explainable. If the decay or any other random subatomic event truly had no causal event, then what is to stop it from decaying into multiple hydrogen atoms, or into a pot of petunias? See, there are causal rules around decay; it happens virtually the same way every time. The fact that the time delay (of an individual nucleus) doesn't seem to follow a rule suggests simply that we don't know it yet.

At the quantum level we have absolutely no idea what forces are at play, and we can't predict what outcome will occur with any level of certainty at all. Arguments can surely be made for an assumption of determinism in nature... but it may be harder to get there than would initially be expected.

Niels Bohr, based on Kantian ideas of the primacy of classical properties, simply denied that there is any underlying level of determinism to be uncovered. David Bohm, assuming the primacy of deterministic natural laws, came to the conclusion that such laws exist but we can in principle never know what they are. According to Bohm, we can only reveal lower and lower levels of an infinitely recursive reality, but we can never explain anything in terms of the deepest level of reality.

 

[DaveC426913]

At the quantum level we have absolutely no idea what forces are at play, and we can't predict what outcome will occur with any level of certainty at all. Arguments can surely be made for an assumption of determinism in nature... but it may be harder to get there than would initially be expected.

Niels Bohr, based on Kantian ideas of the primacy of classical properties, simply denied that there is any underlying level of determinism to be uncovered. David Bohm, assuming the primacy of deterministic natural laws, came to the conclusion that such laws exist but we can in principle never know what they are. According to Bohm, we can only reveal lower and lower levels of an infinitely recursive reality, but we can never explain anything in terms of the deepest level of reality.

Agreed. But that is not tantamount to saying there is no causitive agent.

 

[JoeDawg]

If the decay or any other random subatomic event truly had no causal event, then what is to stop it from decaying into multiple hydrogen atoms, or into a pot of petunias?

This is a standard http://en.wikipedia.org/wiki/Argument_from_ignorance" If its random, then yes, it could result in a pot of petunias, but that's really not just one random event, its quite a few. The hydrogen thing involves fewer.

So yes, there could be a pot of petunias floating somewhere around Orion's belt. But there is no frequency to random events, and the universe is big, so the fact you haven't seen it happen doesn't mean much.

All you've said here is you don't believe in random events.

 

[octelcogopod]

Well, how about this.
Determinism means a causal event chain. A chain that develops over several stages over a period of time.
A random event would be an event that has no event chain.
This would essentially mean it happened instantaneously and that there is no causal process for why it happened.

Using this definition, is there really any way to understand such an event?
I would say no.
I'm not going to try to say that I have all the answers, but it does seem odd to me if such events exist in the universe, either in quantum physics or any other macro/micro level.
The only way I can see it looking random is if something is injected into our universe from another dimension, or similar, because then the process would not have started in OUR universe.
But this is of course speculation to the extreme. Just saying.

 

[DaveC426913]

The only way I can see it looking random is if something is injected into our universe from another dimension, or similar, because then the process would not have started in OUR universe.
But this is of course speculation to the extreme. Just saying.

Or how about less extreme: simply a causal event occurring in a dimension we do not have access to - a la the higher 4 dimensions of string theory.

 

[Descartz2000]

I believe that the reason for decay of a nucleus will eventually be explainable. If the decay or any other random subatomic event truly had no causal event, then what is to stop it from decaying into multiple hydrogen atoms, or into a pot of petunias? See, there are causal rules around decay; it happens virtually the same way every time. The fact that the time delay (of an individual nucleus) doesn't seem to follow a rule suggests simply that we don't know it yet.

I agree. It seems without a cause to determine (in principle) the outcome, then the outcome can be boundless and without order or structure. The fact that the decay of a nucleus occurs the same in multiple events leads one to the notion that there is a form and structure to the decay and this is essentially what determines it (in principle). However, the cause may never be known. Kind of like Chaos theory. If we go far enough down the causal chain we lose sight of the initial conditions, we know the result is caused by them, but the interactions that brought the result is too complicated to understand or it is simply hidden from our knowledge.

 

[SW VandeCarr]

I believe that the reason for decay of a nucleus will eventually be explainable. If the decay or any other random subatomic event truly had no causal event, then what is to stop it from decaying into multiple hydrogen atoms, or into a pot of petunias? See, there are causal rules around decay; it happens virtually the same way every time. The fact that the time delay (of an individual nucleus) doesn't seem to follow a rule suggests simply that we don't know it yet.

It would seem that to predict just which atom will decay and when it will decay would assume that each atomic nucleus of a particular isotope is distinctive in some way. This would imply some hidden variable(s) involving nuclear properties. Does anyone have any idea what properties these might be? For example, are there different kinds or combinations of gluons? I think we know the quark structure of nucleons, and afaik, it is the same for all nucleons.

 

[SW VandeCarr]

My point is, it could not occur in the first place. A penny cannot fall - randomly or otherwise - if it has not first been tossed.

It has long been thought that quantum fluctuations at the Planck scale are indeed random and without cause, as the term is usually understood: a necessary and sufficient agent. Presently, the "new determinism" would assume there is a cause for QF. However, unless and until there is a real theory re hidden variables, I don't think we can just assert that all events require a cause.

 

[kote]

I has long been thought that quantum fluctuations at the Planck scale are indeed random and without cause, as the term is usually understood: a necessary and sufficient agent. Presently, the "new determinism" would assume there is a cause for QF. However, unless and until there is a real theory re hidden variables, I don't think we can just assert that all events require a cause.

Ah, how quickly science gave up on the idea that its purpose was to uncover causal natural laws...

"I can't figure out why I'm getting these results!"
"Screw it. It can't be us. Nature's got to be broken. My theory describes not only everything that could be known, but everything there is to know. Trust me. I'm that awesome."

Edit: I'm not disagreeing with you, and I've personally argued for the standard interpretation of QM from the elevated semantic status I think it suggests we give epistemology. I just find the history of the issue to be pretty hilarious in light of the mechanized world view and all previous definitions of what it meant to be real.

 

[kote]

I don't think we can just assert that all events require a cause.

On another note... this very assertion has been made (and required) by all science ever performed. Science has been the search for the underlying cause. Stopping at anyone level and saying "it's complete, nothing else to see here," stops all scientific exploration. The Catholics made the claim that their explanatory framework was complete and look at the wonders that did for science in the years following Christianity's rise to power.

Unless a theory is deterministic and consistent with all experiments performed, any claims of completeness with regard to the deepest level of reality can only be explained as attempts by the theory's proponents to discredit opposing viewpoints and maintain their own social power.

(Please note that Niels Bohr explicitly claimed that QM was not complete with regard to the real essence of the manifold in his book published in 1934. He claimed that QM was only complete with regard to aspects of our experience, which he held as prime based on a Kantian view of space-time categories. Claims to the contrary have been inconsistent with Bohr's views.)

 

[apeiron]

QM is more fuzzy about random events occurring at the basic physical causal level.

The quantum zeno effect would seem to be a good argument that QM indeterminism is actually a similar story to the randomness of coin tossing.

http://en.wikipedia.org/wiki/Quantum_Zeno_effect

With a "random" event, we are really talking about a global level set of constraints which then leave particular localised outcomes "free". The outcomes are not completely free (no petunias), just free in terms of the global constraints and the degree of control they exert (or rather, do not exert).

So with a coin toss, we create a binary object. Toss a ball and the outcome will always be vague. The question "which side up?" has no meaning. But a coin is geometry constrained so outcomes will be crisply definite.

Then we also have to constrain the action of the toss. We must toss it so high and fast that we quite clearly are not controlling its flight.

So we can see how we are manufacturing randomness by constraining an event of many degrees of freedom - but then with equal global "determination" are making a binary outcome look as much like a free or chance choice as possible.

Now people can get worried about the micro-level causality. If we knew every Newtonian detail, we could predict heads or tails. But then this would go against the original global constraints spirit of the action. Where what we were attempting to produce was a random toss, not a controlled toss.

Globally, of course, we would make the coin land head or tails by increasing the constraints around the process. We would toss the coin very slowly with care. Or just place it. And any process of measuring the coin toss so as to predict the outcome would also be a constraints-based tightening of the rules around the process.

We would not be proving micro-causality IS deterministic, just that causality can be DETERMINED to a very fine degree by global or macro-causality.

With coin tosses, at a Newtonian level of discussion, this all seems obvious. The two troubling areas are of course QM and chaos theory.

With QM, the quantum zeno effect shows I feel that global constraints decohere local potentials. So it is not the location that self-determines in a random fashion to decay or whatever. Rather it is the context that shapes the space of what is possible. And nature can be set up so that the QM "coin-toss" is more generally constrained (the decay seems spontaneous and follows a powerlaw - probability is unaffected by passing time). Or it can be more specifically constrained as in the zeno effect.

It would be interesting if others viewed the situation differently.

With deterministic chaos modelling, the paradox lies in the idea that completely determined equations can produce random looking outcomes. A lot of micro-scale definiteness can produce macro-level confusion.

But again, equations are still global level processes. And the kind of equations that work are the ones that can generate the equivalent of a coin-flip of uncertainty over all scales. There is a grain of "error" built into the process to get things started (well, not error but some particular set of initial condions so exact that we in practice could never measure reality with the infinite precision required to obtain the actual value). And then this "error" is forced - via feedback loops usually - to propagate symmetrically across all available scales.

Anyway, the key to randomness is not to puzzle over an apparent lack of causality at some location, controlling some event, but to look upwards in scale to the nature of the global constraints that obtain. Randomness does not exist in some naked, basic and fundamental way in nature. It only occurs as a feature - a local freedom - within some larger system of constraint. Or rather as a result of what some larger system is not managing to constrain either through choice (human coin tosses) or inability (QM tunnelling, initial conditions measurements).

 

[octelcogopod]

Randomness does not exist in some naked, basic and fundamental way in nature. It only occurs as a feature - a local freedom - within some larger system of constraint. Or rather as a result of what some larger system is not managing to constrain either through choice (human coin tosses) or inability (QM tunnelling, initial conditions measurements).

I'm not sure what you mean to be honest.. The randomness of a coin toss is a completely subjective assessment due to our inability to understand all the variables involved.
In the actual reality there is no randomness or freedom for the coin.. I don't understand what you mean by "local freedom?"
Local freedom in what? In nature?

 

[apeiron]

In the actual reality there is no randomness or freedom for the coin..

Well you are assuming this because you believe that Newtonian determinism is an ontological truth rather than an epistemic convenience.

The point about the coin toss example lies in the obvious care with which humans have to go about things to create something that behaves randomly. To create a global process which produces locally uncontrolled events.

Indeed, any study of the algorithms and mechanisms needed to create perfect random number generators will show just how difficult this kind of task is.

So "in actuality in nature" what I would be arguing is that there is neither determinism or randomness (or freedom) in some pure sense. But systems or processes can be set up so they tend towards either dichotomous extreme. You can prepare a coin toss or routlette wheel so it produce events (something that happens at a spacetime locale) in either a very random looking, or alternatively very determined, fashion.

With a coin toss, we can actually push the process in either of these directions - by either dropping the coin carefully or spinning it wildly.

So in nature, there is us creating the process and the results are dependent on which way we are tuning things. We have in our minds some goal - to be determined or to be random. But neither of these things is actually in nature in that direct way.

Quantum zeno shows how even QM is entangled with the observer (or global decohering context). There is no local property that just is a this or a that in naive fashion.

 

[Descartz2000]

It seems to me that it is difficult to find 'true randomness" (no history, no dependence on previous states). First of all, the implications of this are hard to accept. Basically, this implies certain events are acausal. How can any action occur without a cause? Accepting the notion that the properties involved in the event must have free will, or accepting arguments for a mystical variable can not explain the action. So instead we accept a highly constrained randomness as an explanation. The only problem is that it starts to lose it's true random qualities when we see patterns develop. It should make us ask, "Does true randomness even exist?"

 

[apeiron]

. It should make us ask, "Does true randomness even exist?"

Again, you are not addressing the question completely unless you also specify "at scale x".

So do you mean localised randomness and "acausality" (which you clearly do, speaking of events)? Or do you mean global scale randomness - randomness of systems themselves?

True and naked randomness would not exist at any level of scale, I would argue. But global constraints can determine local events in ways that make some aspect of their behaviour "free" - as random as a coin flip or radioactive decay.

 

[SW VandeCarr]

"Does true randomness even exist?"

Who knows? There's a lot of misunderstanding re randomness. First, there's a difference between statistical tests for randomness and random number generators (RNG) as well as the question as to how well they work. Finally there's the (I believe) metaphysical question as to whether "true" randomness exists. The only thing we can say (I believe) is that randomness reflects our uncertainty if nothing else.

The decimal expansion of pi satisfies statistical tests for uniform randomness (each digit is equally probable for the kth term), but the pi sequence is not random. It's generated by one of several algorithms for which the kth term is always determined.

An RNG can be realistically evaluated by generating a sequence of k terms such that for n trials the number of repeats of a sequence does not exceed expectations. For a digit sequence of k=10, given uniform randomness and n=10^k, the expectation is that there will be just one repeat sequence in 10^10 trials.

This is all we can say about RNGs. Whether some physical processes are "truly" random is a metaphysical question unless and until we have a testable TOE.

 

[Senex01]

I've always understood "random" as a statement that we do not know the causes of an event. Or more precisely, the aspects of the event which are "random" are the ones which we cannot predict from its initial set-up or that we cannot find explanatory causes for afterwards. Therefore, it is not so much a description of the event itself, but of our knowledge of it.

Of course our lack of knowledge could be attributed to the fact that the cause is unknowable in principle (some interpretations of quantum mechanics), or to a practical limitation in making the measurements (a coin-toss).

The decay or otherwise a particle is only "random" in certain respects: we know what kinds of decay are possible under certain conditions, and we know the circumstances under which they would decay. The heads/tails aspect of a coin-toss is "random", but there is nothing "random" about its acceleration. (for example...)