What is the true nature of randomness and its implications in finance?

[wofsy]

It seems to me that when 'randomness' is referred to, the explanation is always describing what 'random' behavior is not. I would equate 'random' with: no history, no dependence on intial conditions, no pattern, no predictable order. It only argues that 'random' behavior or outcomes are based on something other than what is listed above. So, my question is: what 'is' randomness, without descriptions of what it is not?

Randomness has rigorous mathematical models as Hurky says. It is not defined by what it is not. It is defined by mathematical laws. A Brownian motion is completely well defined yet it is entirely unpredictable.

in Nature randomness is an intrinsic feature of Quantum Mechanics. Again this randomness is defined rigorously and there is even a differential equation for how the randomness evolves in time.

In a perfectly efficient securities market where all information is known equally to all market participants and where information is absorbed correctly and instantaneously, the sequence of prices are entirely random. It may not be a Brownian motion but it is a Martingale i.e. a process that can not be predicted. Brownian motion is one example of a Martingale. While markets are not perfectly efficient they are remarkably close.

The idea of an efficient market demonstrates how randomness can occur in Nature. If all information available is reflected in a security's price then its expected future price must be its current price. New information will change the price but the price can not change until this information is revealed. So the efficient market must be a Martingale. even if you knew when new information would be revealed this would not change things. And if you knew the new information already, then it will already be incorporated into the price.

Lastly, I think think the idea of cause or no cause can be irrelevant and confusing. For instance, a random process can be perfectly causal. In the efficient market, the "cause" of a price is the market's assessment of all available information. this is a rigorous cause yet the series is still a Martingale.

second, to say that something has no cause is somewhat meaningless since all things that can be explained have a cause in some sense. And even if you did know the supposed cause you can never know for sure what will happen. Laplace, who knew Newtons Law of gravititation well, still posted daily odds that he was willing to give on whether the Sun would rise the next day or not. Each day that the Sun rose he increased his odds.

 

[Chronos]

Randomness is described and measured by the Poisson function.

 

[apeiron]

Modelling events as locally independent is the easy bit. You assume independence by definition. The philosophical problem arises when your ontology demands that all events are actually, even if secretly, dependent.

So then you have to dig into the notion of dependent relations - causality.

This is when it becomes valuable to recognise two kinds of dependent action. Local acts of construction and global actions of constraint.

Local acts of construction cannot produce independent events. But global constraints can.

It may sound paradoxical until you've thought it through, but independent events depend on the existence of their crisp global constraints.

 

[wofsy]

Modelling events as locally independent is the easy bit. You assume independence by definition. The philosophical problem arises when your ontology demands that all events are actually, even if secretly, dependent.

So then you have to dig into the notion of dependent relations - causality.

This is when it becomes valuable to recognise two kinds of dependent action. Local acts of construction and global actions of constraint.

Local acts of construction cannot produce independent events. But global constraints can.

It may sound paradoxical until you've thought it through, but independent events depend on the existence of their crisp global constraints.

So give me an example.

 

[wofsy]

Or how about less extreme: simply a causal event occurring in a dimension we do not have access to - a la the higher 4 dimensions of string theory.

why would something caused from another dimension be random?

 

[wofsy]

Why aren't all events random? causes are never observable or experienced. they are just explanations. One never sees a cause. We live in a fantasy world where we think just because our minds are able to organize experience that there are actual causes outside of our minds.

Isn't each day's sunrise random, unconnected to anything except itself?

 

[apeiron]

So give me an example.

See post 24 on things like quantum zeno and coin tossing.

 

[wofsy]

See post 24 on things like quantum zeno and coin tossing.

I still don't get it. the global constraint in a coin toss does not at all tell you that the outcomes will be random.

Furthermore constraints eliminate randomness. So why don't constraints actually create determinism?

 

[apeiron]

I still don't get it. the global constraint in a coin toss does not at all tell you that the outcomes will be random.

In what way does the carefully determined circumstances of a coin toss not produce a random outcome?

Oh, you mean that you want to know whether the randomness is "real". Whether it "exists" and is not merely a product of arranged circumstances.

The global system of constraint may not know the particular outcome, but some other imaginary observer could do it differently and track the physics of the spinning all along the way with sufficient accuracy.

Furthermore constraints eliminate randomness. So why don't constraints actually create determinism?

Constraints eliminate degrees of freedom. They suppress what can happen at a locale. But then any degrees of freedom that are not eliminated, by definition, will be expressed. And the more degrees removed, the more visible - meaningful or informational in terms of the system set-up - will be those remaining free actions.

So as I say, toss a ball and the act is so unconstrained that the side it lands on seems pretty meaningless. Toss a coin and now its choice of landings is so constrained that up-down becomes a remarkably arbitrary-looking freedom. It becomes an "event" rather than the non-event of a ball toss. And it becomes "random" in being predictably unpredictable. Where again, with a ball toss, which "way up" the ball falls is so unremarkable as to be vague - neither here nor there.

You can agonise forever whether local events that are the product of constraints are really determined or really free. The question is meaningful only in relation to the context, the system, that is producing those events.

You can insert a laplacean demon into the scenario to try to clarify matters. But positing a micro-observer to bypass the macro-context is not going to answer your ontic question (though it is a useful way to explore the limits of an information theoretic modelling approach).

 

[wofsy]

In what way does the carefully determined circumstances of a coin toss not produce a random outcome?

Oh, you mean that you want to know whether the randomness is "real". Whether it "exists" and is not merely a product of arranged circumstances.

The global system of constraint may not know the particular outcome, but some other imaginary observer could do it differently and track the physics of the spinning all along the way with sufficient accuracy.



Constraints eliminate degrees of freedom. They suppress what can happen at a locale. But then any degrees of freedom that are not eliminated, by definition, will be expressed. And the more degrees removed, the more visible - meaningful or informational in terms of the system set-up - will be those remaining free actions.

So as I say, toss a ball and the act is so unconstrained that the side it lands on seems pretty meaningless. Toss a coin and now its choice of landings is so constrained that up-down becomes a remarkably arbitrary-looking freedom. It becomes an "event" rather than the non-event of a ball toss. And it becomes "random" in being predictably unpredictable. Where again, with a ball toss, which "way up" the ball falls is so unremarkable as to be vague - neither here nor there.

You can agonise forever whether local events that are the product of constraints are really determined or really free. The question is meaningful only in relation to the context, the system, that is producing those events.

You can insert a laplacean demon into the scenario to try to clarify matters. But positing a micro-observer to bypass the macro-context is not going to answer your ontic question (though it is a useful way to explore the limits of an information theoretic modelling approach).

I see what you are saying. I was just asking why constraining the coin to a two sided toss should produce randomness. It doesn't follow in my mind. Why couldn't it be entirely deterministic?

 

[wofsy]

In what way does the carefully determined circumstances of a coin toss not produce a random outcome?

Oh, you mean that you want to know whether the randomness is "real". Whether it "exists" and is not merely a product of arranged circumstances.

The global system of constraint may not know the particular outcome, but some other imaginary observer could do it differently and track the physics of the spinning all along the way with sufficient accuracy.



Constraints eliminate degrees of freedom. They suppress what can happen at a locale. But then any degrees of freedom that are not eliminated, by definition, will be expressed. And the more degrees removed, the more visible - meaningful or informational in terms of the system set-up - will be those remaining free actions.

So as I say, toss a ball and the act is so unconstrained that the side it lands on seems pretty meaningless. Toss a coin and now its choice of landings is so constrained that up-down becomes a remarkably arbitrary-looking freedom. It becomes an "event" rather than the non-event of a ball toss. And it becomes "random" in being predictably unpredictable. Where again, with a ball toss, which "way up" the ball falls is so unremarkable as to be vague - neither here nor there.

You can agonise forever whether local events that are the product of constraints are really determined or really free. The question is meaningful only in relation to the context, the system, that is producing those events.

You can insert a laplacean demon into the scenario to try to clarify matters. But positing a micro-observer to bypass the macro-context is not going to answer your ontic question (though it is a useful way to explore the limits of an information theoretic modelling approach).

Also it seems to me that what part of the ball hits the ground is also random - it's distribution is uniform across its entire surface. The coin toss is less random since it can only have 2 outcomes.

 

[apeiron]

You don't see what I'm saying as you insist on asking which local view is "the real" - the random event or the determined event.

The point is that it is the dynamics of the system that create this difference. It is the choice of the system that is real. Random and determined would be two ends of a spectrum of choices the system could make. The complementary limits.

So a coin toss could be asymptotically random or asymptotically determined. You could even toss a coin as if you didn't care, then record the toss with super-sensitive instruments that in fact from initial conditions and Newtonian mechanics - and all sorts of constraints over the environment which are the standard gear of experimentation - could give a deterministic account of why a particular toss came up heads or tails.

You can then argue endlessly about which is the real picture of causality at the local scale. But you can see that the causality is in fact very different at the global scale. The flip-flop from don't care to do care.

In physics, it is normal to discard global facts. But in philosophy, they are what you need to remember about the "truth" of the total situation.

A coin toss is of course just a way into these thorny issues. Then you can go on to talk about QM and chaos - areas where nature itself is the system.

 

[apeiron]

Also it seems to me that what part of the ball hits the ground is also random - it's distribution is uniform across its entire surface. The coin toss is less random since it can only have 2 outcomes.

Not sure that I can simplify matters for you much further.

Imagine a perfect sphere landing on a perfect plane and resting on only a single point. You can see there would indeed be an infinity of "sides". Outcomes don't come less unconstrained and therefore less distinctive. More vague and less of a crisp event.

A binary choice is then the opposite, the most distinctive and constrained form of choice. Hence why all information theory boils down to the crisp black and white of binary codes.

Perhaps you should offer us your personal definition of random.

A lot of people talk about it as meaning uncaused events. I never said anything about a lack of causality. Instead I was talking about where to look to find the missing causality.

 

[wofsy]

So as I say, toss a ball and the act is so unconstrained that the side it lands on seems pretty meaningless. Toss a coin and now its choice of landings is so constrained that up-down becomes a remarkably arbitrary-looking freedom. It becomes an "event" rather than the non-event of a ball toss. And it becomes "random" in being predictably unpredictable. Where again, with a ball toss, which "way up" the ball falls is so unremarkable as to be vague - neither here nor there.

this just seems wrong. Where the ball lands is not vague. The ball toss is not a non-event and it is no less random than the coin toss. perhaps you should tell me what you mean by random.

 

[apeiron]

this just seems wrong. Where the ball lands is not vague. The ball toss is not a non-event and it is no less random than the coin toss. perhaps you should tell me what you mean by random.

No, after you. Why not start by better explaining why efficient markets would actually produce random prices, the example you introduced.

 

[wofsy]

No, after you. Why not start by better explaining why efficient markets would actually produce random prices, the example you introduced.

i actually thought I gave a good explanation. It is not original to me. It is standard. I could go into the mathematics more if you want. I would be glad to do that. Let me know.

 

[apeiron]

i actually thought I gave a good explanation. It is not original to me. It is standard. I could go into the mathematics more if you want. I would be glad to do that. Let me know.

I'm familiar with the maths and also the literature arguing other models like fractals may be better.

What we are discussing here is the why behind the what. So why do actors with perfect information result in random statistics? I'll give you a clue. It is about local independent choice permitted within global economic contexts - markets, indeed. But I'll let you give your view first.

 

[wofsy]

I'm familiar with the maths and also the literature arguing other models like fractals may be better.

What we are discussing here is the why behind the what. So why do actors with perfect information result in random statistics? I'll give you a clue. It is about local independent choice permitted within global economic contexts - markets, indeed. But I'll let you give your view first.

Your language is difficult for me to penetrate. And because of this I am not sure what to say.

Certainly in the QM course I took, randomness was considered intrinsic to measurement. The whole theory seems based on it. I read your Zeno effect reference and this does not seem to change this. It would help me if you could elaborate on this article to say why it does.

As far as markets go, it is simply that information is processed completely and incorporated into prices. the price can not change until new information arises. Since new information is unknowable today - otherwise it wouldn't be new - the market price must be a Martingale. this fact does not require any model of how new information appears. This idea has nothing to do with the ideas of fractal or chaos. And most market mathematicians that I know do not believe that the market exhibits chaos.

 

[apeiron]

As far as markets go, it is simply that information is processed completely and incorporated into prices. the price can not change until new information arises. Since new information is unknowable today - otherwise it wouldn't be new - the market price must be a Martingale. this fact does not require any model of how new information appears. This idea has nothing to do with the ideas of fractal or chaos. And most market mathematicians that I know do not believe that the market exhibits chaos.

Yes, the superficial explanation is obvious. But it is the deeper commonalities we were discussing.

So a requirement here is a collection of independent traders. And this in turn requires a context of constraints that can produce such crisp local independence.

Although the machinery of markets - the necessary infrastructure - being human-made rather disguises this fact. It is constructed like all machines. So you would have to look beyond that to the sociology that creates the blueprint for the market machinery.

You have to remember that models are convenient fictions. They leave out unnecessary details as much as possible. But when you want to step back to the grander philosophical view, you want to think about exactly what facts have been suppressed.

 

[wofsy]

Yes, the superficial explanation is obvious. But it is the deeper commonalities we were discussing.

So a requirement here is a collection of independent traders. And this in turn requires a context of constraints that can produce such crisp local independence.

Although the machinery of markets - the necessary infrastructure - being human-made rather disguises this fact. It is constructed like all machines. So you would have to look beyond that to the sociology that creates the blueprint for the market machinery.

You have to remember that models are convenient fictions. They leave out unnecessary details as much as possible. But when you want to step back to the grander philosophical view, you want to think about exactly what facts have been suppressed.

the traders do not have to be independent, They only need to incorpaorate information efficiently and quickly. There is no assumption whatsoever of independence. This is not a convenient fiction.

 

[apeiron]

Rubbish. Completely different statistics would result if traders were highly correlated in their actions.

The very fact that all the traders are forced to make decisions at the same moment means there is no time for the communication that would create long scale correlations. The speed of trading is what isolates them. Disappointing that you are not thinking any of this through for yourself.

Check out Wisdom of the Crowds for a pop primer on these kinds of issues. Or Strogatz's Sync.

You've got to get in behind the textbook models to discover what the models presume (what is implicit rather than explicitly represented).

 

[wofsy]

Traders are highly correlated - I was a trader myself. But I think this discussion is fruitless. It has no rigor and I can not sink my teeth into anything that you are saying. I am not smart enough to figure out even the first thing that you are trying to say. I am sorry. I really tried.

 

[apeiron]

Traders are highly correlated - I was a trader myself. But I think this discussion is fruitless. It has no rigor and I can not sink my teeth into anything that you are saying. I am not smart enough to figure out even the first thing that you are trying to say. I am sorry. I really tried.

So you were a quant until the vodoo economics came home to roost? I have had a lot of fun discussing Fama/French, portfolio theory and other such stuff with fund managers over the past few years, watching that good old quadrillion dollar derivatives bubble inflate as they framed their bets in gaussian standard deviation errors. Linear thinking in a non-linear world they were busy creating.

Talk about local independence - as in a lack of suitable intelligent global constraints. No fool would have allowed the derivative markets to operate the way they did if they had a proper systems view. I mean book to market and other idiocies. Not only is market information being incorporated immediately, but also the anticipated profits. What a charade.

Nassim Nicholas Taleb would be an example of how the finance world is waking up to the true nature of randomness - yet also how far people in that world still have to go.

Final word: you protest that traders in life are highly correlated. And indeed they try to be in following the herd (so as to beat the herd). But what we were discussing was the model rather than the reality. Which is a model based on a lack of long range correlations.

You state this yourself in saying the system has no memory and immediately incorporates its future. The long term is erased. Only the shortest possible term exists. And this is "efficient" in that it allows for flocking behaviour with volatility so extreme it models a drunkards walk.

It is "efficient" in the sense that it is exponential growth driving the same way e coli will fill an agar plate. Heaven forbid high finance would model the constraints to growth as well. Let's just create systems that we accelerate as fast as we can until they strike some wall.

 

[wofsy]

So you were a quant until the vodoo economics came home to roost? I have had a lot of fun discussing Fama/French, portfolio theory and other such stuff with fund managers over the past few years, watching that good old quadrillion dollar derivatives bubble inflate as they framed their bets in gaussian standard deviation errors. Linear thinking in a non-linear world they were busy creating.

Talk about local independence - as in a lack of suitable intelligent global constraints. No fool would have allowed the derivative markets to operate the way they did if they had a proper systems view. I mean book to market and other idiocies. Not only is market information being incorporated immediately, but also the anticipated profits. What a charade.

Nassim Nicholas Taleb would be an example of how the finance world is waking up to the true nature of randomness - yet also how far people in that world still have to go.

Final word: you protest that traders in life are highly correlated. And indeed they try to be in following the herd (so as to beat the herd). But what we were discussing was the model rather than the reality. Which is a model based on a lack of long range correlations.

You state this yourself in saying the system has no memory and immediately incorporates its future. The long term is erased. Only the shortest possible term exists. And this is "efficient" in that it allows for flocking behaviour with volatility so extreme it models a drunkards walk.

It is "efficient" in the sense that it is exponential growth driving the same way e coli will fill an agar plate. Heaven forbid high finance would model the constraints to growth as well. Let's just create systems that we accelerate as fast as we can until they strike some wall.

thanks for the try but I do not know what you are talking about. A Martingale does not have to be a random walk and certainly is not in various markets. The distinction of short and long term is irrelevant to the mechanics of the underlying process. I just don't get what you are saying. I'm not sure why you are hammering market participants. I guess they were all stupid. I was not a quant.