B Is there a definition of randomness?

[Zafa Pi]

Defined as... Deterministic Unridiculous Randomness (DUR)

Deterministic? Where did that come from?

 

[Stephen Tashi]

The definition I gave in post #85 has that built in. It's the 1st time I've tried to give a definition, what deficiencies do you find with it?

I suggest that a random sequence is generated by certain specified physical processes. For example coin flipping. More general, lab measurements that correspond to QM measurements that are random variables, e.g. measuring electron spin at 90° from spin up electrons at 0°.

As you said, you haven't given a mathematical definition. Until you explain how to determine which physical processes are among the "certain specified physical processes", we don't have a specific physical definition.

A definition of "randomness" that only classifies a process as "random" or "not random" isn't very useful. It would lump tossing a fair die in the same category as tossing a loaded die.

 

[OCR]

Deterministic? Where did that come from?

It just " locks in " ... Unridiculous .
So you can have randomness, with absolutely no foolishness.

Is that speculation ?

 

[Stephen Tashi]

People interested in attempts to define probability in terms of sequences of numbers might be interested in the attempt by Richard von Mises (e.g. https://pdfs.semanticscholar.o/6f41/fad8b768217a116ea3216ae2656fec42a82a.pdf). I don't know of any work that makes the von Mises "collective" mathematically sound, but a mathematical system wasn't von Mises objective.

 

[stevendaryl]

Random := The value(s) produce by an objective* physical* process that when repeated yields a sequence that passes randomness tests. * objective means repeatable by others. * physical means non-algorithmic, like coin flips.

it seems to me that @andrewkirk gave the most scientifically relevant answer. A process is random or not relative to a theory for predicting it.

A randomness test can at best check whether something is predictable by simple algorithmic means.

Of course, what this means is that some aspects of a sequence might be random, while other aspects are not. For example, there might be a sequence

12121212124121212121212121251212121212121212...

which is mostly an alternation between 1 and 2. So a "randomness test" will fail, saying that it's not random. But the departure of the sequence from predictability may be random.

The other way around would apply, as well. There could be a sequence that looks completely random, but is actually completely predictable. The digits of pi is an example.

If you're interested in how hard it is to predict a sequence, I think computational complexity theory is more appropriate. Given a sequence of digits you can characterize how random it is by how much it can be compressed algorithmically.

 

[Curiose]

But, what if there is a physical, theorectical limit to observation knowledge? This could come from a) there being a limit on how accurate a measurement of position could possibly be; and/or b) a limit to the knowledge of two observables - the HUP (Heisenberg Uncertainty Principle) essentially says that the more accurately you know the position of a particle, the less accurately you know its momentum - in any case, you can't know both to an arbitray precision; and/or c) a piece of data that is intrinsically random, such as the spin on an electron, which may be theorectically unknowable.

This ties into my post above. There is no certainty that Quantum Mechanics will one day be replaced by a "deterministic" theory, where the HUP and the instrinsic probabilities disappear. And, unless that happens, then there are clear limits to observational knowledge.

To be more specific, I'm not necessarily saying that randomness exists, but I'm also not saying that it doesn't exist. I definitely don't have privileges that break quantum observation, lol.

And yes, the limits are clear. Assuming that human beings and human logic are the highest point on some hierarchy we imagine exists, thus creating our measurement error in any possible measurement device that we can currently imagine. To be more precise, we will never have a finalized and perfect definition of the physical world, thus we will never have a fully agreed upon definition of what is random.

Random := The value(s) produce by an objective* physical* process that when repeated yields a sequence that passes randomness tests.
* objective means repeatable by others. * physical means non-algorithmic, like coin flips.

I like this definition of randomness as it is useful for most purposes and clearly delineates physical phenomena from math.

It's theoretical, but I don't think it's fluff. Un-testable by current standards? yes. There is still a humongous debate about whether or not reality is deterministic. I am liking what's going on with Quantum Gravity theory with their quasicrystals and information theoretic notion of reality which takes neither side of the debate. We can only measure phenomena which we can sense (with our senses or some device), and even then, the measurement of that phenomena changes the outcome, plus is very prone to error. Are there dimensions in physical systems which we are not taking into account and could be measured to approximate an outcome, but we simply cannot measure them yet?

I'm not saying that we are in a simulation. That is, I think, too far off topic. But I will give a simulation test example of how randomness can be explained. The problem being that you don't know if physics is the observed result of some higher order algorithm which sits behind the true laws governing the physical system in which the coin is being tossed. You have to take into account "the observer" which is also theoretical, but I mean, if we can't prove randomness exists, then randomness is also theoretical.

Let's take the idea and inverse it.

If you assume that the observer exists, technically any device which senses a physical phenomenon is an observer of that phenomenon whether or not it comprehends what it is sensing. Let's say that I make a program which takes input from a camera, performs a couple of filter operations on the incoming data and creates a 2D space which is scattered with colored points that represent the color edges being sensed in the current frame. So my space has the following dimensions: (x, y), (r, g, b) and the x,y sub-space is filled with points by the edge filter using the r,g,b data from the original video frame. Now let's say that I also populate this space with "observers" which follow some rules that govern their behavior.

What my little simulated observers are sensing are the edge point positions and color that are the result of multiple filters working constantly on some input data. The filter input data is coming from "my reality" or what you refer to as the physical world, whereas the filtered output data is a simplified projection of that raw data. To further complicate things, let's assume that the simulated observers can recall and track edge shapes from the points they observe. They will always ever be observing a filtered 2D projection of a 3D space. Let's also assume that they have some ability to remember patterns and thus predict near-future outcomes like a shape they will see, or where and in what orientation that shape will turn up.

They will never be able to fully predict where and when in their reality, the points that represent my hand will show up, because they don't know my hand exists as a hand. The idea of what a hand is supersedes them, since they don't have hands or bodies. A hand is only represented in their space as a recognizable set of points with specific relationships in 2D+Color+Edge space, which occur in some positions at some rotations, etc. The physics of how a CCD camera works, how the edge filter algorithm works, and how my own decision making process works are all impossible to know for these observers because their observable reality is a product of these perception filters used to create their reality space. Their reality is a projection of my reality that is further filtered before they can even perceive it. These are the observational limits of their perception as set forth by filtering (projection) process.

It is to say, I can accurately predict where the points that represent my hand will show up in their space and in roughly which configuration those points will be, because I am an outside observer of their reality and I can perceive the extra dimensions which affect that reality, but they cannot (imposed measurement limit). No matter how intelligent these lower dimensional observers are, the best they can ever do is create a statistical observation of where my hand might show up with some certainty quantifier. They might even make up equations like the Heisenberg Uncertainty Principle to formally define this and try to explain the phenomenon as random because it saves them the time of trying to imagine something that eludes them about their own perception of what they believe to be reality.

I'm not saying that this microcosmic example is true for our perception of reality, I'm saying that we never stand a chance of knowing if randomness truly exists or not. Perhaps it's just an artifact of the laws that govern the physical space we exist in, which could be generated by a higher-dimensional space in which other laws apply.

It is proven that each perception of reality is not the same as other's perceptions. The Pauli exclusion principle perhaps is a clue that either we are not seeing the same data, or that we are seeing the same input data, but through the chained filters of our "perceptive ability" and the distinct point of view through which we perceive the space. No two measuring devices can exist in the same physio-temporal position, thus no two perceptions of the same data will ever result in the same final input which is then processed by the perceiver. This is yet another filter, the entire process taking time to actually happen. In a very real way, the perceptive filter of our reality is represented physically by a sense organ or device. What we see is what we get. Are we all inside of a higher dimensional reality which, upon perception, collapses into a lower dimensional representation of that reality?

That brings us to the idea of fractals. A fractal is a projection of a higher dimensional space onto a lower dimensional plane. Point by point at any scale or n-dimensional rotation, can be calculated up to the currently working infinity limit, which is rendered in the projection as negative space. When projection happens, one or more dimensions must be collapsed into another dimension on the plane of interception/observation. When your eye perceives light, it generates signals which are rendered in your brain as a 2D projection. If you have two eyes, you have a higher sense of depth. This is a perception of a dimension of physical space which we call depth. This third perceived dimension helps us further define and comprehend the data we are perceiving inside of the space in which we exist.

Is it hard to imagine that the factors operating in a coin toss elude our limits of perception given the limited physical representation of reality at which we interact and observe? No... and that's why we have statistics. Statistics smooth for "randomness" by counting, summing, and the like, in order to account for the error in our ability to recognize data as an ordered pattern due to those perceptive limits.

This is really a great question! Thanks for the inspiration. I'm loving this forum, I feel like I've been missing out on talks like this IRL.

 

[WWGD]

I wonder if it would be meaningful to throw-in the idea of some Godel-like results, if we view a theory as a collection of axioms together with rules of inferences. Then there will always be some non-random events ( not provable within the system).

 

[Zafa Pi]

it seems to me that @andrewkirk gave the most scientifically relevant answer. A process is random or not relative to a theory for predicting it.

Randomness in QM is different, because you have perfect information. You have an ensemble of electrons that are spin-up in the z-direction; you measure their spin in the x-direction and you get spin-up 50% and spin-down 50%.

The theory of QM predicts this and suggests that there is no further information that could possibly be available to you (hidden variables) that would allow you to predict when an electron will be spin-up and spin-down.

Tossing a coin is random because you have inexact information about the experiment.

QM states that measurements are random variables, and it also states that the evolution of the state of an electron (and S/G device) obey a deterministic law.
What ever theory one proposes it is a subjective decision (with consensus) which physical instruments and processes are modeled by the theory.

Here is my simple theory: A coin flip is a two valued uniform r.v.
Most would agree that a "fair" coin flipped from the Eiffel Tower or in a wind tunnel is a physical device that is modeled by the theory. I now define that processes/result as random. And most would agree, though some would say random due to ignorance. I find this silly since saying there is information (but inexact) is math or philosophy not physical (no way to test). Same goes for determinism in general.
Any other r.v. can be generated by the coin flip. Flip it a 100 times and you have have a number from [0,1] chosen uniformly. Take functions of that to get any other continuous distribution. With an average of two flips of the fair coin you can get any other two valued r.v., e.g. 1 with probability 1/π, 0 with probability 1 - 1/π.

Now this may too slow in practice, but so what it's a definition of random. What the OP asked for.
Now, what if someone asks if the results of a S/G apparatus (above) are random. Well what does the consensus say, is it distinguishable from coin flipping? If not then it's random, if it is how so.

 

[entropy1]

The theory of QM predicts this and suggests that there is no further information that could possibly be available to you (hidden variables) that would allow you to predict when an electron will be spin-up and spin-down.

Unless you have/apply ALL the information in the universe?

 

[PeroK]

Unless you have/apply ALL the information in the universe?

One problem is that "all the information in the universe" may not be well defined. Especially if the universe is infinite.

 

[entropy1]

One problem is that "all the information in the universe" may not be well defined. Especially if the universe is infinite.

Ok. Suppose we know ALL there is to know about the universe, EXCEPT the outcome of the spin-measurement of our electron. Would that imply that we THEN could calculate what the outcome will be? (Almost the same question, I realize)

 

[PeroK]

Ok. Suppose we know ALL there is to know about the universe, EXCEPT the outcome of the spin-measurement of our electron. Would that imply that we THEN could calculate what the outcome will be? (Almost the same question, I realize)

I don't believe that is a well defined question. But, to take it at face value, I don't know. There is nothing in physics that demands that we would know the outcome in advance. And QM suggests that we couldn't necessarily know the outcome.

Just to be clear: it's the assumption that ultimately all uncertainty can be, theoretically at least, swept away that I believe is wrong.

 

[entropy1]

I was wondering, if variable A is random, it is said we cannot predict its outcome. However, can't we really? We can't for a single instant, however we can or should very well for the distribution of a set of generated outcomes; that is the principle of probability theory, right? Maybe this is contrary to situations where we can predict a single outcome, but not a distribution, like when I drive through town and I can predict when I will go right or left, but not what the ratio between right or left will be. Anyway, in case of our variable A, the distribution of outcomes should be pretty predictable, right?

 

[stevendaryl]

Ok. Suppose we know ALL there is to know about the universe, EXCEPT the outcome of the spin-measurement of our electron. Would that imply that we THEN could calculate what the outcome will be? (Almost the same question, I realize)

According to quantum mechanics, no. There is nothing in the far reaches of the universe or in the details of subatomic particles that would allow you to predict the result of a measurement, in general. (There are certainly cases where the result is predictable, but in many cases, it is not.)

 

[entropy1]

According to quantum mechanics, no. There is nothing in the far reaches of the universe or in the details of subatomic particles that would allow you to predict the result of a measurement, in general. (There are certainly cases where the result is predictable, but in many cases, it is not.)

So 'who/what' is 'deciding the outcome' then?

 

[stevendaryl]

So 'who/what' is 'deciding the outcome' then?

Quantum mechanics doesn't say anything about what or who decides the outcome. So if you need an answer, then you have to have some theory that goes beyond quantum mechanics.

To say it's random is the same thing as saying that nothing decides the outcome.

 

[entropy1]

To say it's random is the same thing as saying that nothing decides the outcome.

So, if there is no decision, maybe both outcomes are real, right? Depending on the interpretation? And maybe both are real given a certain probability? (for my understanding)

 

[rootone]

If randomness could be defined then it's not random.
Sort of a non sequitur

 

[Zafa Pi]

If randomness could be defined then it's not random.
Sort of a non sequitur

I defined random in post #108. How does that definition make it not random?

 

[entropy1]

I defined random in post #108. How does that definition make it not random?

Isn't a coin flip from the Eiffel Tower random because the effect (the yieling of the result) and the cause (the flipping of the coin) have such a complicated relationship (chaoticly), that the relationship can't be described, not conceived and not even traced back that FAPP there IS no relationship between cause and effect?

 

[entropy1]

In short, QM is a probabilistic theory because it makes probabilistic predictions. But that 'probabilistic' is a property of the theory, not of the universe.

If we have probabilistic definitions of events in QM, doesn't that then imply that we can't make predictions? (on that field)

 

[entropy1]

In your case, you can hypothesise that QM might be replaced - frankly, anyone can do the hypothesising - but the critical question is how your exact, non-probabilistic theory could explain the observed phenomena. What are its key elements that would allow it to do that? And, moreover, what experimental or theoretical justification is there to get your theory going in this or that direction?

I would say for instance: all X-spin directions of all the electrons in the universe add up to 0, for instance. So, one electron's spin is fixed by all the other electrons, and theirs on their turn also. There is however no way to verify that, but maybe it is possible to build a theory around it and get circumstantial evidence, I don't know.

 

[andrewkirk]

If we have probabilistic definitions of events in QM, doesn't that then imply that we can't make predictions? (on that field)

No. It doesn't imply that. It just implies that we cannot make predictions using quantum mechanics alone.

 

[entropy1]

No. It doesn't imply that. It just implies that we cannot make predictions using quantum mechanics alone.

What else would be required then?

 

[andrewkirk]

What else would be required then?

A larger theory, that is compatible with QM within certain constraints that are satisfied by most experiments done to date.

 

[Zafa Pi]

Isn't a coin flip from the Eiffel Tower random because the effect (the yieling of the result) and the cause (the flipping of the coin) have such a complicated relationship (chaoticly), that the relationship can't be described, not conceived and not even traced back that FAPP there IS no relationship between cause and effect?

It's random because no one can predict the outcome of a flip better than 50/50. Chaos is a math concept, a coin flip is physical

 

[Zafa Pi]

No. It doesn't imply that. It just implies that we cannot make predictions using quantum mechanics alone.

This is your answer to:

If we have probabilistic definitions of events in QM, doesn't that then imply that we can't make predictions? (on that field)

Then entropy1 says:

What else would be required then?

and you reply

A larger theory, that is compatible with QM within certain constraints that are satisfied by most experiments done to date.

Do you think it is possible that some grander theory than QM will avoid random (see post #108) outcomes for measurements for spin? Not even super determinism will accomplish that. However, super determinism, as a theory, does say there are no probabilities.

 

[stevendaryl]

Do you think it is possible that some grander theory than QM will avoid random (see post #108) outcomes for measurements for spin? Not even super determinism will accomplish that. However, super determinism, as a theory, does say there are no probabilities.

The question is mixing up the two different meanings of "random". A theory may be deterministic, so that there are no intrinsically random events, but it may be practically impossible to predict some events, and so that they would appear random. But "appearing random" would be a matter of how much computational power you want to put into predicting future outcomes. I suppose that past a point, prediction would be impossible in practice.

 

[entropy1]

It's random because no one can predict the outcome of a flip better than 50/50. Chaos is a math concept, a coin flip is physical

I mean, is being random the result of the decoupling of cause and effect? (and maybe the definition, as you suggest)

 

[Zafa Pi]

The question is mixing up the two different meanings of "random". A theory may be deterministic, so that there are no intrinsically random events, but it may be practically impossible to predict some events, and so that they would appear random. But "appearing random" would be a matter of how much computational power you want to put into predicting future outcomes. I suppose that past a point, prediction would be impossible in practice.

I have only one definition of random (given in post #108), it applies to a physical process. A theory may state that certain things are random variables.
I agree that a deterministic theory may say there are no r.v.s and as a consequence no random events, merely a lack of knowledge, but then as you say prediction would be impossible in practice.

My key objection is (are?) the words "in practice" and "intrinsically". They are unnecessary, superfluous, and misleading. When flipping a coin from the top of the Eiffel Tower or in a wind tunnel there is no information that would predict the result. Some might object and say: if one knew the position and velocities of all the entities in the air, the the position and force from the thumb, etc. the result could be predicted. I say it is a pipe dream to say that information exists. Certainly you can't prove that it does, all one can do is hypothesize its existence based on a nonexistent theory below the Planck scale. As Griffiths said, God doesn't even know.
All is random, and the Law of Large Numbers keeps us sane.

 

[Zafa Pi]

I mean, is being random the result of the decoupling of cause and effect? (and maybe the definition, as you suggest)

In a coin flip what are the cause and effect? After you answer that tell me what decoupling means.

 

[entropy1]

In a coin flip what are the cause and effect? After you answer that tell me what decoupling means.

What are cause and effect is a matter of preference I think; you could say that the tossing device is the cause and the effect is the read-out (heads/tails).

With "decoupling" I mean that there is, practically at least, no discernable relation between the cause and the effect - this would be randomness, that is, for the effect/readout.

 

[entropy1]

Some might object and say: if one knew the position and velocities of all the entities in the air, the the position and force from the thumb, etc. the result could be predicted.

With no identifyable relation between cause and effect, I mean the correlation between cause and effect is just as random as the toss is, regardless of in-principle possible relations. But maybe this is circular - dunno. Probably.

 

[stevendaryl]

My key objection is (are?) the words "in practice" and "intrinsically". They are unnecessary, superfluous, and misleading. When flipping a coin from the top of the Eiffel Tower or in a wind tunnel there is no information that would predict the result.

Well, I disagree completely. To say that something is in practice unpredictable has a clear meaning, and it’s a different meaning from a theory being stochastic, or probabilistic.

 

[Zafa Pi]

What are cause and effect is a matter of preference I think; you could say that the tossing device is the cause and the effect is the read-out (heads/tails).

So you don't think the breeze or the topography of the ground are part of the cause? Right up until the coin settles down.

With "decoupling" I mean that there is, practically at least, no discernable relation between the cause and the effect - this would be randomness, that is, for the effect/readout.

I wonder where that decoupling takes place? If the coin weren't flipped there would be no result.

 

[rootone]

If the coin weren't flipped there would be no result.

Yeah, that would be the null hypothesis.

 

[Zafa Pi]

Well, I disagree completely. To say that something is in practice unpredictable has a clear meaning, and it’s a different meaning from a theory being stochastic, or probabilistic.

Well I disagree that we disagree. I do agree with what you said.
Our difference is semantic. Saying that a coin flip is in practice unpredictable carries no more information than saying it is unpredictable, and what I call random (see #108). I think the "in practice" is a red herring and adds confusion. Determinists say it is due to lack of knowledge and that is "not even wrong".

QT says that measurements are random variables - stochastic, very different than above. But in QT or probability theory the word random or intrinsically random do not appear and not defined. And that's why they should be tossed out in that context.

 

[entropy1]

QT says that measurements are random variables - stochastic, very different than above. But in QT or probability theory the word random or intrinsically random do not appear and not defined. And that's why they should be tossed out in that context.

If randomness is not defined, how do you predict anything that is random or derived from it?

 

[Zafa Pi]

If randomness is not defined, how do you predict anything that is random or derived from it?

You only quoted part of my post. I did define it. Please read my other posts before you reply.

 

[entropy1]

You only quoted part of my post. I did define it. Please read my other posts before you reply.

I did read them. I must say I'm a little confused. Does QM only take place around the Eiffel tower? Or must all QM related experiments be reducible to Eiffel tower coin tosses?

 

[bhobba]

I don't know why this thread has gone on for so long. At present for many pseudo random number generators we have tests that tell us its not random - but many is not all - some pass the lot:
file:///C:/Users/William/Downloads/tuftests.pdf

So the answer is right now we can't tell if a sequence is really random or not - that may change of course.

Thanks
Bill

 

[entropy1]

What I'm wondering: we can't predict an individual outcome of a random variable, but we make assumptions about ensembles of oucomes of a r.v. that in practice can be approached arbitrarily accurate by theory. So is that a property of randomness/probabilistics?

I am thinking of entanglement and correlations, where the correlations seem to have a tangible regularity.

 

[andrewkirk]

Do you think it is possible that some grander theory than QM will avoid random (see post #108) outcomes for measurements for spin?

I can imagine there being such a theory. Whether humans can ever come to know such a theory is a different question, to which I suspect the answer is No.

Not even super determinism will accomplish that.

It sounds like you're thinking of 'super-determinism' as a Theory. In my experience, when that phrase is used, it is not referring to a complete theory, but at most an aspect of a theory.

A theory may state that certain things are random variables.

I don't know of any theory that says that. What the theories I have seen say is that, under the theory, a certain measurable quantity is modeled as a random variable, which is a very different thing. It is not the business of science to say what things 'are', only how they can be modeled. And thank goodness for that, or scientists would get bogged down in unresolvable arguments about the nature of Kantian noumena. There'd be no time left for inventing useful stuff like QM or GR.

 

[Zafa Pi]

Your response to my statement: A theory may state that certain things are random variables. is

I don't know of any theory that says that. What the theories I have seen say is that, under the theory, a certain measurable quantity is modeled as a random variable, which is a very different thing. It is not the business of science to say what things 'are', only how they can be modeled. And thank goodness for that, or scientists would get bogged down in unresolvable arguments about the nature of Kantian noumena. There'd be no time left for inventing useful stuff like QM or GR.

QT says that. My favorite text (Nielsen & Chuang) states as it's 2nd postulate that measurements are random variables (plus details). There are many other sources.
Theories are models.

 

[Zafa Pi]

I don't know why this thread has gone on for so long. At present for many pseudo random number generators we have tests that tell us its not random - but many is not all - some pass the lot:
file:///C:/Users/William/Downloads/tuftests.pdf

So the answer is right now we can't tell if a sequence is really random or not - that may change of course.

The reason why is that there is a great deal of confusion over what random means. Most people on this thread agree that a sequence produced by an algorithm is not random since its values are predictable, regardless of satisfying randomness tests.
Random is not a defined notion in probability theory. I am attempting to define it in terms of a physical process, see post # 108. So far I have not found coherent objections.

 

[bhobba]

The reason why is that there is a great deal of confusion over what random means. Most people on this thread agree that a sequence produced by an algorithm is not random since its values are predictable, regardless of satisfying randomness tests.
Random is not a defined notion in probability theory. I am attempting to define it in terms of a physical process, see post # 108. So far I have not found coherent objections.

That post looks fine.

My issue is simple. Give someone some data, even how you obtained it, such as you did in in the mentioned post eg Most would agree that a "fair" coin flipped from the Eiffel Tower or in a wind tunnel is a physical device that is modeled by the theory. I now define that processes/result as random.

So you can PROVE some future test for true randomness may not tell us its not really random? I think it highly unlikely - but we are speaking of matters of principle here.

As of now you can't tell if something is random - meaning it can't be modeled by some deterministic process - or not. We have deterministic sequences that pass every test we have for randomness. Even QM can't be assured of that - even though the consensus is it truly is random - just like there would be the same consensus for what you mentioned - we can't prove it.

Added Later:
You would think a roulette wheel is random - I know I would have - except for one thing:
https://www.amazon.com/dp/0140145931/?tag=pfamazon01-20

The only thing that looks, in light of things like the above, truly random is QM - but we have no way to prove it.

Thanks
Bill

 

[DrClaude]

This thread has run its course. Time to close.

Thanks to all that participated.