Understanding probability, is probability defined?

[bobby2k]

They aren't justified in any axiomatic sense.

Even if your remove the word "definitely" with "almost surely" or "almost definitely"?, and say that we model the relative frequencies as probabilities?
What is then your comment to what Mr. Kolmogorov wrote?

http://postimg.org/image/ge5tlgazf/

 

[Stephen Tashi]

What is then your comment to what Mr. Kolmogorov wrote?

http://postimg.org/image/ge5tlgazf/

Being "practically certain" expresses a belief.

 

[Bob]

I have taken a course in probability and statistics, and did well, but still I feel that I do not grasp the core of what holds the theory together. It is a little weird that I should use a lot of theory when I do not get the simple building block of the theory.

I am basically wondering if probability is defined in some way?

In the statistics books I have looked in, probability is not defined, but at the beginning of the book, they give a describtion of how we can look at probability, and this is usually the relative frequency model, but they never define it to be this?

These steps is what I seem to see in a statistics books, do they seem fair?

1. Probability is described in terms of events, outcomes and relative frequency, but never defined.

It is sad. the book is about something never defined.

I find this Kolmogorov's probability theory.

http://en.wikipedia.org/wiki/Probability_axioms

 

[bobby2k]

Being "practically certain" expresses a belief.

I think I am starting to understand what you mean.

I also see now that the statement "we model real world relative frequencies as probabilities" likely is wrong.
When I first justified this in my head I thought that saying that since almost sure convergence says that we have convergence most of the time, but leaves room for divergence it seemed ok to use the realtive frequency interpretation on this, however it seems very sketchy to use the rel. freq. int. on the probability from SLLN but not on the original probability, and offcourse we can't use it on the original in order to leave room for divergence.So the model only says that it is probable(probability 1) that the relative frequency will converge to the probability, any attempts to say that probability is relative frequency(Von Mises), or say that relative frequency is probability(what I tried earlier) will fail one way or the other?

I guess the correct way to be very precise in using probability theory on repeatable events is saying that we assign probabilities to an event if we view it highly probable that the relative frequency will will converge to this probability(number). And if a calculated probability for an event is p, the theory says that with probability 1 the relative frequency of this event will converge(independent trials etc.) to p.
And the reason we as humans make decisions based on probability theory is because we accept that our own perception of probability agrees with the axioms and definition of independence, and hence the mathematical description of probability? So if a calculated probability is 1, we view it as probable in our own perception of probability aswell, because we as humans agree with the axioms?

 

[Stephen Tashi]

I guess the correct way to be very precise in using probability theory on repeatable events is saying that we assign probabilities to an event if we view it highly probable that the relative frequency will will converge to this probability(number).

Yes, theorems in probability theory deal with the probability of things happening. When an actual event or an observed frequency is mentioned, the subject is the probability of such a thing.

And if a calculated probability for an event is p, the theory says that with probability 1 the relative frequency of this event will converge(independent trials etc.) to p.

The technical details of "will converge" involve statements about probability .

So if a calculated probability is 1, we view it as probable in our own perception of probability as well, because we as humans agree with the axioms?

I think most humans expect an event with a high probability to actually happen without consulting any axioms. It's an somewhat circular psychology. We don't accept theories that assign high probabilities to events that we don't expect to happen.

If you want to think about this subject in a coherent manner, you must understand and pay attention to the role of definitions. A definition of a concept must be expressed in terms of other concepts. Because of this, mathematics must begin with undefined concepts. (The alternative would be to get into circular definitions - C1 is defined using C2, C2 is defined using C3, C3 is defined using C1 or to get into infinite regressions - C1 is defined using C2, C2 is defined using C3, C3 is defined using C4,...etc.) The standard approach to probability is to define it in terms of a "measure" and if you trace this back to basic undefined concepts, you reach the same undefined concepts that are used to define the concepts of length, area, volume.

It's a natural human desire to seek a basis for probability theory that would employ concepts such as "the tendency for something to actually happen". This would face the formidable task of dealing with the concepts of "actuality" and "tendency". It would get into semantic tangles such as whether the "tendency for something to happen" is "actual" and whether there can be a "tendency of tendency" etc. If someone has attempted to axiomatize probability theory this way, the results aren't widely known.

In the formal mathematical development of a topic, the undefined concepts are not taken as "obvious" or "understood". You can't assert a property of an undefined concept based only on your intuitive interpretation of the concept. Any properties of the undefined concepts must be stated explicitly as assumptions. A person may feel confident that they can answer any questions that arise about "tendencies" and "actualities", but this competence would not constitute a mathematical theory. To have a mathematical theory, they must declare in advance what set of assumptions they were using to answer the questions.

I'm not saying that it would be impossible to axiomatize probability theory using a set of undefined concepts that are pleasing to our intuitive idea of "tendency" and "actuality". However, I think doing this would be very difficult. (Philosophical discussions of "the potential" and "the possible" vs "the actual" go back to Aristotle. Mathematical treatment would be a different matter.)

 

[bobby2k]

Thanks, I have truly learned a lot from you in this thread. I can't believe that I once thought that the mathematical law of large numbers was some kind of guarantee for real world events.

I am sorry for asking a lot of questions, I think this will be the last one, and if you can confirm it, I think I for now have an adequate understanding about the relationship about probability and observable real world relative frequencies. My question is if the two below statements provide a correct way of thinking about the real-world relative frequencies when using probability-theory? It would be nice if you could confirm it.

1. Let's say we happen to have a real world repeatable and indpendent experiment(let's assume we can). And observe the event A in this experiment. If we choose to use probability-theory to analyze the experiment, the theory says that it is probable that the relative-frequencies will converge to the probability. So even though we have no way of knowing if what we are observing is convering, or even if it was converging we wouldn't have control over the episolon, we still approximate the probability with the relative frequency since the theory says it is probable that the relative-frequency would converge.

2. Conversely, if we for some reason have a repeatable experiment, which contain A, and are given the probability of A which we call p. We can not say for sure what the relative frequency of N(A)/N will be. But the theory says that it is probable that this relative frequency will converge to the probability. So if we observe the experiment and it seems that the relative frequencies converge to something else than the given p, we can not say for sure that it was wrong to say that the probability for A was p. But when using probability theory we can say that it is probable that it was not p, because if it was p it is probable that the relative frequency would converge to p.(Again here I have not taken into account that even if we had convergence we would not have control about the epsilon).

If statistics-books had used this formulation instead of formulations like "the relative frequency will converge to the probability", would you then agree with the books?

 

[Hornbein]

Thanks, I have truly learned a lot from you in this thread. I can't believe that I once thought that the mathematical law of large numbers was some kind of guarantee for real world events.

I am sorry for asking a lot of questions, I think this will be the last one, and if you can confirm it, I think I for now have an adequate understanding about the relationship about probability and observable real world relative frequencies. My question is if the two below statements provide a correct way of thinking about the real-world relative frequencies when using probability-theory? It would be nice if you could confirm it.

1. Let's say we happen to have a real world repeatable and indpendent experiment(let's assume we can). And observe the event A in this experiment. If we choose to use probability-theory to analyze the experiment, the theory says that it is probable that the relative-frequencies will converge to the probability. So even though we have no way of knowing if what we are observing is convering, or even if it was converging we wouldn't have control over the episolon, we still approximate the probability with the relative frequency since the theory says it is probable that the relative-frequency would converge.

2. Conversely, if we for some reason have a repeatable experiment, which contain A, and are given the probability of A which we call p. We can not say for sure what the relative frequency of N(A)/N will be. But the theory says that it is probable that this relative frequency will converge to the probability. So if we observe the experiment and it seems that the relative frequencies converge to something else than the given p, we can not say for sure that it was wrong to say that the probability for A was p. But when using probability theory we can say that it is probable that it was not p, because if it was p it is probable that the relative frequency would converge to p.(Again here I have not taken into account that even if we had convergence we would not have control about the epsilon).

If statistics-books had used this formulation instead of formulations like "the relative frequency will converge to the probability", would you then agree with the books?

This is correct.

 

[Stephen Tashi]

If statistics-books had used this formulation instead of formulations like "the relative frequency will converge to the probability", would you then agree with the books?

I agree with the general idea of what you expressed. I'd prefer to see it written in a way that that makes it clear that we act on belief. When you say " since the theory says it is probable that the relative-frequency would converge." you should make it clear that "since" is not used to mean that there is a mathematical deduction involved. (i.e. It isn't like saying x > 2 "since" x - 2 > 0 )

 

[bobby2k]

I agree with the general idea of what you expressed. I'd prefer to see it written in a way that that makes it clear that we act on belief. When you say " since the theory says it is probable that the relative-frequency would converge." you should make it clear that "since" is not used to mean that there is a mathematical deduction involved. (i.e. It isn't like saying x > 2 "since" x - 2 > 0 )

Ah, very good catch. You are very good at distinguishing what the mathematics is precisely and what is not, even though I have gotten better at this, I still tend to mix in this subject.

Would you say that I could express what I wanted there by changing the sentence, so I only dealt with mathematical terms of the theory? Or is it inevitable that we would use "belief" in explaining/justifying that we approximate a probability with a finite relative frequency?

Also, thank you very much Hornbein for taking the time to read what I wrote!

 

[FactChecker]

There is a mathematically rigorous definition of "probability measure" that does not depend on any concepts of statistics or "frequency of occurrence". A probability measure is a non-negative function of all subsets of a set that is additive for disjoint subsets and the value for the entire set is 1. All the general properties of the probability function can be derived from that basic definition. (For instance, see "A Course in Probability Theory" by Kai Lai Chung). This is more for the pure mathematician than for an applied statistician.

 

[Stephen Tashi]

Or is it inevitable that we would use "belief" in explaining/justifying that we approximate a probability with a finite relative frequency?

If you have a mathematical theory that deals with one set of concepts ( e.g. weight, mass, position) and you try to apply it to a situation defined by a different set of concepts (e.g. price, value, utility) then you must introduce assumptions that establish some relation between the different concepts. You can introduce assumptions as formal mathematical axioms (which is necessary if you intend to prove your results) or you can introduce assumptions by your personal beliefs in an informal manner.

As far as I know, nobody has created a set of axioms for probability theory that establishes any deterministic relation between the relative frequency of an event and its probability. So if you want to establish such a relationship, you must do it using your personal beliefs - or else invent the mathematics that does the job.

 

[FactChecker]

There is nothing mystical about probabilities. Given a set of data, it is obvious, and dirt simple, to ask which fraction or percentage satisfies certain conditions. Scaling those numbers to a [0,1] scale or to a [0%, 100%] scale is just computationally convenient. Also, using the past events (statistics of past experiences) to anticipate similar future events (probabilities of future outcomes) is critical to the survival of even the simplest thinking animals.

 

[bobby2k]

If you have a mathematical theory that deals with one set of concepts ( e.g. weight, mass, position) and you try to apply it to a situation defined by a different set of concepts (e.g. price, value, utility) then you must introduce assumptions that establish some relation between the different concepts. You can introduce assumptions as formal mathematical axioms (which is necessary if you intend to prove your results) or you can introduce assumptions by your personal beliefs in an informal manner.

As far as I know, nobody has created a set of axioms for probability theory that establishes any deterministic relation between the relative frequency of an event and its probability. So if you want to establish such a relationship, you must do it using your personal beliefs - or else invent the mathematics that does the job.

Thank you, I now have the understanding I wanted about this subject. Sorry for taking so much time to understand it, but thank you very much for beeing patient!

 

[PeroK]

"On voit, par cet Essai, que la théorie des probabilités n'est, au fond, que le bon sens réduit au calcul; elle fait apprécier avec exactitude ce que les esprits justes sentent par une sorte d'instinct, sans qu'ils puissent souvent s'en rendre compte."

"One sees, from this Essay, that the theory of probabilities is basically just common sense reduced to calculus; it makes one appreciate with exactness that which accurate minds feel with a sort of instinct, often without being able to account for it."

Pierre-Simon Laplace, from the Introduction to Théorie Analytique des Probabilités.