Empirical and theoretical probability

[Rasalhague]

What's the difference? How are empirical and theoretical probability defined in general? If some people use the terms in different ways to others, is one definition more standard?

(This question is part of a broader quest to work out how the statistics terms "population, sample, sampling, probability" relate to the probability theory terms "sample space, observation space, probability measure, distribution, random variable".)

Perhaps empirical probability is a probability measure associated with a sample space, called "the population", representing all possible outcomes of a physical process that happens to be the object of study. http://people.hofstra.edu/stefan_waner/realworld/tutorialsf2/frames6_3.html makes a distinction between empirical probability and estimated probability. Estimated probability sounds like the natural probability measure associated with a sample space which is a finite subset of the population. Statistics texts call such a sample space a "sample". They call a sample space derived somehow from a set of samples, and their associated measures, a "sampling". Perhaps probability measures associated with "samplings" (memorable terminology!), would also be considered estimated probabilities by Waner.

But then there seems to be another tradition, better represented on the top results in Google, according to which empirical probability is the name given to Waner's estimated probability; that is, the natural measure associated with the kind of sample space which context determines is a sample. So these people's empirical probability is relative frequency, a fancy word for proportion? (And synonymous with "posterior probability".) In this tradition, theoretical probability appears to mean a probability measure on the population, considered as an object inaccessible to direct observation, whose properties (called by statesticians "parameters") can only be infered from samples and their (empirical) probability measures, whose parameters (in the general mathematical sense) are called "statistics".

http://www.regentsprep.org/Regents/math/algtrig/ATS6/LTheo.htm
http://en.wikipedia.org/wiki/Empirical_probability

And http://www.mymathforum.com/viewtopic.php?t=421 is someone who identifies a particular result as an empirical, as opposed to theoretical, probability on the (assumed) grounds that it's an estimate based on empirical data, rather than theory... except that it does rely on a theory, namely that the sample - of what has been recorded so far - is representative of what will happen in the future. Or maybe this default theory is so simple it doesn't count.

Ah, here's a link that elaborates on that second (more standard?) tradition:

http://faculty.vassar.edu/lowry/ch2pt3.html

Could we say that no probability measure with an infinite sample space can ever be an empirical probability measure (=Waner's estimated probability measure)? But then, what if our object of interest was itself a mathematical idea, such as the normal probability measure with such-and-such a mean and standard deviation, and the real line for its sample space? Is every real number "capable in principle of being observed"? I guess yes, so, in this context, this normal probability measure would be an empirical probability measure. Maybe we need to include something about context, and how these tools are applied.

Well, I'm probably getting tangled up in philisophy now. That last link has a nice description:

An empirical distribution is one composed of some set of variates—that is, values of X_i—that have either been observed or are capable in principle of being observed. If you were to measure the level of serum cholesterol of 100 adult Canadian males, the resulting 100 values of X_i would constitute an empirical distribution. If you were to speak of the distribution of serum cholesterol levels among adult Canadian males in general, that too would be an empirical distribution, even though you might not have observed all or even most of the multitudinous X_i values of which this distribution is composed.

A theoretical distribution, on the other hand, is one that is derived from certain basic facts, principles, or assumptions, by logical and mathematical reasoning involving a more or less complex sequence of conditional statements of the general form "If such-and-such is true, then so-and-so must also be true." In general, the procedures of inferential statistics begin with one or more empirical distributions and conclude by making reference to a theoretical probability distribution.

 

[Valkarie]

So, to sum up: empirical probability is the probability measure associated with a sample space, considered either as the population or as a finite subset of the population, representing all possible outcomes of a physical process. Estimated probability is the probability measure associated with the sample space which is a finite subset of the population. Theoretical probability is the probability measure associated with the population, considered as an object inaccessible to direct observation, whose properties can only be inferred from samples and their (empirical) probability measures.