Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Statistics vs probability

  1. Jan 11, 2012 #1

    I've just began my first probability class, and let me tell you, it's a doozy. It reminds me a lot of my physics classes: There's a general sort of way to go about solving a problem, but each problem is completely different from the other. If this was the entire major, I'd stop here, it's not interesting, just kind of trivial and frustrating so far.

    Good thing it isn't. For the major at my school, a good 4/5 of it isn't probability, but statistics. The general consensus is once you get the probability theory out of the way, the statistics element is not only a breeze, but actually very engrossing and enjoyable. I've always been interested in data, hence why I signed up for the major in the first place.

    I would like a second opinion though. Are statistics and probability different from each other, even though they are very much related?
  2. jcsd
  3. Jan 12, 2012 #2
    Probability is a formal theory while the term "statistics" can be applied to data as well as to methods for summarizing and making inferences from data. Both probability theory and statistical analysis are concerned with the concept of the random variable defined as a function or mapping from an event space to the interval [0,1]. Probability theory does not address just how a particular value on this interval is selected. For that, a statistical model based on a probability distribution is generally required.

    So for instance, a function which defines a probability distribution obeys the axioms of probability theory which require the integral or sum of the function over its complete range to be unity. The mean and variance are sufficient statistics to define a particular Gaussian distribution. The general form of this distribution is defined by its probability density function.
    Last edited: Jan 12, 2012
  4. Jan 12, 2012 #3


    User Avatar
    Science Advisor
    Gold Member

    Statistics is the application of probability theory to data sets. In general, one makes a hypothesis about the data e.g. what it's mean and standard deviation are then try to get an idea of how likely the hypothesis is to be true. This likelihood generally requires knowing something about the probability distribution of the data set. Classically, statisticians realized that by averaging their data they were sampling from a distribution that was close to normal. Since normal distributions are extremely simple -they have only two parameters, their mean and their standard deviation, and their shape is easy to calibrate -this made the formation of hypothesis possible even without computers. This I think is why much of statistics is about hypothesis formation when the underlying distribution is normal.

    Now a days computers make it possible to form hypotheses about real world distributions without averaging to get normal distributions but normals are still widely used. Often real world distributions are close to normal any many statistical models assume normality. For instance, Brownian motion and its relatives are good approximations for daily returns on stocks and bonds.
    Last edited: Jan 12, 2012
  5. Jan 12, 2012 #4


    User Avatar
    Homework Helper

    An aphorism (of my own invention, as far as I know) that I like is "Probability theory gives perfect answers to unrealistic problems, while statistical theory gives imperfect answers to realistic problems."

    Kind of sums things up. NOT. :biggrin:

    Anyway, other posters have already summed up the salient points in a more serious fashion.
  6. Jan 12, 2012 #5
    This is the most confusing course I've ever had D: Can anyone recommend any resources?
  7. Jan 12, 2012 #6


    User Avatar
    Science Advisor
    Gold Member

    You can ask some questions here. I can answer basic stuff.

    I think once you get the idea of what is going on from a few examples you will find it easier.

    Not sure about resources.
  8. Jan 13, 2012 #7

    Stephen Tashi

    User Avatar
    Science Advisor


    I don't know if your study of statistics will be "frequentist" or "Bayesian". If it's Bayesian, then there shouldn't be much trouble seeing how probability theory relates to it once you understand conditional probability.

    If it's "frequentist" then prepare for confusion. My personal outlook on that situation is the following: If a common sense type of person has data, he wants to know about the probabilities of certain ideas about the world ("the population") given the observed data or the probable value of certain parameters given the observed data. Frequentist statistics calmly listens to such questions and proceeds to tell you things based on the probability of the observed data given that we assume a certain idea or parameter. (It's the distinction between "the probability of A given B" versus "the probability of B given A".) If you want to keep a clear head while studying frequentist statistics, you must keep in mind that when you hear probabilities, they are almost always related to the probability of the data given that we assume certain ideas, not the other way around. The jargon of frequentist statistics ("confidence", "significance" "rejection region" etc.) strongly suggests to laymen that they are getting information about the probability of some property of the population given the observed data. This is not the case. To see clearly how probability theory relates to frequentist statistics, you must understand that frequentist statistics merely advocates certain decision procedures based on the probability of the observed data and the probability of that data is computed by making some specific assumptions about the population it came from.

    Both types of statistics, when applied to real world problems, are subjective. Bayesian statistics needs "prior" distributions. Frequentist statistics needs subjective levels of "significance" or "confidence".
  9. Apr 19, 2012 #8
    Statistics sounds like the summarization of Data and analysis and interpretation. While Probability is more theoretical. Both are actually interconnected to each other. And to me, I used to be very much interested in Statistics.
  10. May 7, 2012 #9
    They are, in probability you apply deduction, that is, you have all the information and you make predictions about individual elements (e.g. 75% of students are male, 25% of students are female, therefore there is a 1/4 probability the person sitting behind you in class is a girl)

    In statistics you apply inference, that is, you have a sample of the information and you make predictions about the whole (e.g. 1 out of 4 persons sitting behind you are girls, therefore you predict 25% of students are girls within a certain confidence).

    I hope this was helpful.

  11. May 7, 2012 #10
    Just a recommendation for Intervenient.

    If you are in your first probability class, just take a look to the book
    "An introduction to Probability Theory and Its Applications Vol.I, 3rd.Ed.", by William Feller.

    This book is a high level one, but if you read only the Introduction and Chapters I, II and maybe V, VI and VII, you will grasp the intuitive idea of probability following one of the masters of it.

    When I begin my probability courses, I had the same feeling that you have now: The textbooks I had were such a mess !!!.

    I found Feller's book, and light become to probability...and I discovered how wonderful (and useful) this theory is.
    I hope this can help you
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook