Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Quantify difference between discrete distributions

  1. Feb 14, 2013 #1
    Hello,

    I am trying to quantify the difference between two discrete distributions. I have been reading online and there seems to be a few different ways such as a Kolmogorov-Smirnov test and a chi squared test.

    My first question is which of these is the correct method for comparing the distributions below?

    The distributions are discrete distributions with 24 bins.

    My second question is that, it pretty obvious looking at the distributions that they will be statistically significantly different, but is there a method to quantify how different they are? I'm not sure, but a percentage or distance perhaps?

    I've been told that if you use the Kolmogorov-Smirnov test, a measure of how different the distributions are will be the p-value. Is that correct?

    I appreciate any help and comments

    Kind Regards

    https://dl.dropbox.com/u/54057365/All/phy.JPG [Broken]
     
    Last edited by a moderator: May 6, 2017
  2. jcsd
  3. Feb 14, 2013 #2

    mathman

    User Avatar
    Science Advisor
    Gold Member

    Note: I am not a statistician. One simple test would be to calculate the correlation coefficient. This is a measure of the statistical difference.
     
  4. Feb 16, 2013 #3
    KS and related distances compare the distribution of two random variables X and Y by finding the maximum difference between Prob[X is in A] with Prob[Y is A] over various sets A. For KS, the sets A are of the form (-inf,b] and for Kuiper they are (a,b]. Both of these distances can be expressed in terms of the cdf so to visualise it you just need to plot the cumulative probabilities.

    Now to measure the significance of the observed distance, keep in mind that many statistical packages assume that the null distribution is continuous and may return inaccurate p-values, e.g. in R the standard function ks.test{stats} assumes continuous distributions but ks.test{dgof} allows discrete distributions.
     
  5. Feb 20, 2013 #4

    ssd

    User Avatar

    Calculation of correlation coefficient is "Not" a test. Neither it is a measure of statistical distance. Simple corr. coeff. is only a measure of linear association between two variables.

    OP may look for Matusita distance.
     
  6. Feb 21, 2013 #5
    It depends on what is important to you. It's purely subjective. I don't think that there is a standard method. If you tell us what you are trying to determine, we might be better able to help.

    Given what we have I can only guess you have a model for the situation and are trying to decide whether or not it is a good one. I'm not sure statistics can help you with that. It could help with deciding which of two models fits the data better. For that you could use the sum of the squares of the differences between the model and the data. There isn't a deep reason for this, it is just the standard method that everyone uses, so you may as well use it unless you have some reason not to.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook