Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Compensating for outliers during Standard Deviation Calculation ?

  1. Mar 15, 2012 #1
    I am trying to calculate the standard deviation of a group of data based on a 20 period sliding window. I have run into a bit of a problem in knowing how to deal with gaps up or down in the readings and was wondering what the correct method for compensating for this was.


    Below is an example of the situation I am trying to compensate for:

    102
    103
    101
    105
    103
    102
    103
    101
    105
    103
    95
    92
    94
    93
    92
    95
    92
    94
    93
    92


    Obviously there is a big gap from 103 to 95, but in this particular situation that is not of significance to me and I would like to filter it out if possible as these spikes tend affect the readings that follow.
     
  2. jcsd
  3. Mar 17, 2012 #2
    Outliers are indiviual measurements that are significantly different from the rest of the collection. You don't have any outliers here, what you have appears to be two separate populations.

    You could measure the SD of this collection of data, but it would be a fairly meaningless number - indeed the mean itself (98) is meaningless.

    Your data are trying to tell you that something happened after the 10th sample; I'd suggest you investigate that rather than try and ignore it.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Compensating for outliers during Standard Deviation Calculation ?
  1. Standard Deviation (Replies: 4)

  2. Standard deviation (Replies: 3)

Loading...