Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Mathematical impact of outliers on accuracy of models

  1. Sep 1, 2009 #1
    Is there a general approach to calculating the impact outliers have on the accuracy of one's (predictive) model?
     
    Last edited: Sep 1, 2009
  2. jcsd
  3. Sep 2, 2009 #2
    I think your question is too general. You need to describe your model.

    If your model is based on regression against approximately normally distributed data, the influence of outliers is well understood.

    I have seen data analysts rerun the model with and without the outlier points.
     
  4. Sep 2, 2009 #3

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    As wofsy said, the question is far too general. There is no one technique for analyzing what outliers will do / have done. Editing outliers is one commonly used technique. Sensors do occasionally go out to lunch. Transmission errors can create huge outliers. Just about the only thing one can do with a 1020 sigma outlier is to delete it. This suggests a refinement on the approach wofsy described in his post. Use some heuristic to delete gross outliers, run the model, delete statistical outliers that the gross heuristics didn't catch, and re-run the model.

    This doesn't always work because it assumes that the heuristics and model are basically correct. Example: The ozone hole over Antarctica was initially discovered by ground observations rather than by satellite observations because of the overaggressive use of this technique on the satellite data.
     
  5. Sep 2, 2009 #4
  6. Sep 2, 2009 #5

    statdad

    User Avatar
    Homework Helper

    The question and answers are dancing around the topic of robust statistical analysis methods. Deleting outliers is one way to deal with them, but unless you know that they are due to errors in measurement (sensors going haywire) eliminating them simply because they are outliers is not a valid statistical procedure.
    It is also important to note these things:
    outliers are rather easy to find in low dimensional problems, but extremely difficult in high dimensional problems.
    in regression, points of high leverage may not appear as outliers in the traditional sense of large residuals - in severe situations the regression line may pass through them, so the residual is zero.

    the point of a robust analysis is to use a process that yields results that can be interpreted in ways similar to the traditional least-squares (normal distribution assumption based) methods but which are not as easily influenced by departures from the hypothesized model as the traditional methods might be.

    perhaps a too long comment, but discussing "tossing out data" in general can lead to dangerous things.
     
  7. Sep 2, 2009 #6
    The partial leverage article was useful. Thanks for all the responses. What i was trying to get at was, is there a general means to determine the probability of a high-leverage point being an influential point?
     
  8. Sep 2, 2009 #7

    statdad

    User Avatar
    Homework Helper

    No, not to find the probability. If you are working with regression, doing a search on regression diagnostics will provide advice on some numerical measures of the severity of a leverage point on your fit.

    Note that all points in statistics are influential - but not all are equally influential, in good or bad ways, so looking for "influential points" may not lead to much that is useful
     
  9. Sep 2, 2009 #8
    Ok, thanks, that answered my question.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Mathematical impact of outliers on accuracy of models
  1. Help on outlier test (Replies: 1)

  2. Outlier detection (Replies: 3)

Loading...