Q - test: How can I find the Q critical value at 95% Confidence?

  • Context: High School 
  • Thread starter Thread starter Another
  • Start date Start date
  • Tags Tags
    Test Value
Click For Summary
SUMMARY

The discussion centers on the use of the Q-test for outlier detection, specifically questioning its applicability for larger datasets, such as one with 180 observations. The consensus is that the Q-test is intended for small sample sizes and should not be used more than once to reject observations. Instead, graphical methods like box plots are recommended for larger datasets to identify potential outliers, which should then be investigated for transcription errors or experimental flaws before making any rejection decisions.

PREREQUISITES
  • Understanding of the Q-test for outlier detection
  • Familiarity with box plots and their application in statistical analysis
  • Knowledge of data integrity issues, such as transcription errors and experimental glitches
  • Basic statistical concepts, including normality assumptions
NEXT STEPS
  • Research the limitations of the Q-test for outlier detection in large datasets
  • Learn how to create and interpret box plots for identifying outliers
  • Explore methods for investigating outliers, including data validation techniques
  • Study alternative outlier detection methods suitable for large sample sizes
USEFUL FOR

Statisticians, data analysts, researchers, and anyone involved in data quality assessment and outlier detection in large datasets.

Another
Messages
104
Reaction score
5
TL;DR
I want to use Q -test to truncate the data. But I have number of data 180 ( n = 180 )

How can i find Q critical value at 95 % Confidence ,When number of data equal to 180 ?
I want to use Q -test to truncate the data. But I have number of data 180 ( n = 180 )
How can i find Q critical value at 95 % Confidence ,When number of data equal to 180 ?
 
Physics news on Phys.org
I fundamentally disagree with the Q test and similar outlier rejection techniques. I tend to use graphical methods to visually assess possible outliers. For any possible outlier I look at the data and see if there was a transcription error or a recorded experimental "glitch" or flaw. If there is a transcription error then I correct it, and if there was a recorded "glitch" then I reject it. I never reject data simply based on distributional assumptions.

In particular, the Q test assumes normality. An outlier can tell you that your data is not normal. If you assume normality anyway and reject the outlier then you are ignoring important information telling you that your assumption is wrong.

That said, the Q test is only intended for small numbers of observations. 180 is too many. You should pick a different test (or use a different approach alltogether).
 
  • Informative
  • Like
Likes   Reactions: Klystron and hutchphd
Dale said:
I fundamentally disagree with the Q test and similar outlier rejection techniques. I tend to use graphical methods to visually assess possible outliers. For any possible outlier I look at the data and see if there was a transcription error or a recorded experimental "glitch" or flaw. If there is a transcription error then I correct it, and if there was a recorded "glitch" then I reject it. I never reject data simply based on distributional assumptions.

In particular, the Q test assumes normality. An outlier can tell you that your data is not normal. If you assume normality anyway and reject the outlier then you are ignoring important information telling you that your assumption is wrong.

That said, the Q test is only intended for small numbers of observations. 180 is too many. You should pick a different test (or use a different approach alltogether).

What do you think, if I try this program?
https://miniwebtool.com/outlier-calculator/
 
I use something similar in my graphical methods. One of the graphical methods I use is a box and whisker plot which uses this calculation to determine if any data points should be plotted individually. I can then look at those individually plotted points and see if there is a transcription error or an experimental reason to reject the data. So I do use that method, not to directly reject data, but to spot data that I should look at in more detail.
 
Dale said:
I use something similar in my graphical methods. One of the graphical methods I use is a box and whisker plot which uses this calculation to determine if any data points should be plotted individually. I can then look at those individually plotted points and see if there is a transcription error or an experimental reason to reject the data. So I do use that method, not to directly reject data, but to spot data that I should look at in more detail.
When number of sample is small , we can use Q - test to select outliers data. If we find that there is still outlier data , we can use Q-test to confirm reject this data again.

When number of sample is large , I want to use Box Plot method to select outliers data.
If I find that there is still outlier data , I can use Box Plot method again ?
 
Another said:
When number of sample is small , we can use Q - test to select outliers data. If we find that there is still outlier data , we can use Q-test to confirm reject this data again.
No. The Q test should never be used more than once and is only used to reject at most one observation.

Another said:
When number of sample is large , I want to use Box Plot method to select outliers data.
If I find that there is still outlier data , I can use Box Plot method again ?
You can use Box Plots as often as you like, but I would not reject any data based only on the Box Plot itself. All that should do is identify data points to investigate. If you investigate and find that there is no transcription error and no experimental problem, then you should keep the outlier.
 
Last edited:

Similar threads

  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 7 ·
Replies
7
Views
2K
Replies
6
Views
3K
  • · Replies 5 ·
Replies
5
Views
4K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 6 ·
Replies
6
Views
1K
  • · Replies 22 ·
Replies
22
Views
4K
  • · Replies 10 ·
Replies
10
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 7 ·
Replies
7
Views
3K