What are the recommended tests for comparing two sets of data?

  • Context: Undergrad 
  • Thread starter Thread starter VVS2000
  • Start date Start date
  • Tags Tags
    Testing
Click For Summary

Discussion Overview

The discussion revolves around the appropriate statistical tests for comparing two sets of data: experimental values and expected values, specifically in the context of measuring differences in focal lengths of a lens. Participants explore various methods to quantitatively assess how different these two datasets are, considering uncertainties and errors in the observed data.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • Some participants suggest using the chi-squared test and ANOVA to compare the two sets of data, questioning whether these tests are sufficient.
  • One participant proposes that the sum of square residuals is a standard measure of distance, but expresses uncertainty about the meaning of the original question.
  • Another participant clarifies that the question of how different the data is from expected values is vague and suggests that the chi-squared goodness of fit test is appropriate for assessing probability fit, while the sum-squared-errors total is suitable for measuring numerical differences.
  • There is a clarification that the data consists of one set of observed values and one set of expected values, with a focus on how to account for uncertainties in the measurements.
  • One participant inquires whether the expected values have associated uncertainties, which is confirmed to be the case, albeit with uniform uncertainty across values.
  • Another participant mentions that chi-squared goodness of fit and linear regression are two possible methods, noting that the former requires categorization of data and the latter assumes a linear relationship.

Areas of Agreement / Disagreement

Participants express differing views on the appropriateness of various statistical tests and the clarity of the original question. There is no consensus on a single recommended method, and multiple competing approaches are discussed.

Contextual Notes

The discussion highlights limitations in the clarity of the initial question and the assumptions regarding the nature of the datasets. The dependence on the definition of uncertainty and the conditions under which different tests apply is also noted.

Who May Find This Useful

This discussion may be useful for researchers or students involved in experimental physics or data analysis, particularly those interested in statistical methods for comparing observed and theoretical data.

VVS2000
Messages
150
Reaction score
17
So I have two columns of data, One containing experimental values and the other having expected values. So I read that chi-squared test and Anova Tests can be used to compare two set of data. My main aim is to quantitatively know how different these two sets of data are, so are these two tests enough or is there any other tests that you would suggest?
 
Physics news on Phys.org
The standard measure of distance would be the sum of square residuals. However, I am not sure what you are asking has real meaning as stated.

As I understand your statement, you don’t have two sets of data, you have one set of data and a model. Any set of data can plausibly come from any model given sufficiently uncertain measurements. So just asking about the sum of square residuals doesn’t say much.

If your model has any free parameters then you can meaningfully ask which parameters minimize the residuals. Or if you have two competing models you can ask which model minimizes the residuals. Or if you can predict the expected uncertainty in your measurements, then you can determine how likely the data is under the model.
 
  • Like
Likes   Reactions: WWGD, VVS2000 and FactChecker
Asking "how different" your data is from the expected values of a model is a vague question.
Are you asking how likely (probability) such data might fit the model like that (or worse)? The Chi-squared goodness of fit test is well suited for that.
Are you asking for some numerical measure of how great the differences are? The sum-squared-errors total is well suited for that.
 
  • Like
Likes   Reactions: VVS2000
Dale said:
The standard measure of distance would be the sum of square residuals. However, I am not sure what you are asking has real meaning as stated.

As I understand your statement, you don’t have two sets of data, you have one set of data and a model. Any set of data can plausibly come from any model given sufficiently uncertain measurements. So just asking about the sum of square residuals doesn’t say much.

If your model has any free parameters then you can meaningfully ask which parameters minimize the residuals. Or if you have two competing models you can ask which model minimizes the residuals. Or if you can predict the expected uncertainty in your measurements, then you can determine how likely the data is under the model.
ok sorry for not being clear. I have two sets of data. One column contains Observed or experimental focal length of a lens at different heights from the axis. The Other column contains the expected or theoretical value of focal length. so yeah I want to quantitatively know different these two sets of data are given the uncertainty and errors in the observed data
 
VVS2000 said:
The Other column contains the expected or theoretical value of focal length
Does this column have uncertainty associated with it?
 
  • Like
Likes   Reactions: WWGD
Dale said:
Does this column have uncertainty associated with it?
yes, but all values have the same uncertainty
 
Two possibilities are Chi-Squared goodness of fit, and linear regression. The first one is very general. It requires that the data be combined into categories. The second one requires that the model of the variable of interest is a linear function of the other variables.
 

Similar threads

Graduate How to set up and interpret Chi^2 test results for my data?
  • · Replies 11 ·
Replies
11
Views
3K
Undergrad Hypothesis Testing: Comparing Gaussian Distributions
  • · Replies 3 ·
Replies
3
Views
2K
Graduate Defining a Symmetry Statistic to test for Normality?
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Akaike Information Criterion Vs Likelihood Ratio Test
  • · Replies 5 ·
Replies
5
Views
4K
High School Chi-squared test stat choosing a critical value
  • · Replies 5 ·
Replies
5
Views
4K
Undergrad Does Returning Marbles After Each Draw Affect Chi-square Test Results?
  • · Replies 8 ·
Replies
8
Views
2K
Graduate Chi squared test for data with error
  • · Replies 7 ·
Replies
7
Views
2K
Graduate Testing PRNG w/ Chi-Squared & Kolmogorov-Smirnov Tests
  • · Replies 3 ·
Replies
3
Views
2K
Graduate Factor Analysis for scaling a test with pre-post data
  • · Replies 2 ·
Replies
2
Views
4K
Undergrad Error bars in Excel (standard deviation, standard error, percentage)
  • · Replies 4 ·
Replies
4
Views
3K