Intraclass Correlation Coefficient (ICC) the appropriate method?

Click For Summary
SUMMARY

The discussion centers on determining the reliability of a single measurement instrument using the Intraclass Correlation Coefficient (ICC). The user is considering using ICC(3,10) to average 10 repeated measures for each individual but is confused about the appropriate specification for single measures. They also contemplate fitting a random effects model to calculate the ICC directly, questioning whether this approach is equivalent to using SPSS for ICC computation. Ultimately, the user seeks clarity on the most suitable method for assessing test-retest reliability in their specific context.

PREREQUISITES
  • Understanding of Intraclass Correlation Coefficient (ICC) and its types (e.g., ICC(3,10), ICC(3,1))
  • Familiarity with SPSS for statistical analysis
  • Basic knowledge of random effects models in statistics
  • Concepts of test-retest reliability and repeated measures
NEXT STEPS
  • Research the differences between ICC(3,10) and ICC(3,1) in the context of repeated measures
  • Explore how to implement random effects models using statistical software like R or Python
  • Learn about multilevel modeling techniques and their application in reliability analysis
  • Investigate methods for calculating confidence intervals for ICC estimates
USEFUL FOR

Statisticians, researchers in psychology or health sciences, and anyone involved in reliability testing of measurement instruments.

SpringPhysics
Messages
104
Reaction score
0

Homework Statement


I wish to determine whether a SINGLE instrument provides reliable measures (test-retest reliability). This single instrument is not random (it's the only one available), and I will take 10 measurements for each individual that is part of a control group (and then again for each individuals that is part of the treatment group).


Homework Equations





The Attempt at a Solution


I am reading a lot about intraclass correlation coefficient (ICC) but they all appear to require several different raters in packages such as SPSS. I am confused about whether I should use the ICC computed using SPSS or if I should fit a random effects model to directly calculate the ICC (or maybe they're the same thing?).

For the ICC, I was thinking of using ICC(3,10) by averaging the 10 observations. However, I am baffled as to how to specify ICC when I want to look at single measures - the documentation I've found for ICC(3,1) doesn't look like it allows for repeated measures for individuals.

For the random effects model, I would have 10 repeated measures for each individual, each as a linear combination of the group mean, the individual mean, plus an error that depends on the instrument. However, there may be interaction with the group, individual, and instrument so this may turn into a multilevel model. In that case, I'm not sure how reliability can be determined, and whether I can obtain confidence intervals for that.

Please advise on which method would be more appropriate. Thanks!
 
Physics news on Phys.org
Sorry for double-posting. The problem has been solved.
 

Similar threads

Negative amplitude of cross-correlation function in ICCS?
  • · Replies 6 ·
Replies
6
Views
2K
Graduate Assessing Interclass Correlation in a Completely Random One Way Design
  • · Replies 3 ·
Replies
3
Views
3K
Undergrad Coefficient sign flip in linear regression with correlated predictors
  • · Replies 13 ·
Replies
13
Views
4K
Simplifying correlation coefficient
  • · Replies 2 ·
Replies
2
Views
1K
Undergrad What is the correct definition of measurement uncertainty?
  • · Replies 21 ·
Replies
21
Views
3K
Graduate Are photons really anti-correlated off their basis vectors?
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Minimal vs Instrumental vs Ensemble
  • · Replies 84 ·
3
Replies
84
Views
7K
Undergrad Non-locality by memory (Jung, 2020) applied to stochastic mechanics
  • · Replies 5 ·
Replies
5
Views
2K
Graduate Serial correlation coefficient (random data)
  • · Replies 7 ·
Replies
7
Views
4K
Graduate Calculating Correlations and Sensitivity Coefficients
  • · Replies 1 ·
Replies
1
Views
1K