Intraclass Correlation Coefficient (ICC) the appropriate method?

In summary, the person is seeking a method to determine the test-retest reliability of a single instrument with 10 measurements for each individual in a control and treatment group. They are considering using the intraclass correlation coefficient (ICC) or a random effects model, but are unsure of which would be more appropriate and how to specify the ICC for single measures.
  • #1
SpringPhysics
107
0

Homework Statement


I wish to determine whether a SINGLE instrument provides reliable measures (test-retest reliability). This single instrument is not random (it's the only one available), and I will take 10 measurements for each individual that is part of a control group (and then again for each individuals that is part of the treatment group).


Homework Equations





The Attempt at a Solution


I am reading a lot about intraclass correlation coefficient (ICC) but they all appear to require several different raters in packages such as SPSS. I am confused about whether I should use the ICC computed using SPSS or if I should fit a random effects model to directly calculate the ICC (or maybe they're the same thing?).

For the ICC, I was thinking of using ICC(3,10) by averaging the 10 observations. However, I am baffled as to how to specify ICC when I want to look at single measures - the documentation I've found for ICC(3,1) doesn't look like it allows for repeated measures for individuals.

For the random effects model, I would have 10 repeated measures for each individual, each as a linear combination of the group mean, the individual mean, plus an error that depends on the instrument. However, there may be interaction with the group, individual, and instrument so this may turn into a multilevel model. In that case, I'm not sure how reliability can be determined, and whether I can obtain confidence intervals for that.

Please advise on which method would be more appropriate. Thanks!
 
Physics news on Phys.org
  • #2
Sorry for double-posting. The problem has been solved.
 

1. What is the purpose of using Intraclass Correlation Coefficient (ICC) for data analysis?

The purpose of using ICC is to measure the degree of similarity between data points within the same group or cluster. It is commonly used in the field of social and behavioral sciences to assess the reliability and consistency of data collected from multiple raters or measurements.

2. How is ICC different from other correlation coefficients?

ICC is specifically used for assessing the agreement or consistency of data within a group, while other correlation coefficients such as Pearson's correlation coefficient measure the strength of linear relationship between two variables.

3. What is the appropriate method for calculating ICC?

The appropriate method for calculating ICC depends on the research design and the type of data being analyzed. For example, if the data is continuous and the raters are fixed, a two-way mixed-effects model would be appropriate. If the data is binary or ordinal, a binary logistic regression model can be used.

4. What is an acceptable range for ICC values?

The acceptable range for ICC values varies depending on the field of study and the purpose of the analysis. In general, an ICC value greater than 0.75 indicates excellent agreement, 0.40-0.75 indicates fair to good agreement, and less than 0.40 indicates poor agreement.

5. Can ICC be used for both interrater and intrarater reliability?

Yes, ICC can be used to assess the reliability of data between multiple raters (interrater reliability) or data collected by the same rater at different time points (intrarater reliability). However, the specific ICC formula used may differ depending on the type of reliability being assessed.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
13
Views
2K
Replies
6
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
2K
  • Calculus and Beyond Homework Help
Replies
1
Views
1K
  • Introductory Physics Homework Help
Replies
2
Views
295
  • Calculus and Beyond Homework Help
Replies
2
Views
948
  • Set Theory, Logic, Probability, Statistics
Replies
21
Views
2K
  • Quantum Interpretations and Foundations
3
Replies
84
Views
2K
  • Quantum Physics
3
Replies
76
Views
6K
Back
Top