I wish to determine whether a SINGLE instrument provides reliable measures (test-retest reliability). This single instrument is not random (it's the only one available), and I will take 10 measurements for each individual that is part of a control group (and then again for each individuals that is part of the treatment group).
The Attempt at a Solution
I am reading a lot about intraclass correlation coefficient (ICC) but they all appear to require several different raters in packages such as SPSS. I am confused about whether I should use the ICC computed using SPSS or if I should fit a random effects model to directly calculate the ICC (or maybe they're the same thing?).
For the ICC, I was thinking of using ICC(3,10) by averaging the 10 observations. However, I am baffled as to how to specify ICC when I want to look at single measures - the documentation I've found for ICC(3,1) doesn't look like it allows for repeated measures for individuals.
For the random effects model, I would have 10 repeated measures for each individual, each as a linear combination of the group mean, the individual mean, plus an error that depends on the instrument. However, there may be interaction with the group, individual, and instrument so this may turn into a multilevel model. In that case, I'm not sure how reliability can be determined, and whether I can obtain confidence intervals for that.
Please advise on which method would be more appropriate. Thanks!