Assessing Interclass Correlation in a Completely Random One Way Design

[_joey]

We have 18 subjects chosen at random, and they are rated by 6 randomly selected raters, and there is no requirement that the same rater rate all the subjects, we have a completely random one way design. Reliability is assessed with a ICC(1,1) or ICC(1) model depending on convention.
The table looks like this

A B C D E F (judges)
1
2
3
.
18
(subjects)

Each judge rates 3 different subject out 18 only.

A question: if there were 10 timeslots available for the subjects was it really necessary to have 6 judges?

I am not sure what the question aks. In particular the part about the timeslots. Is the question asking to rate the subjects in repeated measures?

 

[chiro]

Hey _joey.

I'm also not sure what you are asking. Are you trying to either guarantee some variance requirement or some requirement that each subject gets selected with some minimum probability?

 

[_joey]

The question asks to assess the agreement between various pairs of raters rating subjects (let's say examination papers) and then it talks about 10 timeslots for this exam papers. No idea at all about the timeslot. There is probably some trick to it.

 

[_joey]

Here is the question
http://imageshack.us/photo/my-images/824/questioni.png/

There are 3 ratings per paper in the dataset rated by different judges

I am not sure about part(c)

Thanks!