We have 18 subjects chosen at random, and they are rated by 6 randomly selected raters, and there is no requirement that the same rater rate all the subjects, we have a completely random one way design. Reliability is assessed with a ICC(1,1) or ICC(1) model depending on convention. The table looks like this A B C D E F (judges) 1 2 3 . 18 (subjects) Each judge rates 3 different subject out 18 only. A question: if there were 10 timeslots available for the subjects was it really necessary to have 6 judges? I am not sure what the question aks. In particular the part about the timeslots. Is the question asking to rate the subjects in repeated measures?