When are 2 models comparable using an F Test?

[David Laz]

Say I'm given a bunch of models on the same set of data how does one determine whether a valid comparison can be made between any two of them using an F Test?

Is it that the estimation space of one has to be a subset of the other? Is there any easier, more practical way of determining this?

Thanks

 

[EnumaElish]

Is it that the estimation space of one has to be a subset of the other?

No. For example, in a regression equation, you might want to test whether b_i = b_j. In this case, the relevant test statistic has an F distribution, although the hypothesis does not involve a subset relationship.

http://en.wikipedia.org/wiki/F_test

 

[David Laz]

I think I know what your getting at. But here's an example of the sort of question I need help with. Maybe you could explain/show me.

heres a question from an old exam:

The following table gives the yields from a field experiment on two varieties of wheat, Hard and Common, with four equally spaced levels of applied fertilizer. The plots were allocated at random to the various treatment combinations. Initially it was planned to have four replicates at each combination, but errors in applying the fertilizer reduced the final sample size.


http://img530.imageshack.us/img530/2558/statny0.jpg
(table of the data, probably not need to answer this question)

In what follows Var.f refers to variety treated as a factor with two levels. (1=hard, 2 = common) and Fert.f refers to fertilizer treated as a factor with 4 levels (1,2,3,4)

Since fertilizers are numeric it is possible to use the actual amount of fertilizer as a variable (denoted by x, taking values of 1,2,3,4)

http://img174.imageshack.us/img174/8470/stat2nc9.jpg
(table of different models and their associated deviance and df's)

Where a*b means a + b + a:b (interaction term)
Among all the models which cannot be validly compared using an F-Test?

 

[EnumaElish]

To calculate the F statistic, you need a restricted model and an unrestricted model. The restricted model is a sub-type of the unrestricted model, in the sense that but for the restriction(s) being applied, it would have been identical to the unrestricted model. Put differently, starting from the unrestricted model, you should be able to arrive at the restricted model by imposing one or more linear restrictions on the set of parameters that the model is to estimate. Example: a Var.f + b Fert.f can be obtained from a Var.f + b Fert.f + c Var.f:Fert.f by imposing the linear restriction c = 0. (Each of a, b, c is an estimated parameter that would explain, say, "agricultural yield" as a function of the variable Var.f, Fert.f, Var.f:Fert.f in respective order.)